I was quite surprised to see this question migrated from AI Stack Exchange to Data Science:

There are two reasons that I am surprised:

  • In my opinion, Reinforcement Learning is not really a data science or statistics subject. Some of the toolkit is the same (mainly neural networks), but the resulting system is different.

  • In my opinion, the AI stack exchange is where I would expect to see practical discussions of agents that learn how to behave rationally in environments. This encompasses RL and other approaches to creating behaviours or policies, such as NEAT.

In fact I have just encouraged a recent poster with a practical RL question on Cross Validated (why is a Q learning on Towers of Hanoi not working) to post here . . . should I have done? Do we want that kind of question?

I would like to open a discussion:

What makes a question which is obviously about Reinforcement Learning on or off topic at AI Stack Exchange?

Some thoughts:

  • Are questions about implementing RL algorithms with code snippets on-topic here (assuming code problems are not trivial such as Python syntax errors)?

    • The example question that was migrated to DataScience is in this category.
  • Are questions about theory of maths behind RL on topic here, such as understanding proof of the Policy Improvement Theory or deriving Policy Gradients?

    • The maths of RL is easily as complex as anything discussed on CrossValidated. In fact Cross Validated already has many RL questions about these topics. Are they really statistics questions though - should they in fact be migrated here?

My personal opinion is that both kinds of questions should be on-topic here. In fact I am hard pushed to come up with an RL question which would not be on topic. That doesn't prevent some subset of RL questions being on-topic elsewhere too. But here is where I would expect them all to be on-topic. That is not to say that I would expect them all to remain open or get answers - some might be low quality or not answerable for other reasons.

But is my opinion out of step with others on the site? Have I made some incorrect assumption about the scope of this stack?

| |

The migration of this question to datascience seems really strange to me. Like you said, RL really is pretty much the furthest removed from data science out of all Machine Learning topics, even if it were off-topic on AI for whatever reason, it certainly wouldn't be on-topic on Data Science.

To address specifically the question in the title, the I'd say pretty much any Reinforcement Learning question is on-topic on AI.se (AI certainly seems a better fit for RL questions than either stats.se or datascience.se), except maybe questions that are 100% clearly about programming issues/bugs. For example, a question like "My RL algorithm is crashing, here is the stack trace, what's wrong?" Such questions might be a better fit on StackOverflow (still not datascience).

This particular question that got migrated might fit that description... but it's not certain. The question-asker is not certain if it's a bug in their code, or if there is some issue in choice of algorithm for this particular environment or something along those lines. In my opinion, whenever there is that level of uncertainty, the question is likely to require expertise in AI (specifically, in RL), not just programming expertise because it might not be just a programming issue. That makes, in my opinion, AI.se a better fit than StackOverflow.se (or any other site).

| |
  • $\begingroup$ Let us take a look into todays questions from Datascience datascience.stackexchange.com 1) LSTM error, 2) recommendation system 3) non-linear machine learning 4) comment classifier 5) QA system. If we take all these question back, in SE.AI the workload will be huge. I think these questions are very good suited at Datascience and migrating the “DQN can't learn or converge” to Datascience was a perfect decision. $\endgroup$ – Manuel Rodriguez Oct 8 '18 at 9:30
  • 4
    $\begingroup$ @ManuelRodriguez A "huge workload" on AI.se from such questions would be good, not bad. We want the site to grow. Increased activity (as long as it is on-topic, which questions on those topics you listed likely would be) is a positive, not a negative. Apart from that... none of those topics are RL, this meta question is about RL, so I'm not quite sure I get what your point is. $\endgroup$ – Dennis Soemers Oct 8 '18 at 10:11
  • $\begingroup$ An Inclusionist admin might argue, that it is a good idea to import all 12530 Question from Datascience to SE.AI because this helps to grow the website. The better approach is to decide on a quality base and resist against keeping problematic issue. $\endgroup$ – Manuel Rodriguez Oct 8 '18 at 10:23

I also think every RL question should be on-topic here. Funnily enough, I've been flagging posts for migration to DataScience or CrossValidated according to the help page that defines what is off-topic. But it seems like not everyone really abides by those definitions! I regularly see both implementation and mathematics questions here, related to RL and otherwise. I've stopped flagging these questions because I enjoyed reading and answering them.

So. If no one wants to abide by our current definition of 'on-topic' (including me), we should change it, right?

| |

I like the question “DQN can't learn or converge” very much. Because it shows the limits of a neural network. Somebody has trained a network, has done everything right with Python and OpenAI gym, gets the resulting errorplot of his model and doesn't understand why his agent is not improving anymore.

But in which forum fits this question, in AI.SE or somewhere else? I think, that any question can be migrated to datascience because this helps to reduce the load in SE.AI. And if the traffic is low we can shutdown the website as soon as possible. I'd like to a see a status message that AI.SE is no longer available because of technical problems.

| |
  • $\begingroup$ Deep RL is hard, and often doesn't work well. Which is why we can expect a of of people to ask about it, as despite this, it gets a large amount of positive press due to being sometimes able to do what a lot of other attempts at AI control systems have not been able to do in the past. Not sure if the sarcasm in the second half is great - it's hard to get behind it and upvote something that attacks the site, even if meant tongue-in-cheek. $\endgroup$ – Neil Slater Oct 7 '18 at 9:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .