Which research-related questions are on-topic here?

Question

We have already had questions about how researchers do certain things. For instance, this one How do AI researchers imagine higher dimensions?. However, on our on-topic page, we haven't yet clarified which questions related to research in AI are on-topic here. In fact, someone had complained about why that linked post had not been closed as off-topic, as it's only superficially related to AI. So, which research-related questions should be allowed on our site?

This is a more general question than Is it okay to ask questions related to the practices of researchers in the field? that aims to define the on-topic page more clearly, so that mods can also take decisions (when it comes to closing or not a post as off-topic) without hesitation.

Answer 1

I think it depends on the question to some degree.

There are some questions that have objective answers via, e.g. surveys of AI researchers. For example, the question of when AI researchers believe "Hard AI" will be created is objectively answerable via this method.

I think in general though, questions like the one in that meta-post actually belong on academia.se, a site where you could reasonably expect to get a broad range of opinions from researchers even in a relatively narrow area, and where questions like "How do I get the most out of open sourced code in research papers?" would surely be on-topic.

In general, I think we should adopt the following heuristic:

• If the question is about the opinions of AI researchers about something in AI, it is on topic here and potentially answerable, though we should require quality sources in such answers. Example: "Does the AI research community have a consensus view on whether deep neural networks can be made explainable?".
• If the question is about the practices of AI researchers, or about their opinions about research practices, it is off-topic here and belongs on the academia.se stack. Example: "How does one become an AI researcher?" or "How do AI researchers use the code provided with some papers?".

A grey area is questions about AI-specific research methodologies. (e.g. "What is the right hypothesis test to use when comparing machine learning methods?"). I would say these could potentially be on topic on many sites, and we should generally be permissive about them.