This is my question about a very common problem faced while training several data science and AI algorithms, and most importantly while backpropogating errors in neural networks, which is getting trapped in a local minima while descending gradient.

So, according to the discussion under the qn, it is claimed to be off-topic

However, in the defence of my post, I think it is perfectly on-topic in this site, as it asks about a legit problem faced while training neural nets and several other AI algorithms.

So, I am looking forward to what the community thinks regarding this.

| |

The question is off-topic, as it's about how to the use of machine learning algorithms. (the other questions on neural nets, their architectures, backpropogation, are also off-topic).

Programming, algorithm, modeling, math, philosophy, and history questions should the off-topic, as they are already on-topic in other SE, such as Stats and Data Science.

Data science and the Stats SE already have a huge overlap (>~80%), and I am worried to have a third SE that also significantly overlaps with them. Personally, it would further demotivate me to write any answer, as it gets tiring to copy-paste content, and updating duplicated answers is a pain.

| |
  • $\begingroup$ Thanks for clarifying, Frank. So, if questions regarding Gradient Descent and Monte Carlo search which are one of the most imp. algos. of neural nets and in understanding the Alpha Go bot respectively, I don't really get the scope of the site :( $\endgroup$ – Dawny33 Aug 6 '16 at 2:44
  • $\begingroup$ I personally believe that the (entirely healthy) difference in perspectives on this is indicative of a deeper issue in AI: many areas that were historically AI but are now claimed to be 'solved problems' are in fact not. See my answer to the question about OCR on the main site for more on this. One way of clarifying the scope of this site might usefully be any area where heuristics are still required - this would avoid artificial demarcations by technique. $\endgroup$ – NietzscheanAI Aug 6 '16 at 8:29
  • $\begingroup$ "Programming, algorithm, modeling, math, philosophy, and history questions should the off-topic, as they are already on-topic in other SE" So... what's left for us? I'm yet to to see a question about AI that is NOT on-topic on ANY other SE site. $\endgroup$ – rcpinto Aug 6 '16 at 19:42

Escaping local optima is an extremely ubiquitous problem (in case it's unclear - there are vastly more applications than backprop), leading to many open questions (a great deal of metaheuristics research, indisputably part of AI, is concerned with this).

So, it is much more open-ended (and therefore subject to heuristic/AI solutions) than the more pedestrian questions (with procedural anwers) about e.g. backprop that appear to be within the AI SE remit.

Hence, I'd say it is definitely on topic ;-)

| |
  • 1
    $\begingroup$ I'd say that those are exactly the reasons why it should NOT be here, but rather on Data Science. It's really more of a mathematical/optimization topic than AI. You wouldn't talk about differential equations on Physics.SE either, even though a lot of the subject involves them. $\endgroup$ – Disenchanted Lurker Aug 5 '16 at 12:50
  • 1
    $\begingroup$ @InquisitiveLurker - state space search has always been one of the pillars of AI. Optimization needn't be "of a numerical value" nor is it terribly strongly connected to data science as a discipline. $\endgroup$ – NietzscheanAI Aug 5 '16 at 12:53
  • 1
    $\begingroup$ @InquisitiveLurker please make your opinion an answer, so people can vote. Lets see which way the rest of the community leans $\endgroup$ – Harsh Aug 5 '16 at 14:27
  • $\begingroup$ I feel I am still not getting sufficient clarity on what is on-topic here vs. Data Science and Cross Validated SE sites. $\endgroup$ – edwinksl Aug 5 '16 at 15:08
  • $\begingroup$ @Harsh I've written it now. $\endgroup$ – Disenchanted Lurker Aug 6 '16 at 6:30

Personally, I consider gradient descent something akin to what something like differential equations is to physics - a useful piece of mathematics that has a large array of applications, but not really an AI topic by itself.

When we talk about AI, there are different levels of detail and "technicalness" we can go into and I believe it's necessary to draw the line somewhere.

To illustrate what I mean, let me use the example of self-driving cars:

  • There's the concept of the self-driving car itself
  • The car has some sort of computer-vision system
  • That system might involve a neural network
  • That network needs to be trained somehow - there are different algorithms for that
  • One of the most common ones is backpropagation
  • Backpropagation often uses gradient descent
  • Gradient descent is an optimization algorithm
  • and so on...

We could look at an AI problem at any of those levels. But at some point, it becomes no longer really about AI but rather about mathematics or statistics. And those topics are already covered well by other sites.

Basically, what I believe is that this site should mainly concentrate on the top few lines of that list, and leave the rest to more appropriate venues.

| |
  • 1
    $\begingroup$ A nice decomposition, but AI is arguably not so 'top down' in practice. At the lower levels, we often lack good heuristics. This then is an AI problem in it's own right. By all means, let's direct people to other sites when the best solution method is well understood, but let's not prevent discussion of a gamut of AI alternatives. Restricting the site to a top level 'systems' view would preclude discussion of some open research issues which might best not be pigeonholed as 'about mathematics or statistics'. $\endgroup$ – NietzscheanAI Aug 6 '16 at 6:51
  • $\begingroup$ @InquisitiveLurker Since you've explained technologies behind self-driving cars, you may want to answer this question: What technologies are needed for a self-driving car? $\endgroup$ – kenorb Aug 15 '16 at 19:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .