Asked about The shape of the loss surface
Are local minima the main problem in training deep networks?
An answer that works
Mostly not. In high dimensions, most flat points are saddle points, where some directions still go downhill. Saddle points and plateaus slow training far more often than true local minima trap it.
Reading an answer is not being able to give one. The section that teaches this is The shape of the loss surface, 4 hours of Calculus for ML.
Learn this properlyAlso asked about Calculus for ML
What does the learning rate actually do, and how do you tell it is wrong?What is a gradient?Why do gradients vanish in deep networks?Your loss becomes NaN during training. What do you check?What does momentum do?What is the gradient of cross-entropy loss with softmax, with respect to the logits?Explain the Adam update.