Asked about The shape of the loss surface
What does momentum do?
An answer that works
It accumulates a running average of past gradients, so consistent directions gain speed and oscillations cancel. That carries the optimiser across saddle points and plateaus and damps zigzagging in narrow valleys.
Reading an answer is not being able to give one. The section that teaches this is The shape of the loss surface, 4 hours of Calculus for ML.
Learn this properlyAlso asked about Calculus for ML
What does the learning rate actually do, and how do you tell it is wrong?What is a gradient?Why do gradients vanish in deep networks?Your loss becomes NaN during training. What do you check?Are local minima the main problem in training deep networks?What is the gradient of cross-entropy loss with softmax, with respect to the logits?Explain the Adam update.