Asked about Derivatives and the chain rule
What is a gradient?
An answer that works
The vector of partial derivatives of the loss with respect to every parameter. It points in the direction of steepest increase, so training steps against it to reduce the loss.
Reading an answer is not being able to give one. The section that teaches this is Derivatives and the chain rule, 5 hours of Calculus for ML.
Learn this properlyAlso asked about Calculus for ML
What does the learning rate actually do, and how do you tell it is wrong?Why do gradients vanish in deep networks?Your loss becomes NaN during training. What do you check?Are local minima the main problem in training deep networks?What does momentum do?What is the gradient of cross-entropy loss with softmax, with respect to the logits?Explain the Adam update.