Stochastic gradient descent

Also: SGD

Gradient descent using one small random batch at a time rather than the whole dataset, so a step costs almost nothing and there are millions of them. The noise it introduces turns out to help generalisation.

A definition is not an understanding. This is taught properly in Gradient descent, momentum and Adam from scratch, part of Calculus for ML.

Learn this properly

Terms from the same topic

Gradient descent Adam Gradient clipping Gradient Partial derivative Chain rule