Warmup
Starting a run at a tiny learning rate and raising it over the first few hundred or thousand steps. Early gradients are unreliable, and full-size steps taken against them can wreck a run before it starts.
A definition is not an understanding. This is taught properly in Learning rate schedules and a tuning workflow, part of Hyperparameter tuning.
Learn this properlyTerms from the same topic