Asked about Learning rate schedules and a tuning workflow
Why use learning rate warmup?
An answer that works
Early in training the weights and the optimiser's running statistics are poorly set, so a full learning rate can cause divergence. Ramping up over the first steps avoids that, especially for large models and large batches.
Reading an answer is not being able to give one. The section that teaches this is Learning rate schedules and a tuning workflow, 4 hours of Hyperparameter tuning.
Learn this properly