Asked about Tuning, in order of what matters
Which hyperparameter would you tune first?
An answer that works
Learning rate. It has more effect than the rest combined. Search it by orders of magnitude first, then refine, and only then move to schedule, batch size and regularisation.
Reading an answer is not being able to give one. The section that teaches this is Tuning, in order of what matters, 6 hours of Hyperparameter tuning.
Learn this properly