Asked about Learning rate schedules and a tuning workflow
You have budget for 30 training runs. How do you tune?
An answer that works
One or two runs to confirm the baseline trains, a learning rate range test, then a random or Bayesian search over learning rate and regularisation on log scales with early pruning. Keep a few runs to rerun the top settings with different seeds, since the difference between the top two is often noise.
Then they push further
Why random search rather than a grid?
Usually only one or two hyperparameters matter. A grid wastes most runs repeating the same values of the important one, while random search tries a new value of every parameter on every run.
The best run beats the second best by 0.3%. Which do you pick?
Rerun both with several seeds first. If the gap is inside the seed-to-seed spread, choose on other grounds, such as the simpler or cheaper setting.
What gets marked up
Marked up for prioritising the learning rate, using log scales, pruning, and checking seed variance.
Reading an answer is not being able to give one. The section that teaches this is Learning rate schedules and a tuning workflow, 4 hours of Hyperparameter tuning.
Learn this properly