Asked about Starting from someone else's training run
Why use a lower learning rate when fine-tuning?
An answer that works
The randomly initialised head produces large early gradients, and at a normal learning rate those destroy the pretrained features within a few batches. Train the head first with the rest frozen, then unfreeze at a much lower rate.
Reading an answer is not being able to give one. The section that teaches this is Starting from someone else's training run, 6 hours of Transfer learning.
Learn this properly