Asked about Fine-tuning a pretrained network, in code

What is a per-layer learning rate, and when is it worth the trouble?

An answer that works

You give each layer its own rate, smallest at the input and rising towards the head, usually by a fixed multiplier per block. Early layers hold general features that should barely move, later ones hold task-specific features that should. It lets you unfreeze the whole network at once instead of in stages, and on a small dataset it usually beats staged unfreezing for less wall-clock time.

Reading an answer is not being able to give one. The section that teaches this is Fine-tuning a pretrained network, in code, 5 hours of Transfer learning.

Learn this properly

Also asked about Transfer learning

Why does transfer learning work?Why use a lower learning rate when fine-tuning?When would transfer learning not help?You have 2,000 labelled images for a new classification task. What is your approach?