Transfer learning interview questions
5 questions across 2 sections, in the order you would learn them. Every answer here is one that works — but an answer you have read is not an answer you can give, so use Test yourself and say it out loud first.
Starting from someone else's training run
Read the section that teaches thisIf this comes up in an interview
Why does transfer learning work?
Early layers learn general features of the medium rather than the task, so they transfer. Only the later, specialised layers need replacing, which is why a new head plus a small amount of data gets most of the way.
Why use a lower learning rate when fine-tuning?
The randomly initialised head produces large early gradients, and at a normal learning rate those destroy the pretrained features within a few batches. Train the head first with the rest frozen, then unfreeze at a much lower rate.
When would transfer learning not help?
When your data is far from the pretraining distribution, such as medical scans or spectrograms against a model trained on everyday photographs. The test is empirical: if the frozen version performs near chance, the features are not transferring.
Fine-tuning a pretrained network, in code
Read the section that teaches thisIf this comes up in an interview
You have 2,000 labelled images for a new classification task. What is your approach?
Start from a pretrained model with the matching preprocessing. First a linear probe on frozen features as a baseline. Then train a new head, and if needed unfreeze the last blocks with a lower learning rate and strong augmentation. Compare everything on the same held-out set.
Then they push further
Fine-tuning barely beats the linear probe. What does that tell you?
The pretrained features already capture what the task needs, and the extra complexity is not earning much. Ship the probe, or check whether more data or better labels would help more than more fine-tuning.
Your images are X-rays, very unlike the photos the model was trained on. Does transfer still work?
Usually it still helps, since low-level features transfer, but less. Unfreeze more layers, consider a model pretrained on medical images if one exists, and compare against training from scratch if you have enough data.
What gets marked up. Marked up for a cheap baseline first, staged unfreezing with lower learning rates, and awareness of domain distance.
What is a per-layer learning rate, and when is it worth the trouble?
You give each layer its own rate, smallest at the input and rising towards the head, usually by a fixed multiplier per block. Early layers hold general features that should barely move, later ones hold task-specific features that should. It lets you unfreeze the whole network at once instead of in stages, and on a small dataset it usually beats staged unfreezing for less wall-clock time.
These are the questions for one topic. The whole question bank covers every topic on the site.