Fine-tuning interview questions
13 questions across 3 sections, in the order you would learn them. Every answer here is one that works — but an answer you have read is not an answer you can give, so use Test yourself and say it out loud first.
Prompt, retrieve or fine-tune
Read the section that teaches thisIf this comes up in an interview
When would you fine-tune rather than use RAG?
When the need is consistent form rather than facts: a fixed output format, a domain style, or making a small model good enough at one narrow task to cut cost and latency. If the model is missing knowledge, that is retrieval, because fine-tuning does not reliably install facts and cannot be updated without retraining.
What is LoRA?
Parameter-efficient fine-tuning. The original weights are frozen and a small number of added parameters are trained, reaching most of the benefit at a fraction of the cost. The result is a small adapter that can be swapped per task against one base model.
What is the cost of fine-tuning beyond the training run?
A dataset you must build and keep current, redoing the work when you change base model, and a maintenance obligation a prompt does not have. That is why it sits last in the order after prompting, examples and retrieval.
What is distillation?
Training a smaller model to imitate a larger one's outputs. For LLM products it is frequently the largest cost reduction available, because a distilled small model can match the large one on one narrow task at a fraction of the price.
Running a fine-tune properly
Read the section that teaches thisIf this comes up in an interview
How much data do you need to fine-tune?
Depends on the goal: a few hundred clean examples for format or style, more for new behaviour. Quality and consistency matter more than volume, since the model learns exactly what the examples show.
Then they push further
How would you find out for your own task rather than guessing?
Train at three sizes — say an eighth, a half and all of what you have — and plot held-out accuracy against training set size. The curve rises fast, bends, then flattens, and where the bend is tells you whether more labelling is worth buying. It is a few hours of compute against what is often a quarter of labelling budget.
Why is training accuracy not a useful number here?
Because a model of any reasonable size fits a small training set exactly, whatever the size. At 25 examples and at 800 the training accuracy can both be 100 per cent while held-out accuracy goes from near zero to near perfect. Training loss is also not comparable across sizes, since a larger set is harder to fit, so it can go up as the model gets better.
Where do the examples come from?
Rarely from writing them by hand, which stalls at a few hundred. The two that work are a larger model's output filtered through your own evaluation, which is cheap and bootstraps the first version, and corrected production traffic, which is the version you keep. Each brings a defect worth naming: the distilled data inherits the teacher's mistakes, so the filter is not optional, and log-derived data covers only what the current system already handles well, so the failures you most need have to be sampled deliberately.
What is catastrophic forgetting?
Fine-tuning on a narrow task degrading the model's general abilities. LoRA reduces it by leaving the original weights untouched, and mixing general examples into training helps further. Test general capability as well as the target task.
Then they push further
How would you detect it, given that the training loss looks fine?
The training loss is computed on the fine-tuning data, where nothing is going wrong, so it cannot show this. You need a separate held-out set of the general behaviour you want to keep, scored the same way on every run, from the first run rather than added after somebody notices in production. A short fixed set of general instructions is enough.
What do you do about it, and what does each fix cost?
Two levers. Lowering the learning rate preserves more of the old behaviour and learns the new task correspondingly less well, because forgetting and learning are the same process. Replay — mixing examples of the old behaviour into the training set, typically a fifth to a half of it — is the better lever, because it buys back most of the old capability at a small cost on the new task. LoRA puts a ceiling on the damage, since a rank-8 or rank-16 update has nowhere near the capacity to overwrite general behaviour, and it gives you a rollback by simply not serving the adapter.
How do you know the fine-tune actually beat the prompt?
Score both on the identical held-out set with the identical grader in the same run, and keep the pairing. Report the disagreement counts rather than just the two accuracies: 78 against 71 means something different when the disagreements are 28 to 14 than when they are 70 to 63, and only the first is a result. A paired test such as McNemar uses only the examples the two systems disagreed on, which is where all the information about the difference is; throwing the pairing away and comparing two accuracies can turn a significant result into an interval that contains zero on the same data.
What gets marked up. Passable answers name a held-out set. Strong ones insist on the same examples for both systems, on the best prompt rather than a weak one as the baseline, and on reading the cases the prompt got right and the fine-tune got wrong before shipping.
You have five fine-tunes of one base model to serve. Merged or separate adapters?
Separate. Merging gives five full copies of the base weights, which for an 8B model at fp16 is about fifteen gigabytes each, against one shared base plus five adapters of a few tens of megabytes. Unmerged costs one extra low-rank matmul in the adapted layers, roughly one per cent of their work at r=16, and lets requests using different adapters share a batch if the server supports it. Merge only when you serve one adapter at high steady volume and want the last of the latency — and not at all if the base is quantized, because merging then requires re-quantizing and the rounding can undo part of what was trained.
A LoRA adapter from scratch
Read the section that teaches thisIf this comes up in an interview
Implement a LoRA layer, and explain what r and alpha do.
Keep the base weight frozen and add a parallel branch: x through A, which is (r, d_in), then through B, which is (d_out, r), scaled by alpha over r, added to the base output. A is small random, B is zero, so the adapter contributes nothing at initialisation and training starts from the base model exactly. r caps the rank of the update the adapter can represent, and so how large a behavioural change it can express. alpha is a fixed multiplier on that update, divided by r so that changing r does not require retuning the learning rate; it behaves like a learning rate for the adapter branch and changes nothing about what can be represented.
Then they push further
Why is B initialised to zero rather than randomly?
So the adapted model is identical to the base model at step zero. If both factors were random the model would start from a perturbed version of itself and the first part of training would be spent undoing that. Both cannot be zero, because the product would then have no gradient direction; zeroing one and randomising the other gives an inert start that still trains. On the first step only B receives gradient, since the gradient to A passes through B.
What is the saving, in numbers?
For a 768 by 768 projection at r=8, 12,288 trained parameters against 589,824, about 2%. The memory saving is larger than that, because Adam keeps two momentum states per trainable parameter: a 7B full fine-tune needs roughly 84 GB for weights, gradients and optimiser state, where LoRA needs around 14 GB, or about 5 GB with a 4-bit frozen base.
When would raising r not help?
When the update the task needs is already within rank r, which is the common case for format, tone and domain vocabulary. Rank caps what can be represented, so above the rank actually required, extra capacity buys nothing and costs memory. If raising r keeps improving your evaluation, that is evidence the change you want is large, and you should compare against full fine-tuning rather than keep climbing.
What gets marked up. Saying r is rank and alpha is scaling is passable. Strong answers know that alpha over r is what reaches the model and why the division is there, that B starts at zero for a reason they can state, and that the memory saving comes mostly from optimiser state rather than from the parameter count.
Why can a LoRA adapter be merged into the base weights but an older adapter layer cannot?
Because LoRA is a parallel branch that is added to the output, and addition of a low-rank product into the weight matrix is exact: W plus alpha over r times B times A has the same shape as W and computes the same function. Older adapters inserted a small bottleneck layer with its own nonlinearity in sequence between the existing layers, and a nonlinearity cannot be folded into a neighbouring matrix. That is why they added inference latency and LoRA does not.
You are serving fifty different fine-tunes of one base model. Merged or unmerged?
Unmerged. Fifty merged models is fifty copies of the base weights in memory, where fifty adapters at r=16 are a few tens of megabytes each on top of one shared base, and the adapter can be switched per request in a batch. The cost is the extra low-rank matmul per layer, which is small because it runs at rank r rather than full width. Merge when you are serving one adapter at high volume and want the last of the latency.
Which layers would you put adapters on?
The attention projections first — query and value are the usual default, and they were what the original work found sufficient. Adding key and output projections, and then the feed-forward matrices, covers more of the model at proportionally more cost; the feed-forward layers hold most of the parameters, so putting adapters there is the largest step up. Start narrow and widen only if evaluation says the capacity was the limit, because a wider adapter set is the same kind of capacity increase as a larger r.
These are the questions for one topic. The whole question bank covers every topic on the site.