Overfitting & regularisation interview questions
5 questions across 2 sections, in the order you would learn them. Every answer here is one that works — but an answer you have read is not an answer you can give, so use Test yourself and say it out loud first.
Memorising instead of learning
Read the section that teaches thisIf this comes up in an interview
How do you know a model is overfitting?
Training performance far exceeds held-out performance. If both are poor it is underfitting, which is a different problem needing more capacity rather than less.
Your learning curves have converged and both are poor. What now?
More data will not help, because the model has already extracted what it can. The answer is more capacity, better features, or a different model class.
What is dropout and why does it work?
Randomly ignoring a fraction of units during training, so the network cannot depend on any single one and must learn redundant representations. It behaves like training an ensemble of sub-networks that share weights.
Regularisation and validation, hands on
Read the section that teaches thisIf this comes up in an interview
Your model scores 0.95 AUC in cross-validation and 0.70 in production. What might have happened?
Most often leakage in the validation setup: the same entity on both sides of the split, preprocessing fitted on all data, a feature that is only known after the outcome, or shuffling time-ordered data. Otherwise, production data differs from the training data.
Then they push further
How would you check for the same entity on both sides?
Look for an id such as user, patient or device that repeats across rows. Rerun cross-validation with group folds on that id. A large drop means the earlier score was inflated.
And for a feature that leaks the answer?
Check feature importance for anything suspiciously strong, and for each top feature ask when it becomes known. If it is only recorded after the outcome, it cannot be used.
What gets marked up. Strong answers go straight to the validation setup and name group, time and preprocessing leakage. 'The model overfit' without a mechanism is marked down.
What is the difference between L1 and L2 regularisation?
L2 penalises squared weights and shrinks them all smoothly. L1 penalises absolute weights and drives many to exactly zero, which also performs feature selection.
These are the questions for one topic. The whole question bank covers every topic on the site.