Reading papers interview questions
8 questions across 4 sections, in the order you would learn them. Every answer here is one that works — but an answer you have read is not an answer you can give, so use Test yourself and say it out loud first.
Reading papers without reading papers
Read the section that teaches thisIf this comes up in an interview
How do you keep up with the literature?
Triage rather than volume. Abstract, figures and conclusion for most, method and results for the promising ones, full depth only for what I will build on. Breadth comes from following people with good judgement; depth I do myself.
What makes you sceptical of a paper's result?
A weak baseline first, since most disputes are really about baselines. Then single-seed results with no variance reported, small effects framed as large, and missing ablations, which often mean a component did not help.
Then they push further
Give me a specific phrase you would stop on.
'We use the default settings for the baseline', which usually means the method got a hyperparameter sweep the baseline did not. Also 'results are averaged over multiple runs' without saying how many, and an improvement reported only as a percentage, which hides whether the absolute change is half a point.
What gets marked up. A strong answer names the sentence in the paper that triggers the check, not the abstract category. Naming baselines and variance is passable; saying what you would look up next is what separates it.
A paper reports a gain on a benchmark. How do you decide whether it applies to your system?
Compare the setting rather than the number. Is the baseline the thing I actually run, is the data distribution like mine, and is the effect larger than the seed spread they report? If it survives that and it would be cheap to test, I run it on my own eval rather than adopting it, because my eval is the only one I will be judged on.
When is it worth reimplementing a paper rather than reading it?
When I am about to build on it, and then scoped to one figure at the smallest scale where the effect should still appear, with my own baseline run first. A handful a year. What it teaches is the size of the gap between a paper and a working system: the schedule, the prompt template, the eval harness version and the checkpoint selection are routinely unstated, and after guessing at those once you read every paper differently.
Reading the results section critically
Read the section that teaches thisIf this comes up in an interview
What do you check first in a paper's results?
The absolute size of the improvement, whether there are error bars or multiple seeds, and whether the baselines were run under the same conditions. Then whether the claimed result holds across all settings or only some.
What makes you doubt a result will replicate?
Single-seed results, a large gain from a small change with no mechanism, an untuned baseline, no released code, and improvements that only appear on the authors' own benchmark.
Reading the maths in papers
Read the section that teaches thisIf this comes up in an interview
How do you approach a dense equation in a paper you need to implement?
Identify every symbol and its shape, rewrite it in words, substitute small numbers and check extreme cases, then write it as code with shape comments and test it on a case where I know the answer.
Reproducing a paper
Read the section that teaches thisIf this comes up in an interview
You tried to reproduce a paper and got a result 5 points below theirs. What do you do?
Check the evaluation setup matches exactly, check whether the gap is within seed variance, compare against the official code for undocumented details, verify the baseline matches theirs, and contact the authors. Then write up what I found, including the gap.
These are the questions for one topic. The whole question bank covers every topic on the site.