Asked about Reading the results section critically
What makes you doubt a result will replicate?
An answer that works
Single-seed results, a large gain from a small change with no mechanism, an untuned baseline, no released code, and improvements that only appear on the authors' own benchmark.
Reading an answer is not being able to give one. The section that teaches this is Reading the results section critically, 4 hours of Reading papers.
Learn this properlyAlso asked about Reading papers
How do you keep up with the literature?What makes you sceptical of a paper's result?A paper reports a gain on a benchmark. How do you decide whether it applies to your system?When is it worth reimplementing a paper rather than reading it?What do you check first in a paper's results?How do you approach a dense equation in a paper you need to implement?You tried to reproduce a paper and got a result 5 points below theirs. What do you do?