Asked about Reproducing a paper
You tried to reproduce a paper and got a result 5 points below theirs. What do you do?
An answer that works
Check the evaluation setup matches exactly, check whether the gap is within seed variance, compare against the official code for undocumented details, verify the baseline matches theirs, and contact the authors. Then write up what I found, including the gap.
Reading an answer is not being able to give one. The section that teaches this is Reproducing a paper, 6 hours of Reading papers.
Learn this properlyAlso asked about Reading papers
How do you keep up with the literature?What makes you sceptical of a paper's result?A paper reports a gain on a benchmark. How do you decide whether it applies to your system?When is it worth reimplementing a paper rather than reading it?What do you check first in a paper's results?What makes you doubt a result will replicate?How do you approach a dense equation in a paper you need to implement?