Writing a paper section by section

Every section of a paper has a job, and each has a shape that reviewers are used to. Following the shape is not a lack of imagination; it is what lets a reader find what they need in the two minutes they will actually give you. This section gives the shape for each part, with the words written out, because the advice to write a clear abstract is useless and the five sentences it should contain are not.

The abstract, in five sentences#

One sentence each: the context everyone accepts, the specific problem with it, what you did, what you found with a number in it, and what it means. Nearly every good abstract is this, and reading a few with the pattern in mind makes it hard to unsee.

1. context      Retrieval-augmented systems are now standard for
                answering questions over private documents.
2. problem      Their quality depends heavily on how documents are
                split into chunks, and this choice is usually made
                by guesswork.
3. what you did We compare six chunking strategies across four
                document collections, holding the retriever and the
                generator fixed.
4. what you     Chunking on section boundaries beats fixed-size
   found        chunking by 8.4 points of answer accuracy, and the
                gain is largest on long structured documents.
5. what it      Chunk boundaries matter more than chunk size, which
   means        is the parameter most systems expose.

The introduction, in four paragraphs#

The introduction is the abstract with room to breathe, and it has four jobs in a fixed order. First, the world as it is, ending at the gap. Second, why the gap is hard, which is where you dispatch the obvious solutions a reader is already thinking of. Third, what you did and what you found, stated plainly and early — do not save the result for the end, because a reader who has to wait for it will not. Fourth, your contributions as an explicit list.

paragraph 1   what is true and accepted, narrowing to the gap
paragraph 2   why the obvious fixes do not work
              ("one could simply X, but X fails because ...")
paragraph 3   what we do, and the headline number
paragraph 4   contributions, as a list a reviewer can check off

A method section someone can reimplement#

The test is not whether the method is described. It is whether a competent stranger with your paper and no access to you could produce your numbers. That is a much higher bar, and it is met by including the things that feel too small to mention, because those are exactly the ones that get omitted.

• Every hyperparameter, including the ones you did not tune and the ones you left at their defaults.
• The optimiser, the schedule, the warmup, and the effective batch size after any gradient accumulation.
• Preprocessing in full: tokenizer, normalisation, filtering, deduplication, and what you did with examples that failed it.
• How the splits were made, and whether anything is grouped or time-ordered.
• Which numbers are single runs and which are averaged, over how many seeds.
• Compute: what hardware, and how long. It tells the reader whether they can attempt this.

Write the method in the order the code runs, not the order you discovered things. The history of how you arrived at the method is interesting to you and is a burden to the reader; if it genuinely matters, it is a paragraph in the discussion.

The practical way to hit the reimplementation bar is to write the method from the code rather than from memory, with the file open. Memory reconstructs the version you intended; the file has the learning rate you actually used after the third run, the filter you added on a Friday, and the one line that reorders the data. Those are precisely the details that separate a reimplementation that reproduces your number from one that does not, and the author is the last person who can tell which of them mattered.

Experiments organised by claim#

The common failure is organising this section by experiment — here is table one, here is table two — which makes the reader assemble the argument themselves. Organise it by claim instead, with each subsection named for something you are asserting and containing the evidence for exactly that.

by experiment (weak)              by claim (strong)
  5.1 Main results                   5.1 Boundary-aware chunking
  5.2 Ablations                          improves answer accuracy
  5.3 Additional experiments         5.2 The gain comes from the
  5.4 Analysis                           boundaries, not the size
                                     5.3 It holds across four
                                         document collections
                                     5.4 It costs 4% more indexing
                                         time, and nothing at query
                                         time

Each subsection then follows the same small pattern: state the claim, say which experiment tests it, give the result with its uncertainty, and say what would have falsified it. That last part is rare and is the single thing that most raises a reader's confidence, because it shows the experiment could have come out the other way.

5.2  The gain comes from the boundaries, not the size

CLAIM       Boundary-aware chunking helps because of where it cuts, not
            because its chunks happen to be longer.
TEST        Fixed-size chunking at the mean boundary-aware chunk length
            (612 tokens), against boundary-aware chunking, same corpus,
            same retriever, five seeds.
RESULT      Fixed-size at 612 tokens scores 61.2 (sd 0.7); boundary-aware
            scores 69.4 (sd 0.6). The 8.2-point gap survives matching
            the length.
FALSIFIED   If matching the chunk length had closed the gap to within
            the seed spread, the effect would have been length all
            along and Section 5.1 would not support the claim it makes.

Figures that stand alone#

Assume every figure will be seen by itself, in a talk, on a slide someone else made, or in a screenshot. It has to work without the body text. That means axis labels with units, a caption that states the finding rather than describing the plot, and error bars whose meaning is given in the caption.

weak     Figure 3: Accuracy versus chunk size.

strong   Figure 3: Boundary-aware chunking (blue) beats fixed-size
         chunking (grey) at every chunk size, and the gap widens on
         longer documents. Shaded regions are 95% intervals over
         5 seeds. n = 2,400 questions.

Limitations, early and specific#

Put limitations where they will be read and make them real ones. A limitations paragraph that says results may not generalise to all settings is worse than none, because it signals the authors know the form without meaning it. Name the specific conditions under which you expect your result to fail — a document type you did not test, a scale you could not reach, a baseline you could not run — and say what evidence would settle it. Reviewers reward this, and, more practically, anyone who finds the limitation themselves after you have concealed it stops believing the rest of the paper.