Asked about Reasoning, and improving a prompt honestly
How do you know a prompt change actually improved things?
An answer that works
Score it against a fixed set of inputs with known expected outcomes, before and after. Without a baseline you are comparing against your memory of previous outputs, which is unreliable in a predictable direction.
Then they push further
Your eval went from 80% to 82%. Is that an improvement?
Not necessarily. On a small set that is within noise. Rerun the baseline to see how much it varies on its own, and compare the two versions only on the cases where they disagree.
What gets marked up
Strong answers mention an eval run before and after, a held-out split, and awareness of noise. Weak answers describe trying a few examples by hand.
Reading an answer is not being able to give one. The section that teaches this is Reasoning, and improving a prompt honestly, 4 hours of Prompting.
Learn this properlyAlso asked about Prompting