Why evals exist

Everything else in this topic rests on being able to answer one question: did that change make the system better or worse? This section is about why the question is hard for anything built on a model, why the testing tools you already have do not answer it, and what takes their place. It is worth taking seriously because the alternative compounds: a few months of unmeasured prompt edits produces a system nobody on the team is willing to touch.

The question you are actually asking             What answers it

"did my change break anything?"                 a regression set and a gate
"which of these two prompts is better?"         both on the same cases, paired
"is this good enough to ship?"                  a set shaped like real traffic
"how often does failure mode X happen now?"     a targeted set, one grader for X
"can I trust the grader itself?"                your own labels, and a kappa
"is this agent reliable?"                       the same task run k times

The problem#

You change a prompt. Is the system better? With ordinary code you run the tests. With a model you look at a couple of outputs, they seem fine, and you ship. That is not a measurement, and over a few months it produces a system nobody can safely change.

An eval is the missing measurement: a fixed set of inputs, a way of scoring the output, and a number you can compare between versions.

Why unit tests do not cover it#

Model output varies between runs, even at temperature zero on a hosted API, and there is rarely one correct string. Asserting equality against expected text produces a suite that flakes, gets ignored, and is eventually deleted.

import random

article = "The company reported record profits of 4.1bn for the year."

def summarise(text):
    """Stands in for a model call: same meaning, different words each time."""
    return random.choice(["The company reported record profits.",
                          "Record profits were reported by the company.",
                          "The company posted record profits of 4.1bn."])

def judge_says_accurate(summary, source):
    return "profit" in summary.lower() and "loss" not in summary.lower()

# Will flake, then be deleted
random.seed(0)
try:
    assert summarise(article) == "The company reported record profits."
    print("passed, this time")
except AssertionError:
    print("failed, and the summary was fine")

# Stable, and actually meaningful
for seed in range(5):
    random.seed(seed)
    result = summarise(article)
    assert len(result.split()) < 50
    assert "profit" in result.lower()
    assert judge_says_accurate(result, article)
print("5 seeds, 3 different summaries, all pass")

# failed, and the summary was fine
# 5 seeds, 3 different summaries, all pass

Where most teams are#

• Nobody can say whether last month's prompt change helped.
• A regression is discovered by a user, not by CI.
• Prompt edits are made by whoever is least afraid to touch it.
• Nobody will change the prompt before a demo.

Start small, today#

Twenty examples in a file and a script that scores them is enough to change how a project runs. It does not need a framework and it does not need to be complete before it is useful.

CASES = [
    {"input": "...", "must_contain": ["refund", "5 days"]},
    {"input": "...", "must_be_json": True},
    {"input": "...", "expect_empty": True},
]

The whole harness is about thirty lines. A case is an input, a check, and an id you can say out loud in a stand-up. The check is ordinary Python: the cheapest useful eval is a list of predicates, and you can add a judge later for the cases where no predicate works.

import json

CASES = [
    {"id": "refund-annual", "input": "refund on the annual plan after 20 days?",
     "check": lambda out: "14 days" in out and "30" not in out},
    {"id": "empty-source", "input": "what is the refund policy for the beta tier?",
     "check": lambda out: "do not have" in out or "not sure" in out},
    {"id": "json-shape", "input": "list my open orders as json",
     "check": lambda out: isinstance(json.loads(out), list)},
]

def system(text):
    """Stands in for the thing you are evaluating."""
    if "annual" in text:
        return "Annual plans can be refunded within 14 days."
    if "beta" in text:
        return "Beta tier refunds are handled within 30 days."
    return '[{"id": "A-101"}]'

def run(cases):
    rows = []
    for case in cases:
        try:
            out = system(case["input"])
            ok = bool(case["check"](out))
            note = ""
        except Exception as e:                 # a crash is a failure, not an error
            out, ok, note = "", False, type(e).__name__
        rows.append((case["id"], ok, note, out))
    passed = sum(ok for _, ok, _, _ in rows)
    print(f"{passed}/{len(rows)} passed")
    for cid, ok, note, out in rows:
        if not ok:
            print(f"  FAIL {cid} {note} -> {out[:60]!r}")
    return passed / len(rows)

run(CASES)

# 2/3 passed
#   FAIL empty-source  -> 'Beta tier refunds are handled within 30 days.'

Three things in that harness matter more than they look. An exception counts as a failure rather than crashing the run, because the eval you abandon halfway through is not a measurement. Every case has an id, so a regression is reported as a name and not a percentage. And the failures print with their output attached, since the first thing you do with a red eval is read what actually came back.

What the rest of this topic does with that number#

The sections after this one fill in the parts the thirty-line version leaves out, roughly in the order you will need them. Read traces first, so the cases are shaped like your real failures. Build the set from production rather than imagination. Choose a grader per case, and validate the grader when it is a model. Run the whole thing on every change. Then read the result honestly, which mostly means per category and with an error bar.

Three questions that sit next to evals are answered in neighbouring topics rather than here, and it is worth knowing where they are. How many examples you need before a comparison can decide anything is worked out with numbers in the product judgement topic, under choosing and changing models. What happens to a set you tune against repeatedly is measured in eval-driven development, under improving against a number. And the same statistics applied to real users, including what checking the dashboard repeatedly does to your false-positive rate, is the experiment design topic.