Typing & validation, section 2 of 2, 5h

Pydantic and type checkers in practice

By the end: You can model real data with Pydantic, validate model output with useful errors, and let a type checker catch mistakes before they run.

  • Models, fields and defaults
  • Custom validators
  • Unions with a tag field
  • Validate, then retry with the error
  • Settings from the environment
  • Running a type checker

New to typing & validation? A type says what kind of value something is: a number, a piece of text, a record with particular fields. Validation is code that checks a value really has that shape before anything downstream trusts it. In Python, type hints write the expectation down and a library such as Pydantic enforces it while the program runs, turning an unchecked blob of data into an object whose fields are guaranteed to exist. Why it matters

Model output arrives as text, and text can be anything: the wrong shape, the wrong type, a refund for fifty times what anyone authorised. This section is about the layer that stops all of that at the edge of your system, by declaring what the data must look like, adding the rules a type cannot express, deciding what happens when validation fails, and letting a type checker read the rest of the code for free. The alternative is a dictionary passed around the codebase that every function downstream has to guess about.

The smallest model#

from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

User(name="Jack", age="31")    # works: "31" is converted to 31
User(name="Jack", age="old")   # raises ValidationError, and says which field

Pydantic checks and converts data when you create the object. After that line, the rest of your code can trust that age is an int.

1 validation error for User
age
  Input should be a valid integer, unable to parse string as an integer
  [type=int_parsing, input_value='old', input_type=str]
The real message. It names the field, what was expected, what arrived, and a stable type code you can count and branch on.

Three details about what it does with the data. It coerces where the conversion is unambiguous, so the string '31' becomes 31 and '1.5' becomes a float, but 1.7 into an int field is rejected rather than truncated, because discarding information silently is worse than failing. It ignores unknown fields by default, which is usually right for an API you do not control and wrong for model output, where an unexpected key means the model answered a different question; set extra='forbid' there. And it validates on construction only, so mutating an attribute afterwards skips every check unless you ask for validate_assignment.

class Strict(BaseModel):
    model_config = {"extra": "forbid", "validate_assignment": True}
    a: int

Strict.model_validate({"a": 1, "b": 2})
# ValidationError: b  Extra inputs are not permitted [type=extra_forbidden]

class Loose(BaseModel):
    a: int

Loose.model_validate({"a": 1, "b": 2})     # a=1   the b is silently dropped

class C(BaseModel):
    n: int
C(n="7")      # n=7,  coerced
C(n=1.7)      # ValidationError: Input should be a valid integer,
              #                  got a number with a fractional part
Real outputs. Choose extra='forbid' at a boundary where a surprise field is evidence of a problem, and the default where a provider may add fields you do not care about.

The three entry points do the same job on different input. model_validate takes a dict, model_validate_json takes a JSON string and is the one to use on a model response because it parses and validates in one step with a clearer error when the text is not JSON at all, and constructing the class directly validates keyword arguments. Going the other way, model_dump gives a dict and model_dump_json a string; model_construct skips validation entirely and exists for data you have already checked, not as a way round an error.

Rules that types cannot express#

from pydantic import BaseModel, Field, field_validator

class Refund(BaseModel):
    order_id: str = Field(pattern=r"^A-\d{3,}$")
    amount: float = Field(gt=0, le=500)
    reason: str

    @field_validator("reason")
    @classmethod
    def not_empty(cls, v: str) -> str:
        if not v.strip():
            raise ValueError("reason must say why")
        return v.strip()

Limits like 'at most 500' belong here, next to the data, not scattered through the code that uses it. A model asking for a refund of 5,000 is stopped at the boundary.

Two things about validators are worth knowing precisely. A validator returns the value, which means it can normalise as well as check: the one above strips whitespace, and that stripped value is what the object carries, so every caller downstream sees the tidy version. And it signals failure by raising ValueError, not by returning False and not by raising ValidationError, which pydantic constructs for you by collecting everything that failed.

Refund(order_id="A-102", amount=10, reason="  damaged  ")
# order_id='A-102' amount=10.0 reason='damaged'      note the strip

Refund(order_id="X-1", amount=5000, reason=" ")
# 3 validation errors:
#   order_id  String should match pattern '^A-\d{3,}$'
#   amount    Input should be less than or equal to 500
#   reason    Value error, reason must say why
Real output. All three problems come back at once, which is what makes a retry that feeds the error back worth doing: the model fixes everything in one attempt.

For a rule that spans several fields, a field validator cannot see the others, so use a model validator with mode='after', which runs once the whole object exists and is fully typed. mode='before' runs on the raw input instead and is the place to reshape something the model tends to get wrong, such as accepting a bare string where an object was asked for.

from pydantic import model_validator

class DateRange(BaseModel):
    start: str
    end: str

    @model_validator(mode="after")
    def ordered(self):
        if self.start > self.end:
            raise ValueError("start must not be after end")
        return self       # an 'after' validator returns self, not a value

DateRange(start="2026-05-01", end="2026-01-01")
# ValidationError: Value error, start must not be after end
Verified. The cross-field rules that matter in practice: a range ordered correctly, a discount not exceeding the total, an action that names a team only when it is an escalation.

One of several shapes#

Agents often return one of several actions. A tag field tells Pydantic which shape to expect, which gives far clearer errors than trying each shape in turn.

from typing import Annotated, Literal, Union
from pydantic import BaseModel, Field, TypeAdapter

class Reply(BaseModel):
    type: Literal["reply"]
    text: str

class Escalate(BaseModel):
    type: Literal["escalate"]
    team: Literal["billing", "tech"]

Action = Annotated[Union[Reply, Escalate], Field(discriminator="type")]
action = TypeAdapter(Action).validate_json(model_output)

TypeAdapter is how you validate anything that is not itself a BaseModel: a union like this one, a list[Extraction], a plain dict[str, float]. Build it once and reuse it, because construction compiles the validator and is the expensive part.

ta = TypeAdapter(Action)

ta.validate_json('{"type": "escalate", "team": "billing"}')
# Escalate(type='escalate', team='billing')

ta.validate_json('{"type": "escalate", "team": "legal"}')
# ValidationError: escalate.team
#   Input should be 'billing' or 'tech'

ta.validate_json('{"type": "unknown"}')
# ValidationError: Input tag 'unknown' found using 'type' does not match any of
#                  the expected tags: 'reply', 'escalate'
Real errors. Without the discriminator, pydantic tries every member and reports why each one failed, which for four shapes is a wall of text that names the wrong problem first.

The same pattern is what a tool call is: a name that selects a shape, and arguments validated against that shape. Define one model per tool, union them on the name, and the dispatch becomes a match on the validated object rather than a chain of dictionary lookups that each assume a key exists.

Validate, then retry with the error#

from pydantic import ValidationError

def get_refund(messages, attempts=2):
    for _ in range(attempts):
        raw = call_model(messages)
        try:
            return Refund.model_validate_json(raw)
        except ValidationError as e:
            messages = messages + [
                {"role": "assistant", "content": raw},
                {"role": "user", "content": f"That failed validation: {e}. Fix it."},
            ]
    raise ExtractionFailed(raw)

The error message is specific, so the model usually fixes it in one go. Cap the attempts and count how often the retry happens.

Four things decide whether this helps or just costs twice as much. Feed back the error, not a generic instruction to try again; the errors() output names the field and the constraint, and that is the whole reason the second attempt succeeds. Cap attempts at two, because a third almost never converges and you are now paying three times for a document that needs a human. Do not retry a refusal: if the model declined, the input is the problem and the same prompt will decline again. And record the retry rate per field, because a field that needs a retry on a fifth of documents is a prompt or schema problem that a retry loop is quietly paying to work around for ever.

outcome              share   what it costs        what it means
valid first try      0.91     1 call               working
valid after retry    0.07     2 calls + latency    watch which field
failed after 2       0.02     2 calls, no result   review queue
refused              0.00     1 call               do not retry these
Report it in this shape after any extraction change. A first-try rate that drops after a prompt edit is the fastest signal you have that the edit was wrong.

Settings#

The pydantic-settings package reads configuration from environment variables into a typed object and fails at startup if something is missing. A missing API key should stop the service from starting, not surface as an error on the first request an hour later.

from pydantic import Field
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    anthropic_api_key: str                       # no default: required
    model: str = "claude-sonnet-5"
    max_concurrency: int = Field(default=8, ge=1, le=64)
    request_timeout_s: float = 30.0

    model_config = {"env_file": ".env"}

settings = Settings()      # raises here, at import, if the key is not set
Field names map to environment variables case-insensitively, so ANTHROPIC_API_KEY fills the first. A typo in MAX_CONCURENCY is then a missing variable with a default rather than a silent zero.

Let a type checker read your code#

Pydantic checks data at runtime. A static type checker, such as mypy or pyright, checks your code without running it: passing a str where an int is expected, forgetting that a value can be None. Run one in CI. It catches a class of bugs that tests often miss, especially the None case.

The two are complementary and neither substitutes for the other. A type hint is a claim about your own code that costs nothing at runtime and is not enforced there; pydantic is an actual check on data you did not write. So hints go everywhere inside the system and pydantic goes at the edges, which is the same division the previous section drew between trusted and untrusted.

def cheapest(models: list[str]) -> str | None:
    ...

best = cheapest(available)
print(best.upper())
# mypy: error: Item "None" of "str | None" has no attribute "upper"

# Three annotations worth writing by hand:
from typing import Literal, TypedDict

Role = Literal["user", "assistant"]          # not str
class Message(TypedDict):                     # a dict with known keys
    role: Role
    content: str

def run(messages: list[Message]) -> str: ...
Start with mypy in non-strict mode on new files only, or the first run on an existing codebase produces four hundred errors and gets turned off.

Where people go wrong

Calling json.loads on model output and passing the dict around. Every function downstream then has to guess what is in it.

If this comes up in an interview

Where should validation happen in an AI application?

At the boundaries: when data enters from users, from other services, and from the model. Validate once at the edge into typed objects, and let the inside of the system trust them.

Then they push further

Why treat model output like user input?

Because it can be anything. It can be malformed, out of range, or shaped by text an attacker put in a document. It needs the same checks as anything else that crosses a trust boundary.

Validation fails. What does your system do?

Retry once with the specific error passed back, then fail cleanly: store nothing, send the case to a review queue or return a clear error, and log it. Never store a guess.

What gets marked up. Marked up for treating model output as untrusted input and for a defined path when validation fails.

What is a discriminated union and when would you use one?

A union of shapes with a tag field that says which shape an object is. It suits outputs that can be one of several actions, and gives precise errors because the validator knows which shape to check.

Build this: Model the output of an agent with at least three action types as a discriminated union with field limits. Add a validate-and-retry loop, run it on thirty inputs, and report the first-try and after-retry success rates.

This is one section of Typing & validation, 11 hours across 2 sections. Read it in full to get the whole topic in order.

Stop 7 of 50 on the AI Engineer route

0 of 50 topics done · 0h behind you, 603h to go

Next on this routeAsync & concurrencyWaiting on many slow calls at once.10h

What you have marked done is stored in this browser only. It does not reach a server, and clearing site data will clear it.