Structured output interview questions
10 questions across 3 sections, in the order you would learn them. Every answer here is one that works — but an answer you have read is not an answer you can give, so use Test yourself and say it out loud first.
Asking for JSON versus guaranteeing it
Read the section that teaches thisIf this comes up in an interview
Your extraction pipeline returns malformed JSON 5 percent of the time. What are the likely causes?
Two. Format drift from sampling, which shows as code fences or preamble, fixed by constrained decoding or a lower temperature. And truncation from hitting the token limit, which gives you half an object. The stop reason tells you which one you have.
Then they push further
You switch to a response schema and it still fails sometimes. Why?
Common reasons: the output hit max_tokens and was cut off, the schema uses features the provider does not enforce, or the JSON is valid but the values are wrong. Check the stop reason first.
Valid JSON, wrong values. How do you catch that?
Validate beyond the schema: ranges, consistency between fields, and that extracted values actually appear in the source where they should. Count failures by type so you know which to fix first.
What gets marked up. Marked up for checking the stop reason for truncation, and for separating well-formed output from correct output.
What does a response schema guarantee?
That the output is structurally valid and matches the declared types. It guarantees nothing about whether the values are correct. A schema will happily return well-formed JSON full of invented data.
How does constrained decoding actually work?
At each generation step the sampler is restricted to tokens that keep the output valid against the schema. Invalid continuations are given zero probability, so malformed output is not reachable rather than merely discouraged.
Should every field be required?
No. Marking a field required when the source may not contain it forces the model to invent a value. Make genuinely optional fields optional and instruct the model to return null rather than guess.
Designing schemas the model fills well
Read the section that teaches thisIf this comes up in an interview
Why does field order in a schema matter?
The model generates the output in order. If the conclusion comes first it is chosen before any supporting text is written. Putting evidence or reasoning fields before the conclusion lets the conclusion depend on them.
Then they push further
Does that mean you should always add a reasoning field?
No. It costs output tokens on every call and is only worth it where the decision is hard. Measure accuracy with and without it on your eval. For simple fields it adds cost and nothing else.
What gets marked up. Marked up for linking it to left-to-right generation and for treating the reasoning field as a cost to be measured, not a free win.
How would you reduce invented values in an extraction pipeline?
Allow null for anything that may be absent and say so in the field description, include an escape value in every enum, and ask for the source text of each value so code can check it appears in the document. Then measure how often values fail that check.
Your classification enum has an 'other' option that is now used 30% of the time. What does that tell you?
That the categories no longer fit the traffic. Read a sample of the 'other' cases, group them, and either add categories or route them differently. It is a useful signal, which is exactly why the escape value exists.
Extraction you can measure
Read the section that teaches thisIf this comes up in an interview
How would you measure the accuracy of an invoice extraction pipeline?
Hand-label fifty or more real invoices, normalise values before comparing, and report accuracy per field. For lists such as line items report precision and recall separately. Then read the failures for the worst field first.
Then they push further
Your total field is 79% accurate. What do you look at?
The failures, grouped. Common causes are picking the subtotal instead of the total, missing tax, reading the wrong currency, or documents that parsed badly. Each has a different fix, so count them before changing anything.
How do you keep it accurate after launch?
Sample processed documents every week for review, route validation failures to people, and add every confirmed mistake to the labelled set. Watch the review rate, since new document layouts show up there first.
What gets marked up. Strong answers score per field, normalise before comparing, and look at failures before fixing. Document-level accuracy only, or no labelled set, is marked down.
When would you use a batch API?
For work that can wait hours, such as processing a backlog or nightly jobs. It is much cheaper per request. It is wrong for anything a user is waiting on.
How would you run an extraction over 250,000 documents?
Size the worker pool from the rate limit and the call latency rather than by guessing: requests in flight equals rate times latency, so a 4,000 per minute limit with 2.5 second calls justifies about 167 workers. Append each finished row to a file and start every run by reading back what is already there, so a crash costs only the unfinished part. Price it first on the eval set, and use the batch interface if hours are acceptable.
Then they push further
The job is being throttled. What do you change?
Honour the retry-after header, back off exponentially with jitter, and shrink the pool. Retrying harder under throttling turns a slow job into a stalled one. Check which limit is binding, since the token limit usually binds before the request limit on long documents.
How do you watch quality across all 250,000 when you can only label a few hundred?
Run two proxies that need no labels. Agreement: extract twice under settings that should not change the answer and count the fields that differ, since a field the pipeline cannot agree with itself about is one it does not know. Grounding: check the quoted source text really appears in the document. Both are floors rather than estimates, so keep the labelled set as the number you would quote.
What gets marked up. Strong answers derive the concurrency from the limit rather than picking a number, make the job restartable, and separate the unlabelled proxies from real accuracy instead of presenting them as the same thing.
These are the questions for one topic. The whole question bank covers every topic on the site.