Context windows interview questions
6 questions across 2 sections, in the order you would learn them. Every answer here is one that works — but an answer you have read is not an answer you can give, so use Test yourself and say it out loud first.
The window as a budget
Read the section that teaches thisIf this comes up in an interview
What counts against the context window?
Everything sent plus everything generated: system prompt, tool definitions, all messages, all tool results, retrieved documents and the reply itself. Forgetting to reserve room for the reply is a common cause of failures.
Is recall uniform across a long context window?
No. Content near the start and the end is recalled more reliably than content in the middle. So position matters, and filling a large window because it exists is worse than curating what goes into it.
Why does a long conversation get expensive faster than it gets long?
Because every turn resends everything before it. The conversation grows linearly and the tokens billed grow with the square of the turn count: forty turns of a conversation that ends at 25,800 tokens bills about 564,000, roughly 22 times its own final size. Prefill latency follows the same curve, so turn 40 is also about ten times slower to start than turn 1. Prompt caching and compaction fix both, and caching only works if history is appended rather than rewritten.
What gets marked up. Strong answers name the quadratic and connect it to latency as well as cost.
A request does not fit. What do you cut first?
Old tool results, then middle conversation turns summarised, then low-scoring retrieved documents, then untested instructions. Never the original task, user corrections or agreed constraints.
Working with inputs bigger than you want to send
Read the section that teaches thisIf this comes up in an interview
A model supports a very long context. Why not always send everything?
Cost and latency grow with every token, recall is weaker for material in the middle of a long context, and irrelevant text can distract the model. Sending only what is relevant is usually cheaper, faster and more accurate.
Then they push further
When is sending the whole document the right call?
When the question needs connections across the whole thing, the document fits comfortably, and the volume is low enough that cost does not matter, for example a one-off review of a contract.
How would you prove which approach is better for your case?
Build a small eval with questions of each kind, run retrieval, map-reduce and the long call on it, and compare accuracy, cost and latency side by side.
What gets marked up. Strong answers name cost, latency and uneven recall, and choose by the kind of question. Treating a bigger window as a replacement for retrieval is marked down.
What would you keep when compressing a long conversation?
The goal, ids and numbers, constraints the user stated, decisions made and attempts that failed, ideally in a structured record, plus the most recent turns word for word.
These are the questions for one topic. The whole question bank covers every topic on the site.