Asked about Working with inputs bigger than you want to send
A model supports a very long context. Why not always send everything?
An answer that works
Cost and latency grow with every token, recall is weaker for material in the middle of a long context, and irrelevant text can distract the model. Sending only what is relevant is usually cheaper, faster and more accurate.
Then they push further
When is sending the whole document the right call?
When the question needs connections across the whole thing, the document fits comfortably, and the volume is low enough that cost does not matter, for example a one-off review of a contract.
How would you prove which approach is better for your case?
Build a small eval with questions of each kind, run retrieval, map-reduce and the long call on it, and compare accuracy, cost and latency side by side.
What gets marked up
Strong answers name cost, latency and uneven recall, and choose by the kind of question. Treating a bigger window as a replacement for retrieval is marked down.
Reading an answer is not being able to give one. The section that teaches this is Working with inputs bigger than you want to send, 4 hours of Context windows.
Learn this properly