Asked about Context limits and what fits
A user uploads a 500-page PDF and asks a question about it. How do you handle it?
An answer that works
Do not send all of it. Chunk the document, index it, retrieve the sections relevant to the question, and send only those with citations back to the source. That keeps cost flat as documents grow and lets you show where the answer came from.
Then they push further
The question is 'summarise the whole thing'. Does retrieval still work?
No, because the answer depends on all of it. Summarise in stages instead: summarise sections, then summarise the summaries, keeping track of which section each point came from.
What gets marked up
Strong answers separate questions that need one part of the document from ones that need all of it.
Reading an answer is not being able to give one. The section that teaches this is Context limits and what fits, 2 hours of Tokenization.
Learn this properlyAlso asked about Tokenization