Asked about Context limits and what fits
If a model has a very large context window, why not put everything in it?
An answer that works
Cost scales with every token on every call, latency grows, and recall across a long window is uneven, with material in the middle attended to least reliably. Retrieving the relevant part is usually both cheaper and more accurate.
Reading an answer is not being able to give one. The section that teaches this is Context limits and what fits, 2 hours of Tokenization.
Learn this properlyAlso asked about Tokenization
What is a token?Why use subword tokens rather than words or characters?Roughly how many tokens is 1,000 words of English?Why does the same content cost more in some languages?Why is output usually more expensive than input?Your chatbot costs $0.40 per conversation and needs to be under $0.10. Where do you look first?How would you estimate the monthly cost of a feature before building it?What is a context window?
All 14 questions on Tokenization