Asked about Working inside the window
Why not just use a model with a larger context window?
An answer that works
Cost scales with every token on every call, latency grows, and recall across a long window is uneven with material in the middle attended to least. A bigger window postpones the problem and makes it more expensive.
Reading an answer is not being able to give one. The section that teaches this is Working inside the window, 4 hours of Context management.
Learn this properlyAlso asked about Context management
How do you handle a conversation that outgrows the context window?What gets lost when you compact a conversation, and why does it matter?How would you give an assistant long-term memory?Why not store the whole conversation history as memory?A user says the assistant is confidently wrong about their project. How do you debug it?