Asked about Counting tokens and what they cost
Your chatbot costs $0.40 per conversation and needs to be under $0.10. Where do you look first?
An answer that works
Input tokens across the whole conversation, because history is resent every turn and grows quadratically. Prompt caching on the stable prefix, trimming or summarising old turns, and routing simple turns to a cheaper model, in that order.
Reading an answer is not being able to give one. The section that teaches this is Counting tokens and what they cost, 2 hours of Tokenization.
Learn this properlyAlso asked about Tokenization
What is a token?Why use subword tokens rather than words or characters?Roughly how many tokens is 1,000 words of English?Why does the same content cost more in some languages?Why is output usually more expensive than input?How would you estimate the monthly cost of a feature before building it?What is a context window?What happens if you exceed it?
All 14 questions on Tokenization