Counting tokens and what they cost
By the end: You can measure exactly what a request costs before and after you send it.
- Counting tokens in practice
- Input versus output pricing
- Reading usage data from a response
- Working out cost per request and per user
New to tokenization? A model does not read letters, and it does not quite read words either. Before any text reaches it, the text is cut into tokens: short pieces of characters drawn from a fixed vocabulary the model was trained on. A common word like "the" is one token; "unhappiness" might arrive as three; a space or a bracket is often a token of its own. Why it matters
You know what a token is; this is how to find out how many you are sending and what that comes to in money. Input and output are priced differently, often by a factor of five, so the same total can cost very different amounts depending on which direction it went in. The section covers measuring before you send, reading the usage figures afterwards, and turning both into a cost per request and per user, which is the number somebody will eventually ask you for.
Count, do not guess#
Every provider exposes the token count of a response, and most let you count text before sending it. Use the real number whenever it affects cost or fits in a limit.
response = client.models.generate_content(
model="gemini-3.5-flash-lite",
contents="Explain tokenization in one sentence.",
)
usage = response.usage_metadata
usage.prompt_token_count # what you sent
usage.candidates_token_count # what came backInput and output are priced differently#
Output almost always costs several times more than input. This matters because it changes where you should look when a bill is too high. A long prompt is cheaper than a long answer.
INPUT_PER_MILLION = 1.00
OUTPUT_PER_MILLION = 5.00
def cost(usage) -> float:
return (
usage.prompt_token_count / 1_000_000 * INPUT_PER_MILLION
+ usage.candidates_token_count / 1_000_000 * OUTPUT_PER_MILLION
)
print(f"${cost(usage):.6f}")Where the cost actually goes#
In a chat or an agent, the input grows every turn because you resend the whole history. By turn ten you are paying to re-read nine previous turns, most of which is the model's own earlier output. That, not the length of any single message, is what makes long conversations expensive.
Turn 1: 7 input tokens
Turn 2: 33 input tokens
Turn 3: 48 input tokens
Turn 10: several hundred, for the same short questionLog cost from the first day
Record tokens and cost for every call from the very start of a project. Adding it later means you cannot answer the first question anyone asks, which is what the expensive path is.
Where people go wrong
Measuring cost per request instead of per completed task. A cheaper model that needs three attempts is not cheaper. The number that matters is what it costs to get the job done.
If this comes up in an interview
Why is output usually more expensive than input?
Generating tokens is sequential work: the model produces one token at a time, each conditioned on all the previous ones. Reading input is done in one parallel pass. The pricing reflects the difference in compute.
Your chatbot costs $0.40 per conversation and needs to be under $0.10. Where do you look first?
Input tokens across the whole conversation, because history is resent every turn and grows quadratically. Prompt caching on the stable prefix, trimming or summarising old turns, and routing simple turns to a cheaper model, in that order.
How would you estimate the monthly cost of a feature before building it?
Take a representative prompt and response, count both, apply the per-million rates, and multiply by expected requests. Add a margin for retries and for conversations being longer than the sample.
Build this: Add a cost meter to a script that runs a three-turn conversation. Print running total cost after each turn, and watch the input count climb while your questions stay the same length.
Watch it explained
This is one section of Tokenization, 10 hours across 4 sections. Read it in full to get the whole topic in order.
Stop 3 of 50 on the AI Engineer route
0 of 50 topics done · 0h behind you, 603h to go
What you have marked done is stored in this browser only. It does not reach a server, and clearing site data will clear it.