Asked about What a token is
Why does the same content cost more in some languages?
An answer that works
Tokenizers are trained mainly on English text, so English gets efficient single tokens for common words. Other scripts fall back to smaller fragments, often two to three times as many tokens for the same meaning.
Reading an answer is not being able to give one. The section that teaches this is What a token is, 2 hours of Tokenization.
Learn this properlyAlso asked about Tokenization
What is a token?Why use subword tokens rather than words or characters?Roughly how many tokens is 1,000 words of English?Why is output usually more expensive than input?Your chatbot costs $0.40 per conversation and needs to be under $0.10. Where do you look first?How would you estimate the monthly cost of a feature before building it?What is a context window?What happens if you exceed it?
All 14 questions on Tokenization