Asked about Find out where the money goes
What drives latency in an LLM call?
An answer that works
Two different things. Time to first token is prefill over the prompt, which is compute-bound and superlinear in prompt length because attention is quadratic, so a long prompt delays the first token more than proportionally. Everything after that is decode, roughly a fixed cost per output token, so it scales with how long the answer is. Shorten the prompt to fix a slow start; cap output tokens to fix a slow finish.
Reading an answer is not being able to give one. The section that teaches this is Find out where the money goes, 2 hours of Cost & latency engineering.
Learn this properly