Time to first token

Also: TTFT

How long from sending a request until the first token comes back, dominated by prefill and therefore by prompt length. The number that decides whether a streaming interface feels responsive.

A definition is not an understanding. This is taught properly in Find out where the money goes, part of Cost & latency engineering.

Learn this properly

Terms from the same topic

LatencyTime per output token