Prefill
Also: prompt processing
The first phase of inference, processing the entire prompt in one parallel pass to fill the KV cache and emit the first token. It is compute-bound, and its length is what sets time to first token.
A definition is not an understanding. This is taught properly in Making generation faster, part of Quantization & inference optimisation.
Learn this properlyTerms from the same topic