Term · Compute & AI Hardware
Prefill
The first phase of language model inference, processing the entire input prompt in parallel to populate the KV cache and produce the first output token. It is compute bound and sets time to first token.
- Also known as
- Prompt processing, Context encoding
- Type
- Term
- Domain
- Compute & AI Hardware, Inference and serving
Notes
Because prefill and decode stress different resources, large deployments increasingly run them on separate pools of hardware.
Related terms
Part of the Compute & AI Hardware glossary, with every term in the topic and the lifecycles, tools and standards that use them.
Where it appears on the Hub
Used on 1 page of the Hub. In each group, the pages that mention it most come first.
Skills
Suggest an improvement
Suggest an improvement to this entry
A clearer definition, an example, a correction or a source an editor can check. Editors review every suggestion before the entry changes.
Last reviewed: not yet reviewed in depth. Entry last updated . The full glossary