One token through a decoder-only model Diagram

Prefill is not another model

The prompt's token positions pass through the stack with a causal mask. Keys and values populate the cache.
This schematic shows logical dependency, not wall-clock timing or one-token-at-a-time processing of the prompt.

Decode reuses state

Each generation step embeds the newly sampled ID, reads earlier cached keys and values, and appends the new ones.
The return arrow stops at an end token or the output limit. It does not retokenize the generated text.

Different meanings of decode

A decoder block transforms vectors. Tokenizer decode maps IDs back to text. Neither is post-training.
A GPT-style decoder-only model has no separate encoder stack. RoPE operates on attention queries and keys inside the blocks.

State is not learned knowledge

Weights persist between requests. The KV cache is temporary inference state, not an optimizer, a database of facts or a weight update.
The cache connector denotes access: blocks read earlier KV and store new KV. Residuals, norms, heads and MLP are expanded in the separate block diagram.