Session 1: where does each word belong?

Journey map · Checkpoint · Next: Session 02

Time box: 75-90 minutes. Installation/downloads: none. Goal: explain the difference between model parts, training phases, inference phases and temporary state before touching a billion-parameter checkpoint.

For a visual orientation, open 3Blue1Brown’s GPT introduction. Follow the information flow; you do not need to finish the video or understand all the math before starting the worksheet.

Open the inference map, block map and lifecycle map. Use Live to follow arrows, then Still to explain a step. Motion is a reading aid, not a measured execution trace. Click nodes for their relationships; read the explanatory cards.

0-15 minutes: sort the vocabulary

Put each item into one or more of these four boxes. Explain any overlap:

Physical model or surrounding software.
A stage that changes weights.
A phase that uses existing weights.
Temporary state for this request or training step.

Items: tokenizer, tokenizer encode, tokenizer decode, embedding, transformer encoder, transformer decoder, attention, RoPE, MLP, RMSNorm, residual connection, vocabulary head, sampling, pretraining, supervised fine-tuning, post-training, prefill, autoregressive decode, KV cache, activations, gradients, optimizer.

Self-check: tokenizer encode is not a transformer encoder; tokenizer decode is not a transformer decoder. A GPT-style decoder-only model does not need a separate encoder. Prefill/decode are inference phases; post-training is not.

15-30 minutes: text is not the model’s input type

Use your usual Python interpreter for this standard-library-only demonstration:

text = "Book a room at 3pm."
ids = list(text.encode("utf-8"))
print("text:", text)
print("byte IDs:", ids)
print("round trip:", bytes(ids).decode("utf-8"))

for text in ("room", "Room", " room", "会议室", "🙂"):
    ids = list(text.encode("utf-8"))
    print(repr(text), len(text), "characters;", len(ids), "byte IDs")

This is a byte-level teaching tokenizer, not Qwen’s tokenizer and not BPE. It makes the text/ID boundary visible without a model download. Production subword tokenizers merge learned byte/text patterns; one character need not be one token, and leading spaces/case may change the segmentation.

Prediction first: which examples will have more byte IDs than characters? Check: exact round-trip restoration. Later, repeat with the checkpoint’s actual tokenizer and compare IDs, lengths, special tokens and chat templates. Do not replace a pretrained model’s tokenizer with this byte tokenizer.

30-45 minutes: narrate one generated label

Prompt: Classify: The room picker spins forever.

Without claiming the model will answer correctly, trace the mechanics:

A chat template may add role markers and generation boundaries.
The tokenizer produces IDs; the embedding table looks up vectors.
Decoder blocks contextualize those vectors with causal attention and MLPs.
The vocabulary head scores possible next IDs.
Sampling or greedy selection chooses one ID.
Tokenizer decode makes generated IDs readable.
If generation continues, the chosen ID goes back into the embedding lookup.

Explain aloud: where are weights used? Where is the KV cache created? What changes when the answer is the short label BUG, rather than an essay? Do not assume BUG is a single token until the actual tokenizer is inspected.

45-60 minutes: prefill and decode are different jobs

Imagine a prompt with 128 tokens and an answer with 16 tokens.

prompt_tokens = 128
output_tokens = 16
without_cache = sum(prompt_tokens + step for step in range(output_tokens))
with_cache = prompt_tokens + output_tokens - 1
print("forward-pass token positions without cache:", without_cache)
print("new forward-pass token positions with cache:", with_cache)

Expected counts: 2,168 versus 143 under this simplified full-recomputation versus incremental-KV comparison. These are not elapsed-time predictions.

Prefill processes all 128 prompt positions and supplies logits for the first answer token. Fifteen subsequent incremental decode passes produce the other 15 answer tokens. The final sampled token does not need another forward pass unless generation continues.

KV caching avoids recomputing earlier keys/values and their earlier token representations. It does not make attention to a growing context free, remove the model weights, or guarantee a 15-fold speedup.

60-75 minutes: teach-back without the diagrams

Record a two-minute explanation or write six sentences:

Why does decoder-only not mean “there was an encoder earlier”?
What enters and leaves the embedding layer?
How are attention and the MLP different?
What does the causal mask prevent during training and prefill?
What is cached, and what is not learned during generation?
Which operation would actually change the model’s parameters?

Pass gate: correctly classify all four categories and trace the loop without claiming a tokenizer is the neural decoder, KV caching updates weights, or a fast label proves a new architecture.

If one answer is unclear, repeat only that part. Do not install a training stack to compensate for a missing mental model.

Save this checkpoint

Keep answers and observations in your personal learning notes. No notebook, private source messages, model weights or training run is required today. Update progress.md in Otty with your answers, pass-gate result and next action. If the gate is not met, repeat the confusing section. If it is met, continue to Session 02: tokens and templates. Return to the journey map on your next visit.