The experiment is the teaching vehicle; understanding is the deliverable. By the end, you should be able to trace a prompt through a transformer, distinguish its components from its training/inference phases, change a small model’s behavior with supervised fine-tuning, and explain the measured result. This is Karpathy-inspired learning by construction, not a claim that Karpathy designed or endorsed this course.
Where do I start or resume? Open your learning map. It links all sixteen stops, with materials, exercises, pass gates, recovery instructions and the next stop. Bookmark it rather than using this long reference page as your daily starting point.
On your first visit, go directly to the first-session worksheet. It needs no model download, installation or calculus. On later visits, consult your checkpoint and resume at the first unpassed gate.
| Open locally | The question it answers |
|---|---|
| One token through a decoder-only model | Where do tokens, embeddings, blocks, logits, sampling and the KV cache fit? |
| Inside one decoder block | How do attention, position, normalization, MLPs and residuals work together? |
| Training versus inference | Which stage changes weights, and which only uses them? |
Use the Live/Still control for motion. The maps are conceptual teaching schematics, not timing traces or visualizations of this machine’s actual activations. The block drawing shows the main branch; its cards explain both residual bypasses. The maps are deliberately separate so that phases do not look like physical layers.
The curated visual-materials ledger contains original-author videos, interactive explanations and code. Use them as the watch/read part of each session, not as a giant prerequisite playlist.
The capstone is a small, local, domain-specialized
classifier: given meeting-room feedback, emit BUG,
FEATURE or UNCLEAR, without a reasoning essay.
It is an educational example, not a production classification or
evidence-investigation system.
The domain provides something concrete to label and debug. It is not the main subject of the course. Use handwritten synthetic examples first; later use public, permitted examples with independently checked labels. Do not copy private mail, Teams messages, telemetry or feedback exports into the course.
“System 1-like” is a bounded behavioral/operating goal: specialized decisions, short outputs, measured accuracy and measured latency. It is not a universally defined transformer architecture. Fine-tuning a model does not automatically make it faster, guarantee calibrated confidence, or reproduce Jev’s proprietary training recipe.
The actual comparison is: what changed when we specialized this checkpoint, and which change caused the difference? Jev is background motivation, not the benchmark target or a substitute for public model internals. Its vendor actually claims a different architecture, parallel sampling and RLCD, not just ordinary LLM fine-tuning. Public disclosures are insufficient to reproduce those internals. See the original TypeSafe article and the evidence boundary.
| Term | What it actually is | What you should be able to demonstrate |
|---|---|---|
| Tokenizer | Software/vocabulary that maps text to IDs and IDs back to text | Show actual IDs, leading-space effects, special tokens and round-trip behavior. It is not the neural network. |
Tokenizer encode / decode |
Two operations on the text/ID boundary | Explain why neither implies a transformer encoder or decoder stack. |
| Chat template | Checkpoint-specific formatting of roles, special tokens and assistant boundaries | Show the exact text/IDs produced from the same user message. A formatting mismatch can ruin an otherwise valid checkpoint. |
| Embedding table | Learned lookup from a token ID to a vector | Inspect vocabulary size and hidden width; distinguish integer IDs from floating-point vectors. |
| Position information / RoPE | How position affects attention; Qwen-style RoPE acts on queries and keys | Compare the same words at different positions. Do not draw RoPE as a generic extra tokenizer or assume additive positions everywhere. |
| Attention / Q, K, V | Content-dependent mixing of permitted token information | Point to the query, key and value projections, causal mask, multiple heads and output projection in code. |
| Multi-head / grouped-query attention | Multiple query views; GQA shares fewer KV heads across query-head groups | Read both head counts from the checkpoint and explain why cache size depends on KV heads. |
| MLP / feed-forward / SwiGLU | Per-token feature transformation with learned projections and nonlinearity/gating | Change a token’s feature vector without mixing token positions here. Contrast this with attention. |
| Normalization / RMSNorm | Scale regulation inside and around blocks | Find the pre-attention, pre-MLP and final norms in the implementation. |
| Residual connection | An addition that preserves a saved earlier representation | Trace the two saved branches in one block; do not describe the block as a simple replacement pipeline. |
| Transformer encoder | Typically a stack with bidirectional self-attention, useful for representations/classification | Contrast its attention permissions and output head with a causal decoder. |
| Transformer decoder | In GPT/Qwen-style decoder-only models, a causal stack producing contextual representations | Explain why it does not require a separate encoder. Original encoder-decoder transformers also include cross-attention; not every decoder-only stack does. |
| Vocabulary / LM head | Projection from the final representation to vocabulary logits | Inspect final vector versus vocabulary-wide scores. The head may share weights with embeddings; check rather than assume. |
| Logits / probabilities / sampling | Scores, their normalized distribution, and a rule for choosing an ID | Compare greedy and temperature-controlled generation. A fluent output is not a correctness certificate. |
| Prefill | The inference phase that processes prompt positions and initializes state | Distinguish prompt-token work from output-token work and identify the first generated token. |
| Autoregressive decode | Subsequent generation steps using earlier context | Produce the same greedy tokens with and without a cache, within numerical tolerance. |
| KV cache | Temporary per-layer keys/values for prior positions | Inspect growth with context length; distinguish this from learned weights, external retrieval and the operating system’s file cache. |
| Pretraining | Learning initial parameters, usually from a next-token objective at broad scale | Train only a tiny educational model from random weights; reuse public pretrained weights for the capstone. |
| Continued pretraining | Optional further language-model learning on domain text | Explain why unlabeled domain text alone does not teach a label taxonomy. Not needed for the first capstone. |
| Post-training | Umbrella for later adaptation/alignment, including SFT and optional preference/RL methods | Separate supervised examples, preference pairs, rewards, distillation and inference-time prompting. |
| LoRA | Adapter-based parameter-efficient learning | Show which parameters are trainable and which stay frozen. LoRA is not a new tokenization method. |
| Quantization | Lower-precision representation/computation, with quality and implementation trade-offs | Compare it separately from learning. A quantized base and an adapted model are different experimental factors. |
Minimum math contract: understand arrays and shapes, lookup, a weighted mixture, a probability distribution and “adjust parameters to reduce error.” No derivation of matrix calculus, softmax derivatives or rotary embeddings is required to pass the core path. Use numbers to inspect behavior, not to prove every formula. Deeper derivations are optional.
Do not force a large useful checkpoint to also be a transparent teaching toy.
| Track | Purpose | Scope |
|---|---|---|
| Glass-box toy | See every component and update; deliberately overfit a few examples | Randomly initialized, two small causal blocks, hidden width about 128, short context, byte vocabulary, under roughly one million parameters. Start on CPU. This is a specification for a later lab, not a trained artifact. |
| Useful small checkpoint | Learn real tokenizer/configuration, inference state and supervised adaptation | A verified small dense Qwen checkpoint, running locally with an Apple-Silicon-supported stack. Do not pretrain it from scratch. |
The byte tokenizer makes the toy understandable. The capstone must
keep the checkpoint’s actual tokenizer, embeddings and chat template
consistent. The selected first checkpoint is
Qwen/Qwen3-0.6B, with MLX-LM for the later
Mac adaptation lab. It exposes 28 decoder layers, RoPE, RMSNorm,
grouped-query attention, gated MLPs and a tied vocabulary head: plenty
of real internals to learn without starting with 8B weights.
“Qwen3.8” remains an unresolved identifier. The
official release includes Qwen3-8B, but choosing 0.6B for
this course does not silently reinterpret your wording. Model
configuration, primary sources, supported command examples and the
training-template caveat are in models and Mac sources.
Planning budget: two 90-minute sessions per week, about 24 guided hours, plus reading, repetition and bounded local runs. Reserve 10-12 calendar weeks so setup and difficult concepts have room; the extended material route is roughly 45-65 hours if you follow the source lessons and exercises deeply. These are planning estimates, not promises of proficiency or training duration. Advance by pass gates, not by the calendar.
Each session: roughly 20 minutes watch/explore, 45 minutes do, 15 minutes teach-back, and 10 minutes record observations. A long source video is a reference to revisit, not an instruction to watch it all before coding.
| Week / session | Watch or inspect | Hands-on work | Pass gate and saved evidence |
|---|---|---|---|
| 1 / 1: the map | Local inference/lifecycle maps; introductory visual overview | Complete the no-install worksheet. Sort components, training phases, inference phases and state. | Explain the entire prompt-to-output loop in two minutes; no tokenizer/encoder/decoder confusion. |
| 1 / 2: tokens and templates | Karpathy tokenizer material and minbpe; the checkpoint
tokenizer documentation |
Compare plain text, whitespace, multilingual text and chat-formatted input; inspect special tokens. Do not train a replacement Qwen tokenizer. | Save one exact rendered prompt, IDs and round trip, with the tokenizer revision. |
| 2 / 3: IDs become vectors | Transformer Explainer; embedding/head sections in a small GPT implementation | Inspect shapes at embedding input/output and the vocabulary head; look up the same ID twice. | Explain [batch, positions], hidden vectors and
vocabulary scores without memorizing a derivation. |
| 2 / 4: attention and position | 3Blue1Brown attention animation; causal-mask code | Highlight allowed reads for three token positions; inspect Q/K/V shapes, RoPE and query/KV head counts. | Show that changing a future token cannot affect an earlier causal position; distinguish attention weights from a guaranteed explanation of a decision. |
| 3 / 5: one complete block | Local block map; MLP visual explanation; actual decoder-layer implementation | Trace normalization, attention, the first residual, MLP and the second residual; compare encoder versus decoder masks. | Reconstruct the block from blank paper and identify one architecture-specific detail from the checkpoint. |
| 3 / 6: inference state | KV-cache documentation and a token-by-token generation example | Compare cached/uncached greedy output at short context; record prompt count, cache growth and output count. | Same output IDs under matched settings; explain prefill versus decode and why cache is not training. |
| 4 / 7: what learning changes | Selected micrograd explanation and GPT training loop | Trace one example through prediction, target, loss, gradient and optimizer update. Intentionally overfit one tiny batch. | Show trainable values actually change while a frozen parameter does not. Explain teacher forcing versus generation. |
| 4 / 8: tiny pretraining | Karpathy’s build-GPT lesson and reference code | Use a tiny synthetic corpus and the glass-box specification. Inspect one batch and shifted targets; train for a short bounded run. | A saved checkpoint reloads and reproduces greedy output. Lower training loss is reported only as toy learning, not generalization. |
| 5 / 9: base versus post-trained | The selected checkpoint’s model card; training/inference lifecycle map | Compare an available base versus instruct checkpoint when genuinely comparable; otherwise compare prompting modes without claiming this isolates post-training. | Explain SFT, optional domain pretraining, preference training, RL and distillation; identify which this experiment will not use. |
| 5 / 10: task and baseline | Label rubric below; encoder/classifier concepts | Build source-grouped train/dev/test splits. Establish majority, simple text-classifier and prompted-Qwen baselines. | Split manifest, duplicate check, independently checked labels and baseline outputs are frozen before tuning. |
| 6 / 11: adapters and masked loss | Upstream MLX-LM LoRA documentation and dataset loader | Convert labels into chat examples; inspect serialized templates; confirm loss covers the intended assistant completion. Check trainable parameter counts. | Demonstrate a known example’s target tokens and why user/system tokens should not be the supervised label target. |
| 6 / 12: the smoke run | Upstream training help/config and local memory view | Run a very short adapter training smoke check on training data; evaluate on dev; save/reload adapter; repeat one prediction. | The entire path works locally, no test-set training/evaluation leakage, and memory is stable. A smoke run does not establish accuracy. |
| 7 / 13: controlled specialization | Your own training/dev loss curves | Change one factor at a time: data, rank, learning rate, or number of updates. Use the dev set to stop and choose. | One reproducible candidate beats or fails to beat the fixed baseline honestly; no selecting checkpoints on test performance. |
| 7 / 14: error anatomy | Confusion matrix and individual dev errors | Review ambiguous/mixed cases, paraphrases, invalid outputs and
apparent memorization. Keep UNCLEAR distinct from numerical
confidence. |
Every claimed gain has paired examples and counts; remaining failures are categorized rather than hidden. |
| 8 / 15: speed and memory | Inference/cache diagram and upstream timing output | Compare baseline/adapter on matched warm settings, prompt lengths and output limits; separately label cold runs. | Report time to first token, end-to-end p50/p95, token counts, memory and quality. Do not infer measured speedup from quantized weight size. |
| 8 / 16: teach it back | All three local maps; the actual model config | Rebuild the explanation without notes and walk another person through code, artifacts and a fresh held-out example. | Another person reproduces the saved run and can locate every core component. Explain both what the experiment proves and what it does not. |
| Sessions | Ready original materials |
|---|---|
| 1 | First-session worksheet; 3Blue1Brown GPT overview |
| 2 | Karpathy tokenizer video; minbpe exercises |
| 3 | Transformer Explainer; makemore MLP if embeddings are unfamiliar |
| 4 | 3Blue1Brown attention; LLM Visualization |
| 5 | 3Blue1Brown MLP; Illustrated Transformer; Illustrated GPT-2 |
| 6 | KV cache from scratch; local inference map |
| 7 | Karpathy micrograd; author code |
| 8 | Karpathy build GPT; compact teaching implementation |
| 9 | Karpathy lifecycle deep dive; Qwen model card |
| 10-12 | Course label rubric; MLX LoRA/data instructions; local source-backed lab route |
| 13-16 | Your measured runs, local maps, actual Qwen implementation and config |
NanoGPT is a deprecated but compact educational reference, not the stack for the useful capstone. Its learned positions and LayerNorm differ from Qwen’s RoPE/RMSNorm. Do not carry those differences across without inspection. These materials are linked, not copied or rehosted.
The planning example is an Apple M4, arm64, 32 GiB unified-memory Mac. Check your own available memory, disk and interpreter before starting. No model weights or training dependencies are included in this course.
Use a separate study environment, not another
application’s .venv. Verify a supported interpreter and
wheels against the upstream stack before creating that environment. Do
not downgrade or repurpose the application’s interpreter. The source-backed Mac route supplies
the relevant upstream references.
| Exercise | Initial planning limit, not a measured benchmark |
|---|---|
| First session and visual explorations | No model, no installation. Browser and standard-library Python only. |
| Glass-box toy | Start with two small blocks, context 64-128 and a tiny batch. Inspect one update first; time-box the first training run to a few minutes. CPU is acceptable. |
| Real inference | Start with a small dense checkpoint and short prompts/output. Run only one model process. Quantization is an optional measured memory trade-off, not an accuracy guarantee. |
| Adapter smoke check | Batch size 1, sequence length initially 128-256, low adapter rank and a small bounded update count. Increase only after observing actual peaks. |
| Useful supervised run | Keep a measured configuration and a hard wall-time budget; use dev quality and memory to choose updates, not an arbitrary epoch count. |
Unified memory is shared by the OS, applications and GPU; a 32 GiB machine does not offer 32 GiB exclusively to weights. Training also needs activations, gradients, optimizer state and framework workspaces. KV-cache size is a different inference allocation. Weight-file size alone is not a fit test.
For a concrete optional calculator, the selected checkpoint’s BF16/FP16 KV storage at batch 1 and 2,048 cached positions is about 224 MiB: two tensors times 28 layers times 2,048 positions times 8 KV heads times 128 numbers per head times 2 bytes. This is a simple storage estimate, not a measured process peak, and excludes weights, activations and other buffers.
Stop conditions: sustained yellow/red memory pressure, growing swap, unresponsive desktop, non-finite loss or inability to reload the checkpoint. Stop the one experiment, keep its settings/error, reduce context/batch/model size, and retry one changed factor. Do not launch a second model concurrently or silently switch to a cloud GPU/hosted teacher.
Do not start with a large mixture-of-experts checkpoint, full-parameter fine-tuning, long-context training, or a CUDA/bitsandbytes recipe assumed to work unchanged on macOS. Educational success does not require any of them.
Choose one convention for mixed cases before annotation. For this
teaching exercise use UNCLEAR when a single label cannot
faithfully represent the example. Do not discover a different convention
after seeing the test errors. This rubric is a classroom simplification,
not a production policy.
Twelve handwritten examples are ready for a formatting/smoke exercise: four per label. They are not a benchmark, not independent real-world evidence and not sufficient to claim a useful domain model. Do not use those examples as both training and test evidence.
For the later useful run, a planning target is about 300 train / 60 dev / 120 test labeled examples, grouped by source/conversation/template family. These counts are curriculum targets, not a collected dataset or a sample-size guarantee. Include realistic paraphrases, ambiguous and mixed cases, and negation. Handwritten template siblings must stay in the same split. Use more data if class coverage or uncertainty requires it.
Freeze a split manifest, data hashes, label rubric and model/tokenizer revision. Exact/near-duplicate and source-group checks come before training. Use training examples for learning, dev for choices/stopping, and test only for the final frozen comparison. If you inspect test errors and alter the model, you need a genuinely fresh final holdout.
The first adapter run uses supervised fine-tuning only. Preserve the base checkpoint; record adapters separately. If the starting checkpoint is already instruction-tuned, call the experiment additional domain SFT—not proof you recreated all of post-training.
Start with short label generation using the checkpoint’s supported
chat format. Inspect label tokenization rather than assuming equal
single-token labels. Treat anything outside the allowed labels as an
explicit invalid output; don’t turn it silently into
UNCLEAR.
No reasoning essay is requested or trained as the output. If the model has a documented thinking/non-thinking switch, set and record it consistently in baseline and candidate. An unexplained template change is a confounder.
Before any longer run, inspect one serialized sample and one tokenized batch: input IDs, target IDs, completion mask, sequence length, padding and EOS. Verify that answer tokens are learned while the intended prompt tokens are not scored as the supervised classification answer. Check truncation has not removed the label. Validate these semantics in the actual dataset loader; a flag name alone is not evidence.
Record which attention/MLP projections receive adapters, trainable versus frozen parameter counts, learning rate, rank, batch/accumulation, sequence length, random seed, update count and package versions. The chosen adapter targets should come from the actual checkpoint/module names.
If labels improve but free-form chat quality degrades, report the trade-off. Specialization is not a claim that every general capability improves.
There are two separate report cards:
| Learning outcome | Evidence |
|---|---|
| Know the components | Draw the token-to-text path and one block, then find the corresponding tensors/modules in code. |
| Know the phases | Explain why post-training changes weights while prefill/decode use them; locate all temporary state. |
| Know how training works | Show one batch, one loss computation, nonzero gradients and one update; reload a checkpoint. |
| Know what fine-tuning changed | Compare fixed prompts with base versus adapter and identify exactly which parameters were learned. |
| Know the limits | Correctly distinguish observed facts from architecture guesses, toy memorization and real generalization. |
| Model outcome | Evidence |
|---|---|
| Classification | Per-class precision/recall, macro-F1, confusion matrix, class counts and invalid-output rate on the frozen holdout. |
| Ambiguity handling | Separate error accounting for ambiguous/mixed/OOD challenge cases.
UNCLEAR is a task label; it is not automatically calibrated
abstention. |
| Reliability | Reloaded model/adapter reproduces greedy predictions under pinned settings; repeated seeds or runs when feasible. |
| Latency | Same hardware, prompt distribution, output budget, template, cache policy and precision; report warm/cold separately with sample counts. |
| Memory | Measured process/framework memory and system memory pressure, with context/batch/model details. Framework counters are not total unified-memory usage. |
| Causal attribution | At least one controlled comparison isolates adapter training; separate comparisons isolate model size, output length and quantization. |
Set numeric model acceptance thresholds after observing the baseline and before tuning, according to the chosen domain’s costs. Do not invent a 90%-accuracy requirement without a meaningful dataset. If the adapter doesn’t beat the simple baseline, the experiment can still be an excellent learning result; explain why rather than manufacturing a win.
Minimum result bundle for the capstone: pinned configuration, dataset/split manifest and provenance, training/dev curves, adapter/checkpoint identity, held-out predictions, metric computation, latency samples and a short error review. Another person should be able to reproduce the comparison without trusting either your confidence or an LLM judge.
Ready now: the self-guided learning map, terminology map, four local animated diagrams, the no-install first-session worksheet, original-source material ledgers and twelve synthetic formatting examples.
Not done: model download, environment installation, toy implementation, training, capstone dataset collection, accuracy/latency measurement or a claim of reproducing Jev. Each belongs to a later competency-gated session. The linked upstream examples are reference implementations, not code that has been installed or executed in this preparation.
Return to your learning map, finish the current stop’s pass gate, then follow its next-stop link. The whole course is deliberately prepared without requiring a foundation-model training run or paid hardware.
Track continuity in the course checkpoint. No learner session is marked complete merely because the preparation generated these materials.