Journey map · Course reference.
Verified against original-author pages/source during the October 30, 2026 preparation. These are links, not copied diagrams/videos/code. Study budgets below are recommendations, not video lengths or verified timestamps. The complete extended route is roughly 45-65 hours; the core course uses selected sections, not every minute of every resource.
Grant Sanderson’s original lessons:
Use the token, embedding, attention and MLP sections. Follow arrows and shapes first; skip derivations on the first pass. Return to one operation when the code is confusing. The account of facts in MLPs is an illustrative view, not a complete or exclusive explanation of where a model’s knowledge lives. Extended budget: 4-6 hours.
Original building-micrograd video and author code.
Use the computation graph, parameters, gradient and optimizer-update explanations. The core path needs one understandable update; manually deriving every derivative is optional. A finite-difference comparison is a good later correctness exercise. CPU-scale work. Extended budget: 4-6 hours.
Use these when embeddings, tensor indexing, sampling, loss or train/dev/test splits are unfamiliar. Build only a bounded bigram/MLP example before the transformer. Parts 3-5 are optional mathematical depth, not an entry gate. Extended budget: 7-10 hours.
Original tokenizer video, minbpe, and author exercises.
Learn UTF-8 bytes, BPE merges, vocabulary and special tokens. Try exercise Steps 1-2 on a tiny synthetic corpus. Compare spaces, case, identifiers, numbers, multilingual text and emoji, then verify round trips. Do not replace a pretrained checkpoint’s tokenizer as part of fine-tuning. Extended budget: 5-7 hours.
Original build-GPT video and compact nanoGPT model implementation.
Trace causal attention, heads, the MLP, residual connections, normalization, embeddings, positional information, logits and loss. Inspect one small batch and train a tiny educational model, not the large GPT-2 reproduction recipe. The author’s reduced-computer guidance is in the README. Extended budget: 10-14 hours.
Two boundaries matter: nanoGPT is now marked deprecated by its author and is used here as a compact teaching reference, not the production training stack; its introductory generation loop recomputes context and does not teach a full KV-cache implementation. Also, its learned positional embeddings and LayerNorm are not identical to Qwen’s RoPE and RMSNorm.
Interactive author site and project demonstration video.
Created by Aeree Cho and Georgia Tech collaborators. Inspect embeddings, transformer blocks, output probabilities and architectural details. Change one prompt word and follow affected representations. Change temperature separately to distinguish output sampling from learning. It visualizes GPT-2, not the exact Qwen checkpoint. Extended budget: 2-3 hours.
Interactive walkthrough and author repository.
Follow the tiny working letter-sorting model through embedding lookup, normalization, attention, the MLP, residuals and the output head. Match a visible operation to the small GPT code. This is especially valuable because it makes a genuine transformer inspectable without requiring a useful general-purpose language model. Extended budget: 2-3 hours.
The Illustrated Transformer and The Illustrated GPT-2.
Read the original transformer’s high-level/decoder sections, then
GPT-2’s decoder-only and evaluation-time sections. Identify which
cross-attention sublayer is unnecessary when there is no separate
encoder. Tokenizer encode does not mean there is a
transformer encoder. Extended budget: 3-4 hours.
Use the redundancy, caching and prefill-versus-decode sections to fill the gap left by introductory GPT code. On a tiny causal block, compare the newest position’s full-prefix result with incremental cached computation. Disable dropout and preserve token positions. No VLM training is necessary. Reported speedups belong to the article’s workload, not this Mac. Use executable causal-masking code as the operational reference rather than treating a compact illustrative equation as a complete implementation. Extended budget: 4-6 hours.
Deep Dive into LLMs like ChatGPT.
Use the author-named chapters on pretraining to post-training, conversation data, supervised fine-tuning to reinforcement learning, RL, and RLHF. Classify examples of raw text, supervised demonstrations and preferences. This is lifecycle orientation, not a local fine-tuning command tutorial. Extended budget: 4-6 hours.
Primary lesson pages, author repositories, relevant implementation sections and original video metadata/descriptions were checked. Interactive interfaces were source-verified, not manually exercised on this Mac. Links use moving upstream pages; pin the actual code/package revision when starting a lab.
The local course copies none of these authors’ videos, diagrams or code. Link-only curation is not a license grant to redistribute their work.