Transformer course: models-and-mac

Source verification: October 30, 2026. No model/package installation or training benchmark was performed. Recommendations below are feasibility choices, not measurements.

Identity decisions

Jev’s vendor claims more than small-model fine-tuning. The freshly fetched TypeSafe introduction describes a new architecture, parallel sampler and Reinforcement Learning for Calibrated Decisions (RLCD), with typed probabilistic decisions instead of ordinary autoregressive string generation. These are vendor disclosures, not independently reproduced mechanisms.

That article does not provide the layer-level architecture, parameter count, backbone, objective equations, reproducible training recipe or downloadable weights needed for an exact reproduction. Its “can’t hallucinate” argument concerns type/schema conformity, not guaranteed factual correctness. “System One” invokes Kahneman’s distinction, not a fully specified model class. A Qwen adapter can reproduce an educational behavior target, not Jev’s undisclosed architecture or RLCD.

“Qwen3.8” is unresolved. The official Qwen3 release lists Qwen3-8B, Qwen3-1.7B and Qwen3-0.6B; this does not establish an official model named Qwen3.8, nor prove what the user intended. Do not silently substitute 8B.

Use Qwen/Qwen3-0.6B for the first real-checkpoint lessons. This is an explicit curriculum choice, not a correction of the user’s identifier. The small dense model is sufficient to inspect a real modern decoder and adapt short labels; consider 1.7B or 8B only after measured need/headroom.

What is inspectable in Qwen3-0.6B?

Property	Official card/config or maintainer implementation
Family	Dense causal decoder-only language model; public pretrained and post-trained checkpoint
Layers / hidden width	28 / 1024
MLP intermediate width	3072
Query / KV heads	16 / 8: grouped-query attention
Attention head dimension	128
Vocabulary	151,936 entries
Position / normalization	RoPE / RMSNorm; implementation also normalizes Q and K
Output head	Configuration uses tied embeddings
License	Apache-2.0 per official release/card

These are not a generic architecture specification for every Qwen/GPT model. The official card says 32,768-token context while the inspected config has max_position_embeddings: 40960; neither is the initial training budget. Start short. Do not infer safety or memory fit from a context-limit number.

Qwen3 has documented thinking/non-thinking modes. The hard template switch enable_thinking=False differs from a soft /no_think instruction. A bounded non-thinking output is a useful teaching experiment; it does not prove a different model architecture or that the network performs no internal computation.

Why MLX-LM, not a copied CUDA recipe?

Apple’s MLX-LM supports Apple Silicon inference and LoRA fine-tuning, contains a native Qwen3 implementation, and supports adapting linear layers including quantized linear layers. Its maintained implementation is stronger evidence than an older documentation family list omitting Qwen3.

Qwen’s own MLX page says it is awaiting a Qwen3 update. Use current Apple’s CLI/implementation references below for command syntax, not the older example.

The inspected MLX-LM package metadata requires Python >=3.11 and lists 3.11-3.13 classifiers. That does not establish working wheels for this Python 3.14. Plan a separate arm64 Python 3.12 environment; check wheel/dependency availability at setup time and freeze resolved versions. Do not change another application’s .venv.

Commands for a later lab, not commands executed in this preparation

Run inside the dedicated study directory after checking an installed supported interpreter. runs/ and models/ are local experiment output directories. This is not a fully pinned installation until dependency resolution and the checkpoint revision have been recorded.

python3.12 -m venv .venv
.venv/bin/python -m pip install "mlx-lm[train]"
mkdir -p runs
.venv/bin/python -m pip freeze > runs/environment.freeze.txt

Inspect the upstream version and --help before using flags. The example uses greedy selection to make a bounded comparison reproducible; it is not Qwen’s recommended sampling configuration for all general-purpose chat.

.venv/bin/mlx_lm.generate \
  --model Qwen/Qwen3-0.6B \
  --prompt "Return only BUG, FEATURE or UNCLEAR. Feedback: The room picker spins forever." \
  --chat-template-config '{"enable_thinking": false}' \
  --temp 0 --max-tokens 16 --seed 0

Use one JSON object per line in runs/data/train.jsonl, runs/data/valid.jsonl and runs/data/test.jsonl:

{"messages":[{"role":"system","content":"Return only BUG, FEATURE or UNCLEAR."},{"role":"user","content":"The room picker spins forever."},{"role":"assistant","content":"BUG"}]}

The repository’s twelve raw synthetic examples are not already this training format or a train/dev/test benchmark. Convert and split only at the data session, under the predefined source-group rules.

Before any smoke training, inspect the serialized template and token/loss mask boundary. In the inspected MLX-LM loader, --mask-prompt masks the prefix before the final message; it does not automatically mask every assistant turn in a multi-turn transcript. End each example with the target assistant answer. Raw text datasets do not provide this prompt-masking path.

Important Qwen template gate: the loader does not forward the inference CLI’s enable_thinking option. Static inspection of the cited versions shows that a plain final assistant answer is serialized after an empty <think>...</think> block. The masked training completion includes that block, the label and the turn-ending token. At inference, enable_thinking=False puts the same empty block in the generation prefix and starts generation at the answer. That conditioning is consistent for this example, but the supervised completion is not literally label-only.

This correspondence was established by reading the template/loader, not by executing tokenization. Inspect rendered IDs and the loss-mask boundary before training, particularly after version changes. Do not silently strip template tokens, alter the tokenizer or assume the inference flag changed the loader. No longer run is authorized by the course until these semantics are understood.

.venv/bin/mlx_lm.lora \
  --model Qwen/Qwen3-0.6B \
  --train --data ./runs/data \
  --fine-tune-type lora \
  --adapter-path ./runs/adapters-qwen3-06b \
  --batch-size 1 --num-layers 4 \
  --max-seq-length 256 --grad-checkpoint \
  --mask-prompt --iters 20 \
  --steps-per-eval 10 --val-batches 2 \
  --save-every 20 --seed 0

Record the resolved default adapter rank/targets and learning rate. Increase updates only after reload, stable memory, valid masking and dev evaluation. Do not mistake twenty updates on handwritten examples for a trained domain model.

.venv/bin/mlx_lm.lora \
  --model Qwen/Qwen3-0.6B --adapter-path "" \
  --data ./runs/data --test --test-batches -1 \
  --batch-size 1 --max-seq-length 256 --mask-prompt

Repeat with the saved adapter path only for the final matched comparison. These CLI loss/perplexity outputs do not compute the course’s macro-F1, invalid-label rate, or latency distribution; those require the later explicit task evaluator. Keep the final test set untouched while choosing settings.

.venv/bin/mlx_lm.convert \
  --model Qwen/Qwen3-0.6B \
  --mlx-path ./models/qwen3-06b-4bit \
  -q --q-bits 4 --q-group-size 64

Use the same base representation/revision consistently for training and evaluation. MLX’s quantized adaptation route is not permission to assume a CUDA/bitsandbytes QLoRA tutorial runs unchanged on macOS.

Resource boundaries

Start at 0.6B, batch 1, context 128-256 and a few adapted layers. The first run establishes actual headroom. There is no measured M4 speed, memory or accuracy claim here. Run one model/training process at a time; stop on sustained memory pressure or growing swap. Do not use the full advertised context length or all 32 GiB as a training allocation.

Primary-source ledger