Make a model yours. With LoRA you don't retrain the model — you train a tiny overlay that nudges its behavior. In this lesson you fine-tune the 1B model on your M4 Pro until it changes how it talks, all from an 11 MB file you made. This is the last skill of the arc: run → serve → fine-tune.
🎯 Mission step: adapt a model to your own data with LoRA / QLoRALoRA freezes the model and learns a thin, low-rank overlay instead of new weights. A weight matrix W has millions of numbers; LoRA writes the change you need as ΔW = B·A, two skinny matrices that squeeze through a tiny rank. You train only A and B — here, 0.228% of the weights — and get back a small adapter you can attach, swap, or bake in.
Full fine-tuning nudges every number in the model and hands you a whole new multi-gigabyte copy. LoRA's bet is that the change needed to adapt a model is "small" — it lives in a low-dimensional subspace — so you can force the update to be low-rank and train almost nothing.
W is the printed textbook you don't reprint; A and B are a thin transparency of margin notes laid on top. With rank r=8 and dimension d=4096, you train r·(d+d) ≈ 65K numbers instead of d² ≈ 16.7M — a ~250× cut, per matrix.B·A is the same shape as W, but its rank (independent directions of change) is capped at r. That's plenty to capture a style or a task, while costing a rounding error's worth of parameters. α/r is just a scale that keeps the overlay's strength stable as you change r..jsonlFine-tuning needs a data folder with train.jsonl (required) and optionally valid.jsonl / test.jsonl. Each line is one JSON example. mlx-lm accepts three shapes (LORA.md):
| Format | Line shape | Use when |
|---|---|---|
| chat | {"messages": [{"role","content"}, …]} | multi-turn / assistant behavior |
| completions | {"prompt": "...", "completion": "..."} | input → output pairs ← we use this |
| text | {"text": "..."} | raw style / domain absorption |
For this lesson we taught a pirate persona with ~80 completion pairs. One real line from your ~/projects/learn/public/courses/mlx-lm/practice/lora-data/train.jsonl:
{"prompt": "How do plants grow?",
"completion": "Arr, they drink sun and rain and sprout like rigging up a mast, ye landlubber!"}
Point mlx_lm.lora at the base model and your data:
~/projects/learn/public/courses/mlx-lm/practice/.venv/bin/mlx_lm.lora \ --model mlx-community/Llama-3.2-1B-Instruct-4bit \ --train --data ~/projects/learn/public/courses/mlx-lm/practice/lora-data \ --adapter-path ~/projects/learn/public/courses/mlx-lm/practice/pirate-adapters \ --iters 300 --batch-size 4 --num-layers 8
Because the base is already 4-bit, you're training a LoRA on a quantized model — that's QLoRA. Real log from your M4 Pro:
Trainable parameters: 0.228% (2.818M/1235.814M) # ← the whole point Iter 1: Val loss 5.816 Iter 50: Train loss 1.439, It/sec 2.78, Tokens/sec 734, Peak mem 1.957 GB Iter 300: Train loss 0.051, Val loss 0.046 Saved final weights to .../pirate-adapters/adapters.safetensors
adapters/ folderTraining writes two files (plus periodic checkpoints like 0000300_adapters.safetensors):
The recipe — how to re-attach the weights. Real values: rank: 8, scale: 20.0, num_layers: 8, fine_tune_type: lora.
The learned weights — just the A/B matrices. 11 MB vs the base's gigabytes. This is the whole "trick" you trained.
Now attach it at generation time with --adapter-path and watch the same model change. Both runs are real, on a held-out prompt that was not in the training data:
# BEFORE — base model $ mlx_lm.generate --model …Llama-3.2-1B-Instruct-4bit \ --prompt "What is machine learning?" Machine learning (ML) is a subset of artificial intelligence (AI) that involves training algorithms to learn from data...
# AFTER — same model + your 11 MB adapter $ mlx_lm.generate --model …Llama-3.2-1B-Instruct-4bit \ --adapter-path ~/projects/learn/public/courses/mlx-lm/practice/pirate-adapters \ --prompt "What is machine learning?" Machine learning be the study o' how machines learn from data, rather than bein' programmed to do so.
mlx_lm.generate --model …4bit --adapter-path ~/projects/learn/public/courses/mlx-lm/practice/pirate-adapters --prompt "Give me advice about money." → "Arr, hardtack and barrels, matey — spend yer money wisely as ye sail the seas!"Two ways to ship a LoRA. Keep it separate (--adapter-path) — one base in memory, swap adapters cheaply. Or fuse it into the weights for a standalone model you can serve (Lesson 3) or upload:
~/projects/learn/public/courses/mlx-lm/practice/.venv/bin/mlx_lm.fuse \ --model mlx-community/Llama-3.2-1B-Instruct-4bit \ --adapter-path ~/projects/learn/public/courses/mlx-lm/practice/pirate-adapters \ --save-path ~/projects/learn/public/courses/mlx-lm/practice/pirate-fused
fuse dequantizes, adds B·A, then re-quantizes back to 4-bit, and rounding to 4 bits washed out the small low-rank delta. The fix is --dequantize, which keeps the fused model full-precision so the overlay survives:
# fp16 fused model — NO adapter flag, and it's a pirate: $ mlx_lm.fuse … --dequantize --save-path ~/projects/learn/public/courses/mlx-lm/practice/pirate-fused-fp16 $ mlx_lm.generate --model ~/projects/learn/public/courses/mlx-lm/practice/pirate-fused-fp16 --prompt "What is machine learning?" Machine learning be the study o' how machines learn from data...
| Approach | Size | Pirate? | When |
|---|---|---|---|
--adapter-path (attach) | 11 MB adapter + base | ✓ yes | dev, many adapters, swap freely |
fuse (default, re-quantizes 4-bit) | 663 MB | ✗ washed out | — avoid for small deltas on quantized bases |
fuse --dequantize (fp16) | 2.3 GB | ✓ yes | standalone model to serve / share |
--dequantize (then re-quantize deliberately with mlx_lm.convert, your next lesson, if you want it small again).| Flag | Default | What it trades |
|---|---|---|
--num-layers | 16 | How many of the model's layers get an adapter. More = more capacity + more params. (We used 8.) |
rank (config) | 8 | Width of the bottleneck. Higher = more expressive overlay, bigger adapter. 8–16 is typical. |
scale α (config) | 20 | How strongly the overlay is applied (α/r). Higher = more aggressive adaptation. |
--iters | — | Training steps. Too few = no effect; too many = overfit. Watch val loss stop falling. |
--batch-size | 4 | Examples per step. Bigger = smoother + more memory. |
valid.jsonl) falling too = it's generalizing, not memorizing. When val loss flattens or rises while train keeps dropping, you're overfitting — stop. Ours went 5.8 → 0.046 on val: healthy.Think first, then expand. (Want to fine-tune on your data — your writing, a tool's output format? Ask me and we'll design the dataset.)
A: You didn't save a model — you saved the A/B matrices, which are ~0.228% of the weights. The frozen base is unchanged and reused; the adapter is just the low-rank delta.
A: It generalized rather than memorized — the LoRA captured a consistent transformation (voice/format) that applies to new inputs. That's the goal; a healthy falling val loss is the signal it's happening.
A: The default fuse re-quantizes to 4-bit, and rounding erased the small low-rank delta. Fuse with --dequantize to keep it full-precision (then quantize on purpose later if you want it small).
A: Attach with --adapter-path. Keep one base loaded and swap 11 MB adapters per request — far cheaper than three fused multi-GB models.
Keep handy: Glossary & quick reference (now includes LoRA/QLoRA, rank, adapter, fuse) · ← Lesson 3. You've now done the whole arc — run · serve · fine-tune. Next up → Lesson 5: mlx_lm.convert, the quantization lesson, which also closes the loop on that fuse gotcha: take your fp16 fused pirate and shrink it back to 4-bit on purpose.