One pip install and one command, and a real language model is running on your M4 Pro's GPU — no cloud, no API key, no rented GPU. This is the foundation: every other thing you'll do (quantize, serve, fine-tune) is a variation on the command you'll run here.
mlx-lm turns a Hugging Face model id into local GPU inference in one line. Because Apple Silicon has unified memory (CPU and GPU share the same RAM), there's no "copy the model to the GPU" step like CUDA or PyTorch's .to("mps") — the model is loaded once and the GPU computes on it in place. You point at a model, you get tokens.
Your mission spans four skills — run, quantize, serve, fine-tune. They look different, but under the hood they're all the same move: load a model, then do something with it. mlx_lm.convert loads-and-requantizes, mlx_lm.server loads-and-serves, mlx_lm.lora loads-and-adapts. Master "load and generate" first and the rest are recombinations of parts you already understand.
I've already created a practice environment on your Mac so every example runs locally on your GPU. It's a Python virtual environment (an isolated package sandbox) at ~/projects/learn/public/courses/mlx-lm/practice/.venv with mlx-lm installed. The one-time install was just:
# one-time (already done for you):
python3 -m venv ~/projects/learn/public/courses/mlx-lm/practice/.venv
~/projects/learn/public/courses/mlx-lm/practice/.venv/bin/pip install mlx-lm
Verify it's live — run this and you should see versions plus your GPU as the default device:
~/projects/learn/public/courses/mlx-lm/practice/.venv/bin/python -c "import mlx_lm, mlx.core as mx; \
print('mlx_lm', mlx_lm.__version__); print('device:', mx.default_device())"
# verified on your machine just now: mlx_lm 0.31.3 device: Device(gpu, 0) ← your Metal GPU is the default; no setup needed
pip install a CUDA build and then explicitly move tensors to the GPU. On Apple Silicon there's one pool of RAM shared by CPU and GPU, so mlx-lm just installs and the GPU is already the default device. The number that matters is how much RAM the model needs — that's your real constraint, and why we start with tiny 4-bit models. See MLX docs — unified memory.This is the whole lesson in a single line. It downloads a small model from Hugging Face the first time, then generates on your GPU:
~/projects/learn/public/courses/mlx-lm/practice/.venv/bin/mlx_lm.generate \ --model mlx-community/Llama-3.2-1B-Instruct-4bit \ --prompt "In one sentence, what is Apple's MLX framework?" \ --max-tokens 60
Real output from your M4 Pro (first run also showed a one-time download of 6 files):
========== Apple's MLX framework is a set of hardware and software components that allow developers to create custom, low-level interfaces for interacting with the Apple M1 and M2 processors. ========== Prompt: 47 tokens, 97.166 tokens-per-sec Generation: 36 tokens, 290.721 tokens-per-sec Peak memory: 0.808 GB
========== banners with text between them and a tokens-per-sec line, your Apple GPU just ran a language model end-to-end — downloaded, loaded into unified memory, and decoded token by token. That's Lesson 1 done. Everything below explains what you saw.--model.mlx-lm prints a tiny dashboard after every generation. Learn to read it now; you'll use these numbers to decide what fits and how fast it'll be for the rest of the course.
Two distinct phases hide in those numbers. Prefill processes your whole prompt at once (parallel, compute-bound). Decode emits one token at a time, each depending on the last (sequential, memory-bandwidth-bound) — which is why generation speed barely changes with prompt length but collapses if the model is too big for fast memory.
That string mlx-community/Llama-3.2-1B-Instruct-4bit isn't arbitrary — every part tells you something, and you'll be reading these for the rest of the course.
| Part | Meaning | Why it matters |
|---|---|---|
mlx-community/ | The HF org of pre-converted MLX models | Already in MLX format — no conversion step. browse them |
Llama-3.2-1B | Base model + 1 billion parameters | Param count ≈ how much it "knows" and how much RAM it needs |
Instruct | Tuned to follow instructions / chat | Vs. a base model that just continues text — Instruct = answers prompts |
4bit | Quantized to 4 bits per weight | ~4× smaller & faster than 16-bit, slight quality loss — your Lesson 4 topic |
--model <hf-id-or-local-path> and -h shows its options (try ~/projects/learn/public/courses/mlx-lm/practice/.venv/bin/mlx_lm.generate -h). Same mental model, different verb.One-shot text from a prompt. ← you are here.
Interactive REPL; keeps conversation context. (Lesson 3)
Pull & quantize a model to 4/8-bit. (Lesson 4)
OpenAI-compatible local server. (Lesson 5)
Fine-tune with LoRA/QLoRA. (Lesson 6)
load + generate in code. (Lesson 2 — next)
Quick self-test — think first, then expand. (Stuck or curious? Ask me, your teacher — that's what I'm here for.)
.to("mps")?A: Apple Silicon uses unified memory — CPU and GPU share one RAM pool, and MLX's default device is already the GPU. The model is loaded once and computed in place; there's nothing to copy across a bus. The real constraint isn't "did I move it to the GPU?" but "does it fit in RAM?"
A: 97 tok/s is prefill (reading your prompt — all tokens processed in parallel) and 291 tok/s is decode (generating the answer one token at a time). They measure different phases; decode is the speed you "feel" as the answer streams out.
Instruct and 4bit in the model id tell you?A: Instruct = the model was tuned to follow instructions / chat (so it answers your prompt instead of just continuing text). 4bit = the weights are quantized to 4 bits each, making it ~4× smaller and faster than full precision, at a small quality cost.
A: No download — the model is cached under ~/.cache/huggingface, so it loads straight from disk into unified memory and starts generating in seconds. The download only happens the first time per model.
Keep handy: Glossary & quick reference (CLI map, the canonical Python snippet, every term in one printable page) · MISSION.md. Next up → Lesson 2: the Python surface (load / generate / stream_generate) — the same model, now callable from your own code.