mlx-lm · Lesson 12 · Capstone

Ask Your Own Notes — Local RAG

The whole course in one tool. Embed your documents, find the passages that answer a question, and have a local model write the answer grounded in your text — all on your Mac. We build it over this very course's lessons, then you point it at anything.

🎯 Mission complete: a real local-LLM tool over your own content
The one idea

RAG = retrieve (Lesson 11) + generate (Lesson 2). A model can't know your private notes — so don't ask it to remember. Find the relevant passages by meaning, put them in the prompt, and tell the model to answer using only those. You get current, grounded answers from a small local model, with no training and no cloud.

01 The whole course, composed

This capstone is every prior lesson working together: you run a model (L1–2) at a size that fits (L9), use embeddings to retrieve (L11), and generate a grounded answer (L2) — optionally as structured output (L8) or via tools (L10). We'll index this course's own lessons and ask it questions about what you learned.

INDEX (once) your docs → chunks embed → vectors ASK (each query) question → embed retrieve top-k nearest chunks cosine model context + question → grounded answer
Index once (chunk → embed → store), then per question: embed it, retrieve the nearest chunks by cosine similarity, and hand them to the model as context. The model answers from what you retrieved, not from memory.

02 The code — RAG over this course

Strip the lessons to text, chunk into overlapping windows, embed, retrieve, and generate. The core is ~20 lines (full script: practice/rag.py):

# index: overlapping ~500-char windows keep a whole idea in one chunk
chunks = [t[s:s+500] for doc in lessons for t in [text_of(doc)]
          for s in range(0, len(t)-200, 350)]
cvecs = embed(chunks)                              # Lesson 11

# retrieve top-k by cosine (normalized vectors → dot product)
qv = embed([question])[0]
top = sorted(range(len(chunks)), key=lambda i: -float(mx.sum(qv*cvecs[i])))[:4]
context = "\n".join(f"- {chunks[i]}" for i in top)

# generate, grounded ONLY in the retrieved context (Lesson 2)
msgs = [{"role":"user", "content": f"Answer using ONLY these notes:\n{context}\n\nQuestion: {question}"}]
answer = generate(lm, tok, prompt=tok.apply_chat_template(msgs, add_generation_prompt=True))

Real run on your M4 Pro — 226 windows indexed from the 12 lessons, asked an unseen question:

# Q: "On a Mac, why is there no step to move the model to the GPU?"
# retrieved → Lesson 1 (unified memory). Answer:
On a Mac running Apple Silicon, there is no need to explicitly move the model to
the GPU because Apple Silicon uses unified memory — the CPU and GPU share the same
RAM pool. The model is loaded once and computed directly on the GPU without copying.
🎯 Your capstone win Run ~/projects/learn/public/courses/mlx-lm/practice/.venv/bin/python ~/projects/learn/public/courses/mlx-lm/practice/rag.py. A local model just answered a question about your own course material — correctly, grounded in the exact lesson, with nothing leaving your Mac. Change the glob to point at your notes, a repo, or public/content and it's your knowledge base.

03 The honest part — retrieval is the hard part

⚠️ Garbage retrieval → garbage answer (seen live) The first version of this used sentence-sized chunks and asked "how do I make a fine-tuned model smaller without losing the tuning?" — and it retrieved tangential passages and gave a vague answer, because the precise insight (8-bit keeps a fine-tune, 4-bit washes it out — Lesson 5) got split across chunks and never ranked. Switching to overlapping windows fixed retrieval. In RAG, the model is rarely the bottleneck — your chunking and embeddings are.
LeverWhen retrieval is weak
Chunkingtoo small fragments ideas; too big dilutes them. Overlapping windows are a solid default.
Embedding modela stronger/larger embedder (e.g. a BGE model) separates meanings better than a tiny one.
top-kretrieve a few more chunks so the right one is in the context.
Re-rankingfetch top-20, then re-score with a cross-encoder and keep the best 4.

04 Why "answer using only these notes" matters

That instruction is the whole point of RAG: it grounds the model in retrieved text instead of its (small, dated) memory. Recall Lesson 1, where the bare 1B model claimed MLX was for "M1 and M2 processors" — out of date. Feed it the right passage and it answers from that, not from a fuzzy recollection. Grounding is how a small local model gives trustworthy, current answers about your data.

💡 Compose the rest of the course onto it Want the answer as JSON (with a sources list)? Add Lesson 8's output_type. Want it to decide whether to search vs. answer directly? That's a Lesson 10 tool call. RAG is the backbone; the other lessons are attachments.

05 Check yourself

Pick an answer for instant feedback (your choices are saved locally). This is the last set — you've earned it.

1. Why retrieve passages instead of just asking the model your question directly?

2. The capstone gave a vague answer at first. What was actually wrong?

3. What does "Answer using ONLY these notes" accomplish?

4. Your RAG tool misses relevant docs. Best first thing to improve?

06 You finished the course

Twelve lessons. You can now, entirely on your Mac: run models (CLI + Python), serve them (OpenAI-compatible), fine-tune with LoRA, quantize & convert, use the KV cache, force structured output, do tool calling, compute embeddings, size a model to your RAM, and build RAG over your own content. That's a complete local-LLM toolkit.

Keep handy: Glossary & quick reference · ← Lesson 11 · ↩ Lesson 1. The real next step is yours: point this RAG at the learn content or your notes and make it a tool you actually use — bring it to me and we'll wire it in.