mlx-lm · Lesson 10

Tool Calling — Let the Model Use Your Functions

Give a local model a menu of your Python functions and let it pick the right one with the right arguments. This is the agentic primitive: a model that doesn't just talk, but acts — entirely on your Mac. And it's just Lesson 8 with one more step.

🎯 Mission step: a local model that can call your code
The one idea

Tool calling = structured output (Lesson 8) + dispatch. Describe each tool as a schema, constrain the model to emit one of them, and you get a guaranteed-valid "call this function with these args" object. Your code parses it and runs the real function. The model decides what to do; your code does the doing.

01 Why this is the agentic primitive

Everything "agentic" — assistants, copilots, automations — bottoms out in this loop: the model reads a request, chooses a tool, your code runs it, and the result flows back. Once a local model can reliably call your functions, you can build real automations on-device without sending anything to a cloud. It's the bridge from "a model that answers" to "a model that does."

💡 Two ingredients (1) describe your tools as schemas (name + typed parameters); (2) constrain the model's output to a choice among those schemas. You already have the tool for (2) — Outlines from Lesson 8. A Union of schemas means "emit exactly one valid tool call."

02 The code — a Union of tool schemas

Each tool is a Pydantic model with a tool discriminator and its typed args. Constrain the output to Union[...], parse the guaranteed-valid JSON, and dispatch:

import json, outlines, mlx_lm
from pydantic import BaseModel
from typing import Literal, Union

def add(a, b): return a + b
def get_weather(city): return f"{city}: 21C, clear"

class Add(BaseModel):
    tool: Literal["add"]; a: float; b: float
class GetWeather(BaseModel):
    tool: Literal["get_weather"]; city: str

model = outlines.from_mlxlm(*mlx_lm.load("mlx-community/Qwen2.5-3B-Instruct-4bit"))

def run(user):
    raw = model(f"You can call tools. Pick the right one for: {user}",
                output_type=Union[Add, GetWeather], max_tokens=128)
    c = json.loads(raw)                       # valid by construction
    if c["tool"] == "add":      return add(c["a"], c["b"])
    return get_weather(c["city"])

Real output on your M4 Pro — right tool, right args, every time:

'What is 12 plus 30?'
    -> add(a=12, b=30)            -> 42
"What's the weather in Tokyo?"
    -> get_weather(city='Tokyo')  -> Tokyo: 21C, clear
🎯 Your tangible win Run ~/projects/learn/public/courses/mlx-lm/practice/.venv/bin/python ~/projects/learn/public/courses/mlx-lm/practice/tool_calling.py. A local model just read plain English, chose the correct function from your menu, and filled in the arguments — and because the call is schema-constrained, your code can dispatch it without a single defensive check.

03 The full tool-use loop

One call is a "tool call." A real assistant runs the loop: the model emits a call, you execute it, you feed the result back, and the model uses it to answer (or call another tool). That round-trip is the whole of "agentic."

user ask "12 + 30?" model constrained to tool schemas {"tool":"add", "a":12,"b":30} your code runs add() → 42 result fed back → model answers "42"
The model only ever proposes a call; your code is the only thing that runs it (so you keep control — validate args, gate dangerous actions, log). Feed the result back for the model to use, and you have a working agent loop on-device.

04 The native path — and why we constrain

Instruct models also have a native tool format. Llama 3.2's chat template accepts a tools= argument and asks the model to reply with {"name": …, "parameters": …}; OpenAI-compatible servers (Lesson 3) take a tools field too. It works — but the raw output isn't guaranteed valid, and each model family formats it slightly differently (some wrap it in tags).

💡 Native describes; constrained guarantees Use the native tools= template to tell the model what's available, and constrained decoding to force the output into a shape you can parse. Together: idiomatic prompt + bulletproof output. For quick experiments the native path alone is fine; for code you depend on, constrain it.

⚠️ Callback to Lesson 9: pick an adequate model We used the 3B here, not the 1B from earlier lessons. Constrained decoding guarantees the format of the call, but choosing the right tool and arguments is reasoning — and a 1B model picks wrong more often. Tool selection is exactly the kind of task where stepping up a size pays off.

05 Check yourself

Pick an answer — you'll get instant feedback. (Want to wire a real tool of yours — a shell command, an API, a file action? Ask me and we'll design its schema.)

1. Constraining the output to a Union of tool schemas guarantees that…

2. In tool calling, who actually executes the function?

3. The model called add but with the wrong numbers. Constrained decoding would…

4. Why use the 3B model here instead of the 1B from earlier lessons?

06 Going deeper

Keep handy: Glossary & quick reference · ← Lesson 9. Next up → Lesson 11: Local Embeddings & Semantic Search — turn text into vectors on-device and find things by meaning (the other half of building real local-LLM tools).