Tool use, with the magic removed

The loop, in full

Tool use sounds like the model reaching out into the world. It is not. Here is every step:

1. You include descriptions of the available tools in the prompt: names, parameters, what each does. These are tokens like any other.
2. The model generates text that, by a convention it was trained on, represents a call: a name and some arguments, usually as JSON.
3. Your code parses that, decides whether to run it, runs it, and gets a result.
4. You put the result back into the context as a new message.
5. The model generates again, now with the result available.

The model emits a string. Everything that happens as a consequence is your program’s doing. This is worth being blunt about, because the security implications in the rest of this module rest on it.

What “trained on” means here

The convention is learned. During fine-tuning, models are shown many examples of tool descriptions followed by well-formed calls, so they reliably produce the right shape. That reliability is behavioural, not structural, which is why constrained decoding, from module three, is worth applying to tool calls: it makes an invalid function name or a missing required parameter impossible rather than merely rare.

Where tool use actually fails

Not usually in producing the call. In everything around it:

• Too many tools. Beyond roughly a dozen, selection accuracy declines and the definitions themselves consume real context. If you have forty tools, group them, or use retrieval to select a relevant subset per request.
• Vague descriptions. The description is the entire basis for choosing. “Gets user data” will be called for everything and nothing. Say what it returns, what it costs, and when not to use it: that last clause is the one people omit and it is frequently the most useful.
• Overlapping tools. Two tools that could both plausibly answer a question produce inconsistent selection. Make the boundary explicit in both descriptions.
• Results that blow the budget. A tool returning 30,000 tokens of JSON has just consumed the context window. Summarise, paginate or filter tool output before it goes back in.
• Errors reported as prose. If your tool returns “Error: connection refused” as a plain string, the model will often apologise and try something unrelated. Return structured errors that say whether a retry is sensible.

The permission boundary is yours

Step 3 is the one that matters. The model produces a request; your code decides. That decision point is where every safety property of a tool-using system lives:

• Read and write must be separated. A search tool and a delete tool are not the same risk, and treating them identically because both are “tools” is how systems cause damage.
• Anything irreversible needs a human. Sending a message, spending money, deleting data. The model can propose; a person confirms.
• Validate arguments against a schema and against your own authorisation rules, every time, exactly as you would for input arriving from the internet, because in an important sense it is.

The reason this matters is the subject of the next lesson: text that arrives from outside can influence what the model emits, and the only reliable boundary is the one your code enforces.

Parallel calls and what they change

Most current models can emit several tool calls at once: three searches rather than one, issued together. Your code runs them concurrently and returns all the results in one message. This is a genuine latency win, since the alternative is three full round trips through the model.

It also removes a safety property people did not realise they had. With sequential calls, the model sees each result before choosing the next action, so a failed lookup stops the chain. With parallel calls it has committed to all three before seeing any of them, which means arguments derived from a guess are not corrected mid-flight. Use parallelism for independent reads and keep anything dependent sequential.

The connection to agents

An agent is this loop, run repeatedly, with the model deciding when to stop. Nothing new is added. Everything above compounds: more steps mean more context, more chances to select the wrong tool, and more opportunity for one bad result to derail the rest.

Which suggests the design that works. Fewer tools, described precisely. Structured results, summarised. A step limit. Verification after each action rather than a hopeful check at the end. And an explicit human gate on anything you would not want done twice.