{"course":{"slug":"building-with-ai","title":"Building with AI","url":"https://addaly.com/learn/building-with-ai","level":"intermediate","track":"build"},"lesson":{"slug":"tool-use","title":"Tool use: the model never runs anything","url":"https://addaly.com/learn/building-with-ai/tool-use","position":24,"of":80,"minutes":8,"answer":"The model can only ask for a tool call; your code runs it, so your code must authorise it.","keyIdea":"The model can only ask for a tool call; your code runs it, so your code must authorise it."},"body":{"format":"markdown","text":"## The loop, honestly described\n\n\"Function calling\" is a bad name. The model does not call your function. It cannot reach your database, your filesystem or the internet. What it does is emit a structured request (a tool name and some arguments) and then stop.\n\nYour code decides whether to run it, runs it, and sends the result back as another message. Then the model continues. That is the entire mechanism.\n\n```python\ntools = [{\n    \"name\": \"get_order_status\",\n    \"description\": (\n        \"Look up the delivery status of a single order. \"\n        \"Use when the customer asks where their order is. \"\n        \"Do not use for refunds or cancellations.\"\n    ),\n    \"input_schema\": {\n        \"type\": \"object\",\n        \"properties\": {\"order_id\": {\"type\": \"string\", \"description\": \"Order id like ORD-48213\"}},\n        \"required\": [\"order_id\"],\n    },\n}]\n\nmessages = [{\"role\": \"user\", \"content\": \"where is ORD-48213\"}]\n\nfor _ in range(6):                      # a budget, not a while True\n    r = client.messages.create(model=MODEL, max_tokens=1024, tools=tools, messages=messages)\n    messages.append({\"role\": \"assistant\", \"content\": r.content})\n\n    if r.stop_reason != \"tool_use\":\n        break                            # the model is done, r has the final text\n\n    results = []\n    for block in r.content:\n        if block.type != \"tool_use\":\n            continue\n        try:\n            out = run_tool(block.name, block.input, session=session)\n        except PermissionError:\n            out = {\"error\": \"not_authorised\", \"hint\": \"Ask the customer to sign in.\"}\n        except Exception as e:\n            out = {\"error\": \"tool_failed\", \"detail\": str(e)[:200]}\n        results.append({\"type\": \"tool_result\", \"tool_use_id\": block.id,\n                        \"content\": json.dumps(out)})\n\n    messages.append({\"role\": \"user\", \"content\": results})\n```\n\nNote what happens on failure. The error goes back to the model as a tool result, not up as an exception. A model that is told \"not_authorised\" can say something useful to the customer. A model that never hears back cannot.\n\n## Tool descriptions are prompt text\n\nThe description field is not documentation for you. It is the only thing the model reads when deciding whether this tool applies. Write it like an instruction to a new colleague: what it does, when to use it, when not to.\n\nThe most common bug in tool use is not a broken tool. It is two tools whose descriptions overlap, so the model picks the wrong one about a third of the time. If `search_orders` and `get_order_status` both say \"find order information\", you have built a coin flip. Either sharpen the boundary in the descriptions or merge them into one tool with a parameter.\n\n> Figure: Six steps, and the model only takes one of them. Steps: 1. The model emits a tool_use block (A tool name and arguments. Nothing has run and nothing has been read.); 2. Your code looks the name up (An unknown name becomes an error result, not an exception); 3. Your handler authorises against the session (order.customer_id must equal session.customer_id, or PermissionError); 4. Your code runs the tool (The only place in the loop where anything executes); 5. The result goes back as a message (Errors go back too, so the model can say something useful to the customer); 6. The model continues, or stops (stop_reason tells you which, and the loop counter decides how often). The model's whole contribution is step one: a name and some arguments, produced from text a user largely wrote. Everything that touches data happens in your process, which is why authorisation belongs to the session and never to an argument the model chose.\n\nKeep the set small. Five well-separated tools work far better than twenty with fuzzy edges.\n\n## Authorise against the session, never the arguments\n\nThis is the part that becomes a security incident.\n\nThe arguments in a tool call are text the model produced, largely from text the user wrote. They are untrusted input, exactly like a query string. If your handler does this:\n\n```python\ndef get_order_status(order_id):\n    return db.query(\"SELECT * FROM orders WHERE id = ?\", order_id)   # wrong\n```\n\nthen anyone who can get the model to emit a different order id can read a stranger's order. And getting a model to emit a different string is not hard.\n\nThe fix is not in the prompt. It is in the handler:\n\n```python\ndef get_order_status(order_id, *, session):\n    order = db.get_order(order_id)\n    if order is None or order.customer_id != session.customer_id:\n        raise PermissionError\n    return order.public_view()\n```\n\nThe session comes from your auth layer and the model cannot influence it. Every privileged tool gets this shape. Write the rule down for your team: **a tool call is a request from an untrusted source; the handler decides.**\n\n## Make tools boring\n\nA few habits that save you later:\n\n- Prefer read tools. Every write tool is a way for a confused model to do something you cannot undo.\n- Make writes idempotent, keyed on something stable, so a retried call does not double-charge anyone.\n- Return small results. A tool that dumps 40KB of JSON pushes everything else out of context and costs money on every subsequent turn.\n- Log every tool call with its arguments and result. When someone asks \"why did it do that\", this log is the only answer you will have."},"check":{"question":"A shopping assistant with an order-lookup tool goes live. A user reports seeing another customer's delivery address after some back-and-forth with the bot. The tool loop worked exactly as written. Where is the bug?","options":[{"text":"The tool description was ambiguous about whose orders may be looked up, so the model over-reached; a clearer description fixes it.","correct":false,"tempting":"Descriptions really do change tool-selection behaviour, so prompt-tuning feels like the natural lever to pull."},{"text":"The system prompt should have said \"only look up the signed-in customer's own orders\", and did not.","correct":false,"tempting":"It reads like a rule the model can follow, and it usually does follow it, which hides the hole for months."},{"text":"The model was given database access with too broad a permission scope; the model's credentials need narrowing.","correct":false,"tempting":"The phrase 'function calling' makes it sound as though the model executes code itself, so the fix seems to be about what the model can reach."},{"text":"The handler trusted the order id in the tool arguments; authorisation has to be checked against the session, because tool arguments are untrusted input.","correct":true}]},"prev":{"slug":"extraction-at-scale","title":"Extraction that runs on ten thousand documents","url":"https://addaly.com/learn/building-with-ai/extraction-at-scale"},"next":{"slug":"designing-a-tool-surface","title":"Designing tools a model can use correctly","url":"https://addaly.com/learn/building-with-ai/designing-a-tool-surface"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Tool use: the model never runs anything', in Building with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/building-with-ai/tool-use (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Tool use: the model never runs anything","courseTitle":"Building with AI","module":{"index":3,"slug":"structure-tools-and-the-loop","title":"Structure, tools and the loop"},"answer":null,"keyIdea":"The model can only ask for a tool call; your code runs it, so your code must authorise it.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"flow","steps":[{"label":"The model emits a tool_use block","detail":"A tool name and arguments. Nothing has run and nothing has been read."},{"label":"Your code looks the name up","detail":"An unknown name becomes an error result, not an exception"},{"label":"Your handler authorises against the session","detail":"order.customer_id must equal session.customer_id, or PermissionError"},{"label":"Your code runs the tool","detail":"The only place in the loop where anything executes"},{"label":"The result goes back as a message","detail":"Errors go back too, so the model can say something useful to the customer"},{"label":"The model continues, or stops","detail":"stop_reason tells you which, and the loop counter decides how often"}],"title":"Six steps, and the model only takes one of them","caption":"The model’s whole contribution is step one: a name and some arguments, produced from text a user largely wrote. Everything that touches data happens in your process, which is why authorisation belongs to the session and never to an argument the model chose."},"alt":"Figure: Six steps, and the model only takes one of them. Steps: 1. The model emits a tool_use block (A tool name and arguments. Nothing has run and nothing has been read.); 2. Your code looks the name up (An unknown name becomes an error result, not an exception); 3. Your handler authorises against the session (order.customer_id must equal session.customer_id, or PermissionError); 4. Your code runs the tool (The only place in the loop where anything executes); 5. The result goes back as a message (Errors go back too, so the model can say something useful to the customer); 6. The model continues, or stops (stop_reason tells you which, and the loop counter decides how often). The model’s whole contribution is step one: a name and some arguments, produced from text a user largely wrote. Everything that touches data happens in your process, which is why authorisation belongs to the session and never to an argument the model chose."}},"blocks":[{"t":"h2","id":"the-loop-honestly-described","runs":[{"text":"The loop, honestly described"}]},{"t":"p","runs":[{"text":"“Function calling” is a bad name. The model does not call your function. It cannot reach your database, your filesystem or the internet. What it does is emit a structured request (a tool name and some arguments) and then stop."}]},{"t":"p","runs":[{"text":"Your code decides whether to run it, runs it, and sends the result back as another message. Then the model continues. That is the entire mechanism."}]},{"t":"code","lang":"python","label":"Python","text":"tools = [{\n    \"name\": \"get_order_status\",\n    \"description\": (\n        \"Look up the delivery status of a single order. \"\n        \"Use when the customer asks where their order is. \"\n        \"Do not use for refunds or cancellations.\"\n    ),\n    \"input_schema\": {\n        \"type\": \"object\",\n        \"properties\": {\"order_id\": {\"type\": \"string\", \"description\": \"Order id like ORD-48213\"}},\n        \"required\": [\"order_id\"],\n    },\n}]\n\nmessages = [{\"role\": \"user\", \"content\": \"where is ORD-48213\"}]\n\nfor _ in range(6):                      # a budget, not a while True\n    r = client.messages.create(model=MODEL, max_tokens=1024, tools=tools, messages=messages)\n    messages.append({\"role\": \"assistant\", \"content\": r.content})\n\n    if r.stop_reason != \"tool_use\":\n        break                            # the model is done, r has the final text\n\n    results = []\n    for block in r.content:\n        if block.type != \"tool_use\":\n            continue\n        try:\n            out = run_tool(block.name, block.input, session=session)\n        except PermissionError:\n            out = {\"error\": \"not_authorised\", \"hint\": \"Ask the customer to sign in.\"}\n        except Exception as e:\n            out = {\"error\": \"tool_failed\", \"detail\": str(e)[:200]}\n        results.append({\"type\": \"tool_result\", \"tool_use_id\": block.id,\n                        \"content\": json.dumps(out)})\n\n    messages.append({\"role\": \"user\", \"content\": results})"},{"t":"p","runs":[{"text":"Note what happens on failure. The error goes back to the model as a tool result, not up as an exception. A model that is told “not_authorised” can say something useful to the customer. A model that never hears back cannot."}]},{"t":"h2","id":"tool-descriptions-are-prompt-text","runs":[{"text":"Tool descriptions are prompt text"}]},{"t":"p","runs":[{"text":"The description field is not documentation for you. It is the only thing the model reads when deciding whether this tool applies. Write it like an instruction to a new colleague: what it does, when to use it, when not to."}]},{"t":"p","runs":[{"text":"The most common bug in tool use is not a broken tool. It is two tools whose descriptions overlap, so the model picks the wrong one about a third of the time. If "},{"text":"search_orders","code":true},{"text":" and "},{"text":"get_order_status","code":true},{"text":" both say “find order information”, you have built a coin flip. Either sharpen the boundary in the descriptions or merge them into one tool with a parameter."}]},{"t":"p","runs":[{"text":"Keep the set small. Five well-separated tools work far better than twenty with fuzzy edges."}]},{"t":"h2","id":"authorise-against-the-session-never-the-arguments","runs":[{"text":"Authorise against the session, never the arguments"}]},{"t":"p","runs":[{"text":"This is the part that becomes a security incident."}]},{"t":"p","runs":[{"text":"The arguments in a tool call are text the model produced, largely from text the user wrote. They are untrusted input, exactly like a query string. If your handler does this:"}]},{"t":"code","lang":"python","label":"Python","text":"def get_order_status(order_id):\n    return db.query(\"SELECT * FROM orders WHERE id = ?\", order_id)   # wrong"},{"t":"p","runs":[{"text":"then anyone who can get the model to emit a different order id can read a stranger’s order. And getting a model to emit a different string is not hard."}]},{"t":"p","runs":[{"text":"The fix is not in the prompt. It is in the handler:"}]},{"t":"code","lang":"python","label":"Python","text":"def get_order_status(order_id, *, session):\n    order = db.get_order(order_id)\n    if order is None or order.customer_id != session.customer_id:\n        raise PermissionError\n    return order.public_view()"},{"t":"p","runs":[{"text":"The session comes from your auth layer and the model cannot influence it. Every privileged tool gets this shape. Write the rule down for your team: "},{"text":"a tool call is a request from an untrusted source; the handler decides.","b":true}]},{"t":"h2","id":"make-tools-boring","runs":[{"text":"Make tools boring"}]},{"t":"p","runs":[{"text":"A few habits that save you later:"}]},{"t":"ul","items":[[{"text":"Prefer read tools. Every write tool is a way for a confused model to do something you cannot undo."}],[{"text":"Make writes idempotent, keyed on something stable, so a retried call does not double-charge anyone."}],[{"text":"Return small results. A tool that dumps 40KB of JSON pushes everything else out of context and costs money on every subsequent turn."}],[{"text":"Log every tool call with its arguments and result. When someone asks “why did it do that”, this log is the only answer you will have."}]]}],"check":{"question":[{"text":"A shopping assistant with an order-lookup tool goes live. A user reports seeing another customer’s delivery address after some back-and-forth with the bot. The tool loop worked exactly as written. Where is the bug?"}],"options":[{"text":[{"text":"The tool description was ambiguous about whose orders may be looked up, so the model over-reached; a clearer description fixes it."}],"correct":false,"tempting":[{"text":"Descriptions really do change tool-selection behaviour, so prompt-tuning feels like the natural lever to pull."}]},{"text":[{"text":"The system prompt should have said “only look up the signed-in customer’s own orders”, and did not."}],"correct":false,"tempting":[{"text":"It reads like a rule the model can follow, and it usually does follow it, which hides the hole for months."}]},{"text":[{"text":"The model was given database access with too broad a permission scope; the model’s credentials need narrowing."}],"correct":false,"tempting":[{"text":"The phrase ‘function calling’ makes it sound as though the model executes code itself, so the fix seems to be about what the model can reach."}]},{"text":[{"text":"The handler trusted the order id in the tool arguments; authorisation has to be checked against the session, because tool arguments are untrusted input."}],"correct":true,"tempting":null}],"order":[0,3,2,1],"why":null},"prev":{"slug":"extraction-at-scale","title":"Extraction that runs on ten thousand documents","minutes":9},"next":{"slug":"designing-a-tool-surface","title":"Designing tools a model can use correctly","minutes":9}}}