{"course":{"slug":"agents-and-automation","title":"Agents and Automation","url":"https://addaly.com/learn/agents-and-automation","level":"intermediate","track":"build"},"lesson":{"slug":"designing-a-tool-surface","title":"Tools that survive being called by a machine","url":"https://addaly.com/learn/agents-and-automation/designing-a-tool-surface","position":16,"of":72,"minutes":9,"answer":"Design tools for a caller that retries, invents arguments and never reads the documentation: idempotent, paginated, timed out, enumerated, and safe to dry run.","keyIdea":"Design tools for a caller that retries, invents arguments and never reads the documentation: idempotent, paginated, timed out, enumerated, and safe to dry run."},"body":{"format":"markdown","text":"## A different kind of caller\n\nYou already know that a tool description is prompt, and that error strings are feedback. This lesson is about the other half: the engineering of the tool itself, for a caller with an unusual set of habits.\n\nYour caller retries without noticing. It invents plausible arguments. It never reads a changelog. It will call the same thing five times in a row if the result is ambiguous. Design accordingly.\n\n## Assume every call happens twice\n\nStripe puts an `Idempotency-Key` header on every mutating request and stores the result against that key for 24 hours; a repeat with the same key replays the stored response instead of charging again. Build the same thing inside your harness.\n\n```python\nkey = sha256(f\"{run_id}:{tool}:{canonical_json(args)}\").hexdigest()\nif row := seen.get(key):\n    return row.result          # replay, do not re-execute\n```\n\nThe rule that catches people: **the key is generated once, before the first attempt, and reused on every retry.** A fresh key per attempt is not idempotency, it is a second request wearing a hat.\n\n## Return identifiers, not payloads\n\nA tool that returns a 40,000-token document has ended the run, and it did it on step two. Return the smallest thing that lets the model decide what to do next:\n\n- `doc_id`, title, a 200-character snippet, and a `read_section(doc_id, heading)` tool for when it actually needs the text\n- a row count and five example rows, not four thousand rows\n- a file path, when the content is going to be re-read later\n\n## Paginate, cap, and never truncate silently\n\nGive every list tool a `limit` with a sane default of about 20 and a hard maximum. Then return the total:\n\n```\nShowing 20 of 3,412 matching orders (sorted newest first).\nNarrow with status= or since= to see the rest.\n```\n\nSilent truncation is the worst possible behaviour, because the model reasons confidently about a subset it believes is the whole set. If you cut, say so **in the string the model reads**. A `has_more: true` field in JSON is fine; a sentence is better.\n\n## Timeouts belong to you\n\nNever let a tool hang the loop. Wrap every call in a timeout (ten seconds is a reasonable default) and return something the model can act on:\n\n```\nTimed out after 10s calling the shipping API. It may be slow right now.\nYou can continue without shipping data, or try once more.\n```\n\nCompare that with a request that never returns, which shows up in your logs as an agent that \"stopped\".\n\n## Narrow the arguments so wrong ones cannot be expressed\n\nEvery free-text argument is an invitation to invent. Use enums:\n\n```json\n{ \"status\": { \"type\": \"string\",\n              \"enum\": [\"pending\", \"shipped\", \"delivered\", \"cancelled\"] } }\n```\n\nNow the model cannot ask for `\"in transit\"`. Providers that constrain decoding will not emit it; providers that do not will fail your validation, and you return a clean message listing the four legal values. Either way you get a readable failure instead of a query that quietly matches nothing.\n\nThe same applies to identifiers. **Any id in a tool call that did not come out of a previous tool result should be treated as fabricated until checked.** Enforce that in the harness, not in the prompt.\n\n## A dry run flag on anything destructive\n\nGive every destructive tool a `commit` parameter defaulting to false. Called dry, it returns exactly what would happen:\n\n```\nDRY RUN: would refund 2499 to card ending 4412 for ORD-48213.\nCall again with commit=true to execute.\n```\n\nThis buys three things at once: you can test the whole loop with zero blast radius, you can show a human a real diff when you add approval in module four, and a fumbled call costs nothing.\n\n## Version tools like an API\n\nChanging what a tool returns changes agent behaviour and quietly invalidates every evaluation you have run. Treat a tool's contract as public: additive changes are fine, changes of meaning get a new name."},"check":{"question":"An agent calls `create_shipment`. The call succeeds at the carrier but the response is lost to a network timeout, so the harness retries. The carrier now has two shipments. Which fix addresses the cause?","options":[{"text":"Compute an idempotency key from the run id and canonical arguments before the first attempt, and have the carrier or harness replay the stored result on retry.","correct":true},{"text":"Stop retrying on timeouts, since a timeout might mean the request succeeded.","correct":false,"tempting":"Trades duplicates for silent failures. Timeouts are the archetypal transient error, and never retrying means genuine drops go unnoticed."},{"text":"Add to the tool description that shipments must not be created twice.","correct":false,"tempting":"Puts a control in the prompt when the duplicate came from the harness retrying, not from the model choosing."},{"text":"Generate a fresh UUID for each attempt so the two calls can be told apart in the logs.","correct":false,"tempting":"Names the exact anti-pattern: a new key per attempt makes each retry a new request, which is how duplicates happen."}]},"prev":{"slug":"planning-and-decomposition","title":"Plan first, or just start?","url":"https://addaly.com/learn/agents-and-automation/planning-and-decomposition"},"next":{"slug":"observations-that-fit","title":"The observation is where runs die","url":"https://addaly.com/learn/agents-and-automation/observations-that-fit"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Tools that survive being called by a machine', in Agents and Automation. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/agents-and-automation/designing-a-tool-surface (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Tools that survive being called by a machine","courseTitle":"Agents and Automation","module":{"index":2,"slug":"how-the-loop-runs","title":"The machinery: transcript, tools, loop"},"answer":null,"keyIdea":"Design tools for a caller that retries, invents arguments and never reads the documentation: idempotent, paginated, timed out, enumerated, and safe to dry run.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"a-different-kind-of-caller","runs":[{"text":"A different kind of caller"}]},{"t":"p","runs":[{"text":"You already know that a tool description is prompt, and that error strings are feedback. This lesson is about the other half: the engineering of the tool itself, for a caller with an unusual set of habits."}]},{"t":"p","runs":[{"text":"Your caller retries without noticing. It invents plausible arguments. It never reads a changelog. It will call the same thing five times in a row if the result is ambiguous. Design accordingly."}]},{"t":"h2","id":"assume-every-call-happens-twice","runs":[{"text":"Assume every call happens twice"}]},{"t":"p","runs":[{"text":"Stripe puts an "},{"text":"Idempotency-Key","code":true},{"text":" header on every mutating request and stores the result against that key for 24 hours; a repeat with the same key replays the stored response instead of charging again. Build the same thing inside your harness."}]},{"t":"code","lang":"python","label":"Python","text":"key = sha256(f\"{run_id}:{tool}:{canonical_json(args)}\").hexdigest()\nif row := seen.get(key):\n    return row.result          # replay, do not re-execute"},{"t":"p","runs":[{"text":"The rule that catches people: "},{"text":"the key is generated once, before the first attempt, and reused on every retry.","b":true},{"text":" A fresh key per attempt is not idempotency, it is a second request wearing a hat."}]},{"t":"h2","id":"return-identifiers-not-payloads","runs":[{"text":"Return identifiers, not payloads"}]},{"t":"p","runs":[{"text":"A tool that returns a 40,000-token document has ended the run, and it did it on step two. Return the smallest thing that lets the model decide what to do next:"}]},{"t":"ul","items":[[{"text":"doc_id","code":true},{"text":", title, a 200-character snippet, and a "},{"text":"read_section(doc_id, heading)","code":true},{"text":" tool for when it actually needs the text"}],[{"text":"a row count and five example rows, not four thousand rows"}],[{"text":"a file path, when the content is going to be re-read later"}]]},{"t":"h2","id":"paginate-cap-and-never-truncate-silently","runs":[{"text":"Paginate, cap, and never truncate silently"}]},{"t":"p","runs":[{"text":"Give every list tool a "},{"text":"limit","code":true},{"text":" with a sane default of about 20 and a hard maximum. Then return the total:"}]},{"t":"code","lang":"","label":"Text","text":"Showing 20 of 3,412 matching orders (sorted newest first).\nNarrow with status= or since= to see the rest."},{"t":"p","runs":[{"text":"Silent truncation is the worst possible behaviour, because the model reasons confidently about a subset it believes is the whole set. If you cut, say so "},{"text":"in the string the model reads","b":true},{"text":". A "},{"text":"has_more: true","code":true},{"text":" field in JSON is fine; a sentence is better."}]},{"t":"h2","id":"timeouts-belong-to-you","runs":[{"text":"Timeouts belong to you"}]},{"t":"p","runs":[{"text":"Never let a tool hang the loop. Wrap every call in a timeout (ten seconds is a reasonable default) and return something the model can act on:"}]},{"t":"code","lang":"","label":"Text","text":"Timed out after 10s calling the shipping API. It may be slow right now.\nYou can continue without shipping data, or try once more."},{"t":"p","runs":[{"text":"Compare that with a request that never returns, which shows up in your logs as an agent that “stopped”."}]},{"t":"h2","id":"narrow-the-arguments-so-wrong-ones-cannot-be-expressed","runs":[{"text":"Narrow the arguments so wrong ones cannot be expressed"}]},{"t":"p","runs":[{"text":"Every free-text argument is an invitation to invent. Use enums:"}]},{"t":"code","lang":"json","label":"JSON","text":"{ \"status\": { \"type\": \"string\",\n              \"enum\": [\"pending\", \"shipped\", \"delivered\", \"cancelled\"] } }"},{"t":"p","runs":[{"text":"Now the model cannot ask for "},{"text":"\"in transit\"","code":true},{"text":". Providers that constrain decoding will not emit it; providers that do not will fail your validation, and you return a clean message listing the four legal values. Either way you get a readable failure instead of a query that quietly matches nothing."}]},{"t":"p","runs":[{"text":"The same applies to identifiers. "},{"text":"Any id in a tool call that did not come out of a previous tool result should be treated as fabricated until checked.","b":true},{"text":" Enforce that in the harness, not in the prompt."}]},{"t":"h2","id":"a-dry-run-flag-on-anything-destructive","runs":[{"text":"A dry run flag on anything destructive"}]},{"t":"p","runs":[{"text":"Give every destructive tool a "},{"text":"commit","code":true},{"text":" parameter defaulting to false. Called dry, it returns exactly what would happen:"}]},{"t":"code","lang":"","label":"Text","text":"DRY RUN: would refund 2499 to card ending 4412 for ORD-48213.\nCall again with commit=true to execute."},{"t":"p","runs":[{"text":"This buys three things at once: you can test the whole loop with zero blast radius, you can show a human a real diff when you add approval in module four, and a fumbled call costs nothing."}]},{"t":"h2","id":"version-tools-like-an-api","runs":[{"text":"Version tools like an API"}]},{"t":"p","runs":[{"text":"Changing what a tool returns changes agent behaviour and quietly invalidates every evaluation you have run. Treat a tool’s contract as public: additive changes are fine, changes of meaning get a new name."}]}],"check":{"question":[{"text":"An agent calls "},{"text":"create_shipment","code":true},{"text":". The call succeeds at the carrier but the response is lost to a network timeout, so the harness retries. The carrier now has two shipments. Which fix addresses the cause?"}],"options":[{"text":[{"text":"Compute an idempotency key from the run id and canonical arguments before the first attempt, and have the carrier or harness replay the stored result on retry."}],"correct":true,"tempting":null},{"text":[{"text":"Stop retrying on timeouts, since a timeout might mean the request succeeded."}],"correct":false,"tempting":[{"text":"Trades duplicates for silent failures. Timeouts are the archetypal transient error, and never retrying means genuine drops go unnoticed."}]},{"text":[{"text":"Add to the tool description that shipments must not be created twice."}],"correct":false,"tempting":[{"text":"Puts a control in the prompt when the duplicate came from the harness retrying, not from the model choosing."}]},{"text":[{"text":"Generate a fresh UUID for each attempt so the two calls can be told apart in the logs."}],"correct":false,"tempting":[{"text":"Names the exact anti-pattern: a new key per attempt makes each retry a new request, which is how duplicates happen."}]}],"order":[3,2,1,0],"why":null},"prev":{"slug":"planning-and-decomposition","title":"Plan first, or just start?","minutes":9},"next":{"slug":"observations-that-fit","title":"The observation is where runs die","minutes":8}}}