Tools that survive being called by a machine

A different kind of caller

You already know that a tool description is prompt, and that error strings are feedback. This lesson is about the other half: the engineering of the tool itself, for a caller with an unusual set of habits.

Your caller retries without noticing. It invents plausible arguments. It never reads a changelog. It will call the same thing five times in a row if the result is ambiguous. Design accordingly.

Assume every call happens twice

Stripe puts an Idempotency-Key header on every mutating request and stores the result against that key for 24 hours; a repeat with the same key replays the stored response instead of charging again. Build the same thing inside your harness.

key = sha256(f"{run_id}:{tool}:{canonical_json(args)}").hexdigest()
if row := seen.get(key):
    return row.result          # replay, do not re-execute

The rule that catches people: the key is generated once, before the first attempt, and reused on every retry. A fresh key per attempt is not idempotency, it is a second request wearing a hat.

Return identifiers, not payloads

A tool that returns a 40,000-token document has ended the run, and it did it on step two. Return the smallest thing that lets the model decide what to do next:

• doc_id, title, a 200-character snippet, and a read_section(doc_id, heading) tool for when it actually needs the text
• a row count and five example rows, not four thousand rows
• a file path, when the content is going to be re-read later

Paginate, cap, and never truncate silently

Give every list tool a limit with a sane default of about 20 and a hard maximum. Then return the total:

Showing 20 of 3,412 matching orders (sorted newest first).
Narrow with status= or since= to see the rest.

Silent truncation is the worst possible behaviour, because the model reasons confidently about a subset it believes is the whole set. If you cut, say so in the string the model reads. A has_more: true field in JSON is fine; a sentence is better.

Timeouts belong to you

Never let a tool hang the loop. Wrap every call in a timeout (ten seconds is a reasonable default) and return something the model can act on:

Timed out after 10s calling the shipping API. It may be slow right now.
You can continue without shipping data, or try once more.

Compare that with a request that never returns, which shows up in your logs as an agent that “stopped”.

Narrow the arguments so wrong ones cannot be expressed

Every free-text argument is an invitation to invent. Use enums:

{ "status": { "type": "string",
              "enum": ["pending", "shipped", "delivered", "cancelled"] } }

Now the model cannot ask for "in transit". Providers that constrain decoding will not emit it; providers that do not will fail your validation, and you return a clean message listing the four legal values. Either way you get a readable failure instead of a query that quietly matches nothing.

The same applies to identifiers. Any id in a tool call that did not come out of a previous tool result should be treated as fabricated until checked. Enforce that in the harness, not in the prompt.

A dry run flag on anything destructive

Give every destructive tool a commit parameter defaulting to false. Called dry, it returns exactly what would happen:

DRY RUN: would refund 2499 to card ending 4412 for ORD-48213.
Call again with commit=true to execute.

This buys three things at once: you can test the whole loop with zero blast radius, you can show a human a real diff when you add approval in module four, and a fumbled call costs nothing.

Version tools like an API

Changing what a tool returns changes agent behaviour and quietly invalidates every evaluation you have run. Treat a tool’s contract as public: additive changes are fine, changes of meaning get a new name.