Every tool costs tokens before anyone calls it

The bill arrives before the tool runs

A tool feels like a function: it costs something when you call it. In a language model request it is not like that. Every tool you make available is serialised into the prompt (name, description, and a full JSON Schema for its parameters) on every single call, including the calls where the model uses no tool at all.

Here is a modest one:

{
  "name": "search_orders",
  "description": "Search a customer's orders by date range or status. Returns at most 20 orders, newest first. Only covers the last 90 days; older orders are in the archive and must be fetched with get_archived_order.",
  "input_schema": {
    "type": "object",
    "properties": {
      "customer_id": {"type": "string", "description": "Internal id, format c_NNNN."},
      "status": {"type": "string", "enum": ["placed", "shipped", "delivered", "cancelled", "any"]},
      "since": {"type": "string", "description": "ISO 8601 date, e.g. 2026-07-01."}
    },
    "required": ["customer_id"]
  }
}

That is around 180 tokens once rendered. It is a good definition: the description does real work, which we will come back to. But it is 180 tokens on every request for the rest of its life.

Twenty tools of that size is roughly 3,600 tokens per call. At a million calls a month that is 3.6 billion input tokens you are paying for before the user has typed anything. Most of it is cacheable, which we will use, but cacheable is not free and the cache is not always warm.

Count it, do not estimate it

The rendering is done by the provider, not by you, so the only honest count is the one the API reports. Every major API returns an input token count on the response, and several offer a count-tokens endpoint that charges nothing. Run one request with the full toolset and one with tools omitted entirely. The difference is the standing cost of your tools.

Do this once and write the number in the budget ledger next to the system prompt. Teams are routinely surprised: a toolset assembled over six months by six people is frequently the second-largest block in the request, behind only retrieved passages.

If you are on an open model and want to see the rendering itself, transformers will show you. tokenizer.apply_chat_template(messages, tools=tools, tokenize=False) prints the exact string the model receives, tool schemas and all. It costs nothing and it is the fastest way to discover that your framework is injecting a paragraph you never wrote.

The second cost is worse than the first

Tokens are the cost you can see. The cost you cannot see is that selection accuracy falls as the toolset grows.

The mechanism is not mysterious. Choosing a tool is a classification problem the model solves by reading descriptions, and adding options to a classification problem makes it harder, especially when the options overlap. With four tools that do obviously different things, selection is close to perfect. With forty, several of which could plausibly serve the same request, the model starts choosing inconsistently: the same question on two runs picks two different tools, both defensible, one of them wrong for your system.

Published figures for where this begins vary widely, because it depends entirely on how distinct your descriptions are, and you should distrust any single number quoted at you. What does not vary is the direction, and that you can measure yourself. Write thirty representative user messages. Record which tool you believe should be called for each. Run them and compare. That is a tool-selection accuracy score, it costs thirty cheap calls, and it is the only figure about your toolset that means anything.

Where they sit, and why order matters

Tool definitions are rendered near the top of the request, above the conversation, usually adjacent to the system prompt. That is good news: they belong to the stable prefix and are cached along with it.

It is good news with a condition. Prefix caching matches from the first token forward, so the order of your tools is part of the cache key. If your code builds the tool list from a Python dictionary, or from a database query with no ORDER BY, or from a set, the order can change between deployments, or between processes, and every change is a full cache miss on the whole prefix.

tools = sorted(registry.enabled_for(user), key=lambda t: t["name"])

One line. It costs nothing and it removes an entire class of invisible cost regression.

The same logic governs adding a tool. Appending a twenty-first tool to the end of the list invalidates the cache from that point on, which is cheap. Inserting it alphabetically into the middle invalidates everything after it, which is not. If you sort, accept that adding a tool named archive_order is a bigger cache event than adding one named zip_export, and schedule the deploy accordingly rather than being puzzled by the bill.

The limitation

None of this tells you whether a tool earns its place. Token cost you can measure exactly; selection damage you can measure roughly; the value of a tool being available is the hardest of the three, because it only shows up on the requests that needed it, which may be two percent of traffic.

The test that works is removal. Take the tool out, run the golden set, and see what breaks. If nothing breaks, the set does not cover what the tool is for, which is itself worth knowing, or the tool is dead weight. Teams almost never do this, which is why almost every mature toolset has at least one tool nothing has called in months, still being paid for on every request.