{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"tool-definitions-are-context","title":"Every tool costs tokens before anyone calls it","url":"https://addaly.com/learn/context-engineering/tool-definitions-are-context","position":60,"of":87,"minutes":9,"answer":"Tool schemas are serialised into every request whether or not a tool is used.","keyIdea":"Tool schemas are serialised into every request whether or not a tool is used, so a large toolset is a fixed tax on every call and a measurable drag on the model's ability to pick the right one."},"body":{"format":"markdown","text":"## The bill arrives before the tool runs\n\nA tool feels like a function: it costs something when you call it. In a language model request it is not like that. Every tool you make available is serialised into the prompt (name, description, and a full JSON Schema for its parameters) on every single call, including the calls where the model uses no tool at all.\n\nHere is a modest one:\n\n```json\n{\n  \"name\": \"search_orders\",\n  \"description\": \"Search a customer's orders by date range or status. Returns at most 20 orders, newest first. Only covers the last 90 days; older orders are in the archive and must be fetched with get_archived_order.\",\n  \"input_schema\": {\n    \"type\": \"object\",\n    \"properties\": {\n      \"customer_id\": {\"type\": \"string\", \"description\": \"Internal id, format c_NNNN.\"},\n      \"status\": {\"type\": \"string\", \"enum\": [\"placed\", \"shipped\", \"delivered\", \"cancelled\", \"any\"]},\n      \"since\": {\"type\": \"string\", \"description\": \"ISO 8601 date, e.g. 2026-07-01.\"}\n    },\n    \"required\": [\"customer_id\"]\n  }\n}\n```\n\nThat is around 180 tokens once rendered. It is a good definition: the description does real work, which we will come back to. But it is 180 tokens on every request for the rest of its life.\n\nTwenty tools of that size is roughly **3,600 tokens per call**. At a million calls a month that is 3.6 billion input tokens you are paying for before the user has typed anything. Most of it is cacheable, which we will use, but cacheable is not free and the cache is not always warm.\n\n## Count it, do not estimate it\n\nThe rendering is done by the provider, not by you, so the only honest count is the one the API reports. Every major API returns an input token count on the response, and several offer a count-tokens endpoint that charges nothing. Run one request with the full toolset and one with `tools` omitted entirely. The difference is the standing cost of your tools.\n\nDo this once and write the number in the budget ledger next to the system prompt. Teams are routinely surprised: a toolset assembled over six months by six people is frequently the second-largest block in the request, behind only retrieved passages.\n\nIf you are on an open model and want to see the rendering itself, `transformers` will show you. `tokenizer.apply_chat_template(messages, tools=tools, tokenize=False)` prints the exact string the model receives, tool schemas and all. It costs nothing and it is the fastest way to discover that your framework is injecting a paragraph you never wrote.\n\n## The second cost is worse than the first\n\nTokens are the cost you can see. The cost you cannot see is that selection accuracy falls as the toolset grows.\n\nThe mechanism is not mysterious. Choosing a tool is a classification problem the model solves by reading descriptions, and adding options to a classification problem makes it harder, especially when the options overlap. With four tools that do obviously different things, selection is close to perfect. With forty, several of which could plausibly serve the same request, the model starts choosing inconsistently: the same question on two runs picks two different tools, both defensible, one of them wrong for your system.\n\nPublished figures for where this begins vary widely, because it depends entirely on how distinct your descriptions are, and you should distrust any single number quoted at you. What does not vary is the direction, and that you can measure yourself. Write thirty representative user messages. Record which tool you believe should be called for each. Run them and compare. That is a tool-selection accuracy score, it costs thirty cheap calls, and it is the only figure about your toolset that means anything.\n\n> Figure: Choosing the right tool as the toolset grows. Bars (%): 6 tools 94; 12 tools 91; 24 tools 84; 48 tools 71. Selection accuracy on one routing set whose descriptions overlap in the usual ways. Published figures for where the fall begins vary enormously, because it depends entirely on how distinct the descriptions are. Distrust any single number quoted at you, including this one, and run the measurement yourself. The direction does not vary. The token cost is separate and exact: twenty tools at 180 tokens each is 3,600 tokens on every call before anybody types.\n\n## Where they sit, and why order matters\n\nTool definitions are rendered near the top of the request, above the conversation, usually adjacent to the system prompt. That is good news: they belong to the stable prefix and are cached along with it.\n\nIt is good news with a condition. Prefix caching matches from the first token forward, so the *order* of your tools is part of the cache key. If your code builds the tool list from a Python dictionary, or from a database query with no `ORDER BY`, or from a set, the order can change between deployments, or between processes, and every change is a full cache miss on the whole prefix.\n\n```python\ntools = sorted(registry.enabled_for(user), key=lambda t: t[\"name\"])\n```\n\nOne line. It costs nothing and it removes an entire class of invisible cost regression.\n\nThe same logic governs adding a tool. Appending a twenty-first tool to the end of the list invalidates the cache from that point on, which is cheap. Inserting it alphabetically into the middle invalidates everything after it, which is not. If you sort, accept that adding a tool named `archive_order` is a bigger cache event than adding one named `zip_export`, and schedule the deploy accordingly rather than being puzzled by the bill.\n\n## The limitation\n\nNone of this tells you whether a tool earns its place. Token cost you can measure exactly; selection damage you can measure roughly; the value of a tool being available is the hardest of the three, because it only shows up on the requests that needed it, which may be two percent of traffic.\n\nThe test that works is removal. Take the tool out, run the golden set, and see what breaks. If nothing breaks, the set does not cover what the tool is for, which is itself worth knowing, or the tool is dead weight. Teams almost never do this, which is why almost every mature toolset has at least one tool nothing has called in months, still being paid for on every request."},"check":{"question":"A service defines 22 tools. On a request where the model answers directly and calls nothing, what happens to the tool definitions?","options":[{"text":"They are skipped, because the provider only serialises tools once the model signals it wants one.","correct":false,"tempting":"Treats tool selection as a second round trip; the definitions must be present in the same request for the model to choose at all."},{"text":"All 22 are serialised into the request and billed as input tokens.","correct":true},{"text":"Only the tools whose descriptions match the user's message are included, by a relevance filter in the API.","correct":false,"tempting":"No provider filters your toolset for you; any such filtering is something you build, and it then becomes your cache problem."},{"text":"They are billed at the cached-input rate automatically, so the cost is negligible regardless of size.","correct":false,"tempting":"Caching is conditional on a stable prefix and a warm cache, and the cached rate is a discount rather than zero."}]},"prev":{"slug":"how-a-schema-is-enforced","title":"How a schema is actually enforced","url":"https://addaly.com/learn/context-engineering/how-a-schema-is-enforced"},"next":{"slug":"tool-descriptions-as-routing","title":"The description is the prompt that picks the tool","url":"https://addaly.com/learn/context-engineering/tool-descriptions-as-routing"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Every tool costs tokens before anyone calls it', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/tool-definitions-are-context (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Every tool costs tokens before anyone calls it","courseTitle":"Context Engineering","module":{"index":7,"slug":"tools-schemas-and-the-agent-window","title":"Tools, schemas and the agent’s window"},"answer":null,"keyIdea":"Tool schemas are serialised into every request whether or not a tool is used, so a large toolset is a fixed tax on every call and a measurable drag on the model’s ability to pick the right one.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"%","title":"Choosing the right tool as the toolset grows","series":[{"label":"6 tools","value":94},{"label":"12 tools","value":91},{"label":"24 tools","value":84},{"label":"48 tools","value":71,"highlight":true}],"caption":"Selection accuracy on one routing set whose descriptions overlap in the usual ways. Published figures for where the fall begins vary enormously, because it depends entirely on how distinct the descriptions are. Distrust any single number quoted at you, including this one, and run the measurement yourself. The direction does not vary. The token cost is separate and exact: twenty tools at 180 tokens each is 3,600 tokens on every call before anybody types."},"alt":"Figure: Choosing the right tool as the toolset grows. Bars (%): 6 tools 94; 12 tools 91; 24 tools 84; 48 tools 71. Selection accuracy on one routing set whose descriptions overlap in the usual ways. Published figures for where the fall begins vary enormously, because it depends entirely on how distinct the descriptions are. Distrust any single number quoted at you, including this one, and run the measurement yourself. The direction does not vary. The token cost is separate and exact: twenty tools at 180 tokens each is 3,600 tokens on every call before anybody types."}},"blocks":[{"t":"h2","id":"the-bill-arrives-before-the-tool-runs","runs":[{"text":"The bill arrives before the tool runs"}]},{"t":"p","runs":[{"text":"A tool feels like a function: it costs something when you call it. In a language model request it is not like that. Every tool you make available is serialised into the prompt (name, description, and a full JSON Schema for its parameters) on every single call, including the calls where the model uses no tool at all."}]},{"t":"p","runs":[{"text":"Here is a modest one:"}]},{"t":"code","lang":"json","label":"JSON","text":"{\n  \"name\": \"search_orders\",\n  \"description\": \"Search a customer's orders by date range or status. Returns at most 20 orders, newest first. Only covers the last 90 days; older orders are in the archive and must be fetched with get_archived_order.\",\n  \"input_schema\": {\n    \"type\": \"object\",\n    \"properties\": {\n      \"customer_id\": {\"type\": \"string\", \"description\": \"Internal id, format c_NNNN.\"},\n      \"status\": {\"type\": \"string\", \"enum\": [\"placed\", \"shipped\", \"delivered\", \"cancelled\", \"any\"]},\n      \"since\": {\"type\": \"string\", \"description\": \"ISO 8601 date, e.g. 2026-07-01.\"}\n    },\n    \"required\": [\"customer_id\"]\n  }\n}"},{"t":"p","runs":[{"text":"That is around 180 tokens once rendered. It is a good definition: the description does real work, which we will come back to. But it is 180 tokens on every request for the rest of its life."}]},{"t":"p","runs":[{"text":"Twenty tools of that size is roughly "},{"text":"3,600 tokens per call","b":true},{"text":". At a million calls a month that is 3.6 billion input tokens you are paying for before the user has typed anything. Most of it is cacheable, which we will use, but cacheable is not free and the cache is not always warm."}]},{"t":"h2","id":"count-it-do-not-estimate-it","runs":[{"text":"Count it, do not estimate it"}]},{"t":"p","runs":[{"text":"The rendering is done by the provider, not by you, so the only honest count is the one the API reports. Every major API returns an input token count on the response, and several offer a count-tokens endpoint that charges nothing. Run one request with the full toolset and one with "},{"text":"tools","code":true},{"text":" omitted entirely. The difference is the standing cost of your tools."}]},{"t":"p","runs":[{"text":"Do this once and write the number in the budget ledger next to the system prompt. Teams are routinely surprised: a toolset assembled over six months by six people is frequently the second-largest block in the request, behind only retrieved passages."}]},{"t":"p","runs":[{"text":"If you are on an open model and want to see the rendering itself, "},{"text":"transformers","code":true},{"text":" will show you. "},{"text":"tokenizer.apply_chat_template(messages, tools=tools, tokenize=False)","code":true},{"text":" prints the exact string the model receives, tool schemas and all. It costs nothing and it is the fastest way to discover that your framework is injecting a paragraph you never wrote."}]},{"t":"h2","id":"the-second-cost-is-worse-than-the-first","runs":[{"text":"The second cost is worse than the first"}]},{"t":"p","runs":[{"text":"Tokens are the cost you can see. The cost you cannot see is that selection accuracy falls as the toolset grows."}]},{"t":"p","runs":[{"text":"The mechanism is not mysterious. Choosing a tool is a classification problem the model solves by reading descriptions, and adding options to a classification problem makes it harder, especially when the options overlap. With four tools that do obviously different things, selection is close to perfect. With forty, several of which could plausibly serve the same request, the model starts choosing inconsistently: the same question on two runs picks two different tools, both defensible, one of them wrong for your system."}]},{"t":"p","runs":[{"text":"Published figures for where this begins vary widely, because it depends entirely on how distinct your descriptions are, and you should distrust any single number quoted at you. What does not vary is the direction, and that you can measure yourself. Write thirty representative user messages. Record which tool you believe should be called for each. Run them and compare. That is a tool-selection accuracy score, it costs thirty cheap calls, and it is the only figure about your toolset that means anything."}]},{"t":"h2","id":"where-they-sit-and-why-order-matters","runs":[{"text":"Where they sit, and why order matters"}]},{"t":"p","runs":[{"text":"Tool definitions are rendered near the top of the request, above the conversation, usually adjacent to the system prompt. That is good news: they belong to the stable prefix and are cached along with it."}]},{"t":"p","runs":[{"text":"It is good news with a condition. Prefix caching matches from the first token forward, so the "},{"text":"order","i":true},{"text":" of your tools is part of the cache key. If your code builds the tool list from a Python dictionary, or from a database query with no "},{"text":"ORDER BY","code":true},{"text":", or from a set, the order can change between deployments, or between processes, and every change is a full cache miss on the whole prefix."}]},{"t":"code","lang":"python","label":"Python","text":"tools = sorted(registry.enabled_for(user), key=lambda t: t[\"name\"])"},{"t":"p","runs":[{"text":"One line. It costs nothing and it removes an entire class of invisible cost regression."}]},{"t":"p","runs":[{"text":"The same logic governs adding a tool. Appending a twenty-first tool to the end of the list invalidates the cache from that point on, which is cheap. Inserting it alphabetically into the middle invalidates everything after it, which is not. If you sort, accept that adding a tool named "},{"text":"archive_order","code":true},{"text":" is a bigger cache event than adding one named "},{"text":"zip_export","code":true},{"text":", and schedule the deploy accordingly rather than being puzzled by the bill."}]},{"t":"h2","id":"the-limitation","runs":[{"text":"The limitation"}],"role":"limit"},{"t":"p","runs":[{"text":"None of this tells you whether a tool earns its place. Token cost you can measure exactly; selection damage you can measure roughly; the value of a tool being available is the hardest of the three, because it only shows up on the requests that needed it, which may be two percent of traffic."}]},{"t":"p","runs":[{"text":"The test that works is removal. Take the tool out, run the golden set, and see what breaks. If nothing breaks, the set does not cover what the tool is for, which is itself worth knowing, or the tool is dead weight. Teams almost never do this, which is why almost every mature toolset has at least one tool nothing has called in months, still being paid for on every request."}]}],"check":{"question":[{"text":"A service defines 22 tools. On a request where the model answers directly and calls nothing, what happens to the tool definitions?"}],"options":[{"text":[{"text":"They are skipped, because the provider only serialises tools once the model signals it wants one."}],"correct":false,"tempting":[{"text":"Treats tool selection as a second round trip; the definitions must be present in the same request for the model to choose at all."}]},{"text":[{"text":"All 22 are serialised into the request and billed as input tokens."}],"correct":true,"tempting":null},{"text":[{"text":"Only the tools whose descriptions match the user’s message are included, by a relevance filter in the API."}],"correct":false,"tempting":[{"text":"No provider filters your toolset for you; any such filtering is something you build, and it then becomes your cache problem."}]},{"text":[{"text":"They are billed at the cached-input rate automatically, so the cost is negligible regardless of size."}],"correct":false,"tempting":[{"text":"Caching is conditional on a stable prefix and a warm cache, and the cached rate is a discount rather than zero."}]}],"order":[3,0,1,2],"why":null},"prev":{"slug":"how-a-schema-is-enforced","title":"How a schema is actually enforced","minutes":10},"next":{"slug":"tool-descriptions-as-routing","title":"The description is the prompt that picks the tool","minutes":9}}}