{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"anatomy-of-a-request","title":"What the model literally receives","url":"https://addaly.com/learn/context-engineering/anatomy-of-a-request","position":4,"of":87,"minutes":10,"answer":"Your message list is rendered into one token sequence by a chat template that adds delimiters, role labels and serialised tool schemas: you pay for all of it, and the system role's authority is training.","keyIdea":"Your message list is rendered into one token sequence by a chat template that adds delimiters, role labels and serialised tool schemas: you pay for all of it, and the system role's authority is training, not a permission bit."},"body":{"format":"markdown","text":"## The message list is not the prompt\n\nYou send a list of dictionaries. The model receives a single sequence of integers. Between those two facts sits a chat template, and it is where a surprising share of your token budget and several of your bugs live.\n\nHere is the transformation, made visible with an open-weights model so you can run it yourself:\n\n```python\nfrom transformers import AutoTokenizer\ntok = AutoTokenizer.from_pretrained(\"Qwen/Qwen2.5-7B-Instruct\")\nmsgs = [\n    {\"role\": \"system\", \"content\": \"You are terse.\"},\n    {\"role\": \"user\", \"content\": \"Hello\"},\n]\nprint(tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True))\n```\n\nThe output is not `You are terse. Hello`. It is a marked-up string along the lines of:\n\n```\n<|im_start|>system\nYou are terse.<|im_end|>\n<|im_start|>user\nHello<|im_end|>\n<|im_start|>assistant\n```\n\nSeven words of yours became roughly twenty tokens. The wrapper is small here and stops being small once you have twelve turns and six tools.\n\n## What the wrapper actually contains\n\nFour things you did not write are in every request:\n\n1. **Turn delimiters.** `<|im_start|>`, `<|eot_id|>`, `[INST]`, depending on the family. Two to four tokens per message, every message, forever.\n2. **Role labels.** The literal strings `system`, `user`, `assistant`, `tool`.\n3. **A generation prompt.** The trailing tokens that tell the model it is now the assistant's turn. Omit it in a local setup and the model will happily continue writing the user's message instead: a classic first-day bug when people build their own serving loop.\n4. **Tool definitions, serialised.** This is the expensive one. Your six JSON schemas are rendered into the prompt as text, usually with an instruction paragraph about how to call them. Hosted APIs do it server-side, which is why the cost is invisible until you read the usage numbers.\n\nOn a hosted API you never see this string. You still pay for it, and your token estimate is wrong by however much of it you ignored.\n\n> Figure: What the chat template hands to the model. Layers, top to bottom: Turn delimiters (Two to four tokens per message, added by the template rather than by you); Role labels (The literal strings system, user, assistant and tool, once per turn); Your system content (The only part you wrote, and the only part most estimates count); Tool definitions, serialised (Six schemas at 150 to 250 tokens each: about 1,200 tokens on every call); Every earlier turn (Resent whole and wrapped again, which is where the quadratic bill comes from); The generation prompt (The trailing tokens saying the assistant speaks next). Seven words became roughly twenty tokens in the lesson's example, and the wrapper stops being small at twelve turns and six tools. On a hosted API you never see this string. You pay for all of it, and your estimate is wrong by whatever part of it you ignored.\n\n## Roles are a convention, not a permission system\n\nThe most consequential fact in this lesson: the `system` role is not privileged at the level of the machine. It is a string in a particular position, wrapped in particular delimiters, that the model was trained to weight heavily.\n\nThat training is real and it works most of the time. It is also the entire basis of the system role's authority. There is no flag in the tensor that says *obey this one*. Which is why a sufficiently insistent instruction inside a retrieved document sometimes wins, and why the defence against that is never \"put it in the system prompt more firmly\".\n\nThe practical corollary: if you serialise a user's text into your system prompt through string interpolation, you have handed them the strongest position in the request. Interpolating untrusted text into the system prompt is the single highest-severity assembly bug, and it is usually written by somebody trying to be helpful with an f-string.\n\n## Tool definitions cost real money\n\nA modest tool schema (name, description, four parameters with descriptions) serialises to roughly 150 to 250 tokens. Six tools is around 1,200 tokens on every single call, whether or not any tool is used.\n\nAt a typical hosted input price and a million calls a month, that block alone is a line item worth arguing about. It is also usually the least edited text in the whole system, because it lives in code next to the function rather than in a prompt file that somebody owns.\n\nMeasure it directly:\n\n```python\n# Send the same request with and without tools; the difference is the schema cost.\na = client.messages.count_tokens(model=M, messages=msgs)\nb = client.messages.count_tokens(model=M, messages=msgs, tools=TOOLS)\nprint(b.input_tokens - a.input_tokens)\n```\n\n## Multimodal blocks are tokens too\n\nAn image is not free and it is not one token. Most vision models cut an image into patches and each patch becomes a token or a small group of them. A 1024×1024 image commonly lands somewhere between 700 and 1,600 tokens depending on the model and how it tiles.\n\nTwo consequences. A four-page scanned PDF sent as images can cost more than the same document sent as extracted text, often several times more. And a chat where the user has pasted six screenshots is a context problem, not a UX problem.\n\nAudio behaves similarly, priced per second of input rather than per patch, and video is currently the most expensive thing you can put in a window by a wide margin.\n\n## Prefill caching happens on this string\n\nOne more reason to know what the wrapper looks like: caching operates on the token sequence, including the delimiters. Two requests that differ only in a whitespace character inside the system content produce different token sequences and therefore two different cache entries.\n\nThis is why a template that stamps the current timestamp into the system prompt destroys the cache for the entire prefix below it. The lesson on caching does the arithmetic; the point here is that the arithmetic operates on this string, not on your message list.\n\n## The exercise\n\nTake a real request from your system. Render it with `apply_chat_template` for the nearest open model you can find, print it, and read it as text. Almost everybody who does this the first time finds one of these: a stray blank line multiplying across turns, a tool description written as a note to a colleague rather than to a model, or an entire block they forgot they were sending."},"check":{"question":"A developer interpolates a user's uploaded document into the system prompt string, reasoning that the system role makes the model treat surrounding instructions as authoritative. What is wrong with the reasoning?","options":[{"text":"System prompts are truncated first when the window overflows, so the document would be lost.","correct":false,"tempting":"Describes a truncation policy that is neither universal nor the issue; the danger is present even when nothing is truncated."},{"text":"The system role's weight comes from training on a position and delimiter pattern, so text placed there inherits that weight, including instructions inside the user's document.","correct":true},{"text":"Documents must be sent as user messages because the API rejects long system content.","correct":false,"tempting":"Cites a limit that generally does not exist; system content can be long, and the objection is about trust rather than size."},{"text":"Interpolation prevents prefix caching, which is the main cost of the mistake.","correct":false,"tempting":"Names a genuine secondary effect and mistakes it for the primary one; the cache loss is money, the trust inversion is a security failure."}]},"prev":{"slug":"the-four-kinds-of-context","title":"Four kinds of context, four different problems","url":"https://addaly.com/learn/context-engineering/the-four-kinds-of-context"},"next":{"slug":"system-prompt-versus-turn","title":"What belongs in the system prompt","url":"https://addaly.com/learn/context-engineering/system-prompt-versus-turn"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What the model literally receives', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/anatomy-of-a-request (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What the model literally receives","courseTitle":"Context Engineering","module":{"index":1,"slug":"the-window-as-a-surface","title":"The window as an engineering surface"},"answer":null,"keyIdea":"Your message list is rendered into one token sequence by a chat template that adds delimiters, role labels and serialised tool schemas: you pay for all of it, and the system role’s authority is training, not a permission bit.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"stack","title":"What the chat template hands to the model","layers":[{"label":"Turn delimiters","detail":"Two to four tokens per message, added by the template rather than by you"},{"label":"Role labels","detail":"The literal strings system, user, assistant and tool, once per turn"},{"label":"Your system content","detail":"The only part you wrote, and the only part most estimates count"},{"label":"Tool definitions, serialised","detail":"Six schemas at 150 to 250 tokens each: about 1,200 tokens on every call"},{"label":"Every earlier turn","detail":"Resent whole and wrapped again, which is where the quadratic bill comes from"},{"label":"The generation prompt","detail":"The trailing tokens saying the assistant speaks next"}],"caption":"Seven words became roughly twenty tokens in the lesson’s example, and the wrapper stops being small at twelve turns and six tools. On a hosted API you never see this string. You pay for all of it, and your estimate is wrong by whatever part of it you ignored."},"alt":"Figure: What the chat template hands to the model. Layers, top to bottom: Turn delimiters (Two to four tokens per message, added by the template rather than by you); Role labels (The literal strings system, user, assistant and tool, once per turn); Your system content (The only part you wrote, and the only part most estimates count); Tool definitions, serialised (Six schemas at 150 to 250 tokens each: about 1,200 tokens on every call); Every earlier turn (Resent whole and wrapped again, which is where the quadratic bill comes from); The generation prompt (The trailing tokens saying the assistant speaks next). Seven words became roughly twenty tokens in the lesson’s example, and the wrapper stops being small at twelve turns and six tools. On a hosted API you never see this string. You pay for all of it, and your estimate is wrong by whatever part of it you ignored."}},"blocks":[{"t":"h2","id":"the-message-list-is-not-the-prompt","runs":[{"text":"The message list is not the prompt"}]},{"t":"p","runs":[{"text":"You send a list of dictionaries. The model receives a single sequence of integers. Between those two facts sits a chat template, and it is where a surprising share of your token budget and several of your bugs live."}]},{"t":"p","runs":[{"text":"Here is the transformation, made visible with an open-weights model so you can run it yourself:"}]},{"t":"code","lang":"python","label":"Python","text":"from transformers import AutoTokenizer\ntok = AutoTokenizer.from_pretrained(\"Qwen/Qwen2.5-7B-Instruct\")\nmsgs = [\n    {\"role\": \"system\", \"content\": \"You are terse.\"},\n    {\"role\": \"user\", \"content\": \"Hello\"},\n]\nprint(tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True))"},{"t":"p","runs":[{"text":"The output is not "},{"text":"You are terse. Hello","code":true},{"text":". It is a marked-up string along the lines of:"}]},{"t":"code","lang":"","label":"Text","text":"<|im_start|>system\nYou are terse.<|im_end|>\n<|im_start|>user\nHello<|im_end|>\n<|im_start|>assistant"},{"t":"p","runs":[{"text":"Seven words of yours became roughly twenty tokens. The wrapper is small here and stops being small once you have twelve turns and six tools."}]},{"t":"h2","id":"what-the-wrapper-actually-contains","runs":[{"text":"What the wrapper actually contains"}]},{"t":"p","runs":[{"text":"Four things you did not write are in every request:"}]},{"t":"ol","start":1,"items":[[{"text":"Turn delimiters.","b":true},{"text":" "},{"text":"<|im_start|>","code":true},{"text":", "},{"text":"<|eot_id|>","code":true},{"text":", "},{"text":"[INST]","code":true},{"text":", depending on the family. Two to four tokens per message, every message, forever."}],[{"text":"Role labels.","b":true},{"text":" The literal strings "},{"text":"system","code":true},{"text":", "},{"text":"user","code":true},{"text":", "},{"text":"assistant","code":true},{"text":", "},{"text":"tool","code":true},{"text":"."}],[{"text":"A generation prompt.","b":true},{"text":" The trailing tokens that tell the model it is now the assistant’s turn. Omit it in a local setup and the model will happily continue writing the user’s message instead: a classic first-day bug when people build their own serving loop."}],[{"text":"Tool definitions, serialised.","b":true},{"text":" This is the expensive one. Your six JSON schemas are rendered into the prompt as text, usually with an instruction paragraph about how to call them. Hosted APIs do it server-side, which is why the cost is invisible until you read the usage numbers."}]]},{"t":"p","runs":[{"text":"On a hosted API you never see this string. You still pay for it, and your token estimate is wrong by however much of it you ignored."}]},{"t":"h2","id":"roles-are-a-convention-not-a-permission-system","runs":[{"text":"Roles are a convention, not a permission system"}]},{"t":"p","runs":[{"text":"The most consequential fact in this lesson: the "},{"text":"system","code":true},{"text":" role is not privileged at the level of the machine. It is a string in a particular position, wrapped in particular delimiters, that the model was trained to weight heavily."}]},{"t":"p","runs":[{"text":"That training is real and it works most of the time. It is also the entire basis of the system role’s authority. There is no flag in the tensor that says "},{"text":"obey this one","i":true},{"text":". Which is why a sufficiently insistent instruction inside a retrieved document sometimes wins, and why the defence against that is never “put it in the system prompt more firmly”."}]},{"t":"p","runs":[{"text":"The practical corollary: if you serialise a user’s text into your system prompt through string interpolation, you have handed them the strongest position in the request. Interpolating untrusted text into the system prompt is the single highest-severity assembly bug, and it is usually written by somebody trying to be helpful with an f-string."}]},{"t":"h2","id":"tool-definitions-cost-real-money","runs":[{"text":"Tool definitions cost real money"}]},{"t":"p","runs":[{"text":"A modest tool schema (name, description, four parameters with descriptions) serialises to roughly 150 to 250 tokens. Six tools is around 1,200 tokens on every single call, whether or not any tool is used."}]},{"t":"p","runs":[{"text":"At a typical hosted input price and a million calls a month, that block alone is a line item worth arguing about. It is also usually the least edited text in the whole system, because it lives in code next to the function rather than in a prompt file that somebody owns."}]},{"t":"p","runs":[{"text":"Measure it directly:"}]},{"t":"code","lang":"python","label":"Python","text":"# Send the same request with and without tools; the difference is the schema cost.\na = client.messages.count_tokens(model=M, messages=msgs)\nb = client.messages.count_tokens(model=M, messages=msgs, tools=TOOLS)\nprint(b.input_tokens - a.input_tokens)"},{"t":"h2","id":"multimodal-blocks-are-tokens-too","runs":[{"text":"Multimodal blocks are tokens too"}]},{"t":"p","runs":[{"text":"An image is not free and it is not one token. Most vision models cut an image into patches and each patch becomes a token or a small group of them. A 1024×1024 image commonly lands somewhere between 700 and 1,600 tokens depending on the model and how it tiles."}]},{"t":"p","runs":[{"text":"Two consequences. A four-page scanned PDF sent as images can cost more than the same document sent as extracted text, often several times more. And a chat where the user has pasted six screenshots is a context problem, not a UX problem."}]},{"t":"p","runs":[{"text":"Audio behaves similarly, priced per second of input rather than per patch, and video is currently the most expensive thing you can put in a window by a wide margin."}]},{"t":"h2","id":"prefill-caching-happens-on-this-string","runs":[{"text":"Prefill caching happens on this string"}]},{"t":"p","runs":[{"text":"One more reason to know what the wrapper looks like: caching operates on the token sequence, including the delimiters. Two requests that differ only in a whitespace character inside the system content produce different token sequences and therefore two different cache entries."}]},{"t":"p","runs":[{"text":"This is why a template that stamps the current timestamp into the system prompt destroys the cache for the entire prefix below it. The lesson on caching does the arithmetic; the point here is that the arithmetic operates on this string, not on your message list."}]},{"t":"h2","id":"the-exercise","runs":[{"text":"The exercise"}]},{"t":"p","runs":[{"text":"Take a real request from your system. Render it with "},{"text":"apply_chat_template","code":true},{"text":" for the nearest open model you can find, print it, and read it as text. Almost everybody who does this the first time finds one of these: a stray blank line multiplying across turns, a tool description written as a note to a colleague rather than to a model, or an entire block they forgot they were sending."}]}],"check":{"question":[{"text":"A developer interpolates a user’s uploaded document into the system prompt string, reasoning that the system role makes the model treat surrounding instructions as authoritative. What is wrong with the reasoning?"}],"options":[{"text":[{"text":"System prompts are truncated first when the window overflows, so the document would be lost."}],"correct":false,"tempting":[{"text":"Describes a truncation policy that is neither universal nor the issue; the danger is present even when nothing is truncated."}]},{"text":[{"text":"The system role’s weight comes from training on a position and delimiter pattern, so text placed there inherits that weight, including instructions inside the user’s document."}],"correct":true,"tempting":null},{"text":[{"text":"Documents must be sent as user messages because the API rejects long system content."}],"correct":false,"tempting":[{"text":"Cites a limit that generally does not exist; system content can be long, and the objection is about trust rather than size."}]},{"text":[{"text":"Interpolation prevents prefix caching, which is the main cost of the mistake."}],"correct":false,"tempting":[{"text":"Names a genuine secondary effect and mistakes it for the primary one; the cache loss is money, the trust inversion is a security failure."}]}],"order":[2,3,0,1],"why":null},"prev":{"slug":"the-four-kinds-of-context","title":"Four kinds of context, four different problems","minutes":8},"next":{"slug":"system-prompt-versus-turn","title":"What belongs in the system prompt","minutes":8}}}