{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"where-the-instruction-goes","title":"Top, bottom, or both","url":"https://addaly.com/learn/context-engineering/where-the-instruction-goes","position":34,"of":87,"minutes":8,"answer":"Caching pulls instructions to the top and attention pulls them towards the question.","keyIdea":"Caching pulls instructions to the top and attention pulls them towards the question, so keep the large stable block cached at the front and pay for a short uncached reminder of the format and the one critical constraint next to the question."},"body":{"format":"markdown","text":"## The conflict\n\nTwo good rules point in opposite directions.\n\n**Caching wants instructions first.** Prefix caching matches from the first token forward, so everything stable belongs at the top. Instructions are the most stable thing you have.\n\n**Attention favours the end.** In a long window, an instruction 40,000 tokens above the question is competing with everything in between, and the model's strongest attention is near where generation begins.\n\nWith a 2,000-token request the conflict does not exist; put the instructions at the top and stop thinking about it. With a 60,000-token request it is real, and teams discover it as \"the model stopped following the format when we added the documents\", which is exactly what happened, and the format instruction did not move, the distance did.\n\n## The resolution most systems land on\n\nA long stable block at the top and a short reminder at the bottom.\n\n```\n[system: role, policy, output contract, exemplars]     <- cached, ~1,200 tokens\n[retrieved passages]                                    <- variable, ~9,000 tokens\n[conversation history]                                  <- variable\n[user question]\n[reminder: the two constraints that matter most]        <- ~40 tokens, uncached\n```\n\nThe reminder is not a copy of the system prompt. It is the one or two things that break when they are far away, which in practice are almost always the output format and the single most-violated constraint. Forty tokens, adjacent to the question.\n\nYou keep the cache discount on the large stable block, and you pay 40 uncached tokens for the part that needs proximity. That is the trade, and it is a good one.\n\n## Where the question goes\n\nSeparately from the instructions: with a long body of evidence, putting the question **after** the material is the common default, and it usually beats putting it only before.\n\nThe reasoning is causal attention. A question at the top is read before any evidence exists; the model cannot use it to guide reading, because there is nothing yet to read. A question at the bottom sits closest to the point of generation, in the strongest position available.\n\nThe stronger version is both: a short framing at the top so the model knows what it is reading for, and the full question at the bottom. This costs a few dozen tokens and is worth measuring, because the gain varies by model and by how long the evidence is.\n\n## What does not work\n\n**Making the instruction louder.** Capital letters, \"IMPORTANT\", \"YOU MUST\", exclamation marks. These have a small effect at best, and they degrade as context grows, because the problem is distance rather than emphasis. Teams escalate the shouting instead of moving the text, and the prompt becomes unreadable to the humans who maintain it while behaving no better.\n\n**Repeating the whole prompt at the end.** Doubles the instruction cost, and creates two copies that will eventually disagree when somebody edits one. The next lesson covers how to repeat safely.\n\n**Assuming the system role will carry it.** The system role gets strong weighting at the top of a short request. In a 100,000-token context it is still just tokens, a very long way away.\n\n## Measure it the same way\n\nThe experiment is the same shape as the position probe. Fix everything, vary the instruction placement across three arms (top only, bottom only, top and bottom) and score adherence rather than answer quality, because format compliance is what you are testing and it is cheap to grade with a regex or a schema validator.\n\nExpect small differences at short lengths and larger ones at long. Expect the \"both\" arm to win or tie, and to cost slightly more. Expect the result to differ between two models from the same vendor, which is the reason to re-run it when you upgrade.\n\n> Figure: Where the output contract sits, in a 60,000-token request. Bars (%): Top only 71; Bottom only 84; Top and bottom 91. Three arms, everything else held fixed, scoring adherence rather than answer quality. At 2,000 tokens the same three arms sat within a point of each other, which is why placement is a long-context problem and caching should decide it below about eight thousand. The winning arm is a forty-token reminder next to the question, not a second copy of the system prompt.\n\n## The rule of thumb, stated as a rule of thumb\n\nBelow roughly 8,000 tokens, placement barely matters and caching should decide it. Above roughly 30,000 tokens, a bottom reminder is usually worth its tokens. Between the two, measure.\n\nThose numbers are a starting point from common practice, not a finding. The point of the probe is that you can replace them with your own within a day, and then they are facts about your system rather than folklore about somebody else's."},"check":{"question":"A system that reliably produced valid JSON at 3,000 tokens starts producing prose at 40,000 tokens. The prompt text is unchanged. What is the most likely fix?","options":[{"text":"Emphasise the format instruction with stronger wording, since the model is under-weighting it.","correct":false,"tempting":"Treats a distance problem as an emphasis problem; emphasis does not shorten the gap between the instruction and generation."},{"text":"Restate the output contract in a short block immediately before or after the question.","correct":true},{"text":"Move the retrieved passages into the system message so they inherit its weighting.","correct":false,"tempting":"Puts variable, possibly untrusted material in the most authoritative position and destroys the cached prefix, for no positional gain."},{"text":"Reduce `max_tokens` so the model cannot produce long prose instead of JSON.","correct":false,"tempting":"Constrains length rather than form; a truncated prose answer is still not JSON."}]},"prev":{"slug":"building-a-position-probe","title":"Measuring your own position curve","url":"https://addaly.com/learn/context-engineering/building-a-position-probe"},"next":{"slug":"repeating-the-instruction","title":"Saying it twice without saying two things","url":"https://addaly.com/learn/context-engineering/repeating-the-instruction"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Top, bottom, or both', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/where-the-instruction-goes (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Top, bottom, or both","courseTitle":"Context Engineering","module":{"index":4,"slug":"position-and-order","title":"Position, order and what the model actually reads"},"answer":null,"keyIdea":"Caching pulls instructions to the top and attention pulls them towards the question, so keep the large stable block cached at the front and pay for a short uncached reminder of the format and the one critical constraint next to the question.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"%","title":"Where the output contract sits, in a 60,000-token request","series":[{"label":"Top only","value":71},{"label":"Bottom only","value":84},{"label":"Top and bottom","value":91,"highlight":true}],"caption":"Three arms, everything else held fixed, scoring adherence rather than answer quality. At 2,000 tokens the same three arms sat within a point of each other, which is why placement is a long-context problem and caching should decide it below about eight thousand. The winning arm is a forty-token reminder next to the question, not a second copy of the system prompt."},"alt":"Figure: Where the output contract sits, in a 60,000-token request. Bars (%): Top only 71; Bottom only 84; Top and bottom 91. Three arms, everything else held fixed, scoring adherence rather than answer quality. At 2,000 tokens the same three arms sat within a point of each other, which is why placement is a long-context problem and caching should decide it below about eight thousand. The winning arm is a forty-token reminder next to the question, not a second copy of the system prompt."}},"blocks":[{"t":"h2","id":"the-conflict","runs":[{"text":"The conflict"}]},{"t":"p","runs":[{"text":"Two good rules point in opposite directions."}]},{"t":"p","runs":[{"text":"Caching wants instructions first.","b":true},{"text":" Prefix caching matches from the first token forward, so everything stable belongs at the top. Instructions are the most stable thing you have."}]},{"t":"p","runs":[{"text":"Attention favours the end.","b":true},{"text":" In a long window, an instruction 40,000 tokens above the question is competing with everything in between, and the model’s strongest attention is near where generation begins."}]},{"t":"p","runs":[{"text":"With a 2,000-token request the conflict does not exist; put the instructions at the top and stop thinking about it. With a 60,000-token request it is real, and teams discover it as “the model stopped following the format when we added the documents”, which is exactly what happened, and the format instruction did not move, the distance did."}]},{"t":"h2","id":"the-resolution-most-systems-land-on","runs":[{"text":"The resolution most systems land on"}]},{"t":"p","runs":[{"text":"A long stable block at the top and a short reminder at the bottom."}]},{"t":"code","lang":"","label":"Text","text":"[system: role, policy, output contract, exemplars]     <- cached, ~1,200 tokens\n[retrieved passages]                                    <- variable, ~9,000 tokens\n[conversation history]                                  <- variable\n[user question]\n[reminder: the two constraints that matter most]        <- ~40 tokens, uncached"},{"t":"p","runs":[{"text":"The reminder is not a copy of the system prompt. It is the one or two things that break when they are far away, which in practice are almost always the output format and the single most-violated constraint. Forty tokens, adjacent to the question."}]},{"t":"p","runs":[{"text":"You keep the cache discount on the large stable block, and you pay 40 uncached tokens for the part that needs proximity. That is the trade, and it is a good one."}]},{"t":"h2","id":"where-the-question-goes","runs":[{"text":"Where the question goes"}]},{"t":"p","runs":[{"text":"Separately from the instructions: with a long body of evidence, putting the question "},{"text":"after","b":true},{"text":" the material is the common default, and it usually beats putting it only before."}]},{"t":"p","runs":[{"text":"The reasoning is causal attention. A question at the top is read before any evidence exists; the model cannot use it to guide reading, because there is nothing yet to read. A question at the bottom sits closest to the point of generation, in the strongest position available."}]},{"t":"p","runs":[{"text":"The stronger version is both: a short framing at the top so the model knows what it is reading for, and the full question at the bottom. This costs a few dozen tokens and is worth measuring, because the gain varies by model and by how long the evidence is."}]},{"t":"h2","id":"what-does-not-work","runs":[{"text":"What does not work"}]},{"t":"p","runs":[{"text":"Making the instruction louder.","b":true},{"text":" Capital letters, “IMPORTANT”, “YOU MUST”, exclamation marks. These have a small effect at best, and they degrade as context grows, because the problem is distance rather than emphasis. Teams escalate the shouting instead of moving the text, and the prompt becomes unreadable to the humans who maintain it while behaving no better."}]},{"t":"p","runs":[{"text":"Repeating the whole prompt at the end.","b":true},{"text":" Doubles the instruction cost, and creates two copies that will eventually disagree when somebody edits one. The next lesson covers how to repeat safely."}]},{"t":"p","runs":[{"text":"Assuming the system role will carry it.","b":true},{"text":" The system role gets strong weighting at the top of a short request. In a 100,000-token context it is still just tokens, a very long way away."}]},{"t":"h2","id":"measure-it-the-same-way","runs":[{"text":"Measure it the same way"}]},{"t":"p","runs":[{"text":"The experiment is the same shape as the position probe. Fix everything, vary the instruction placement across three arms (top only, bottom only, top and bottom) and score adherence rather than answer quality, because format compliance is what you are testing and it is cheap to grade with a regex or a schema validator."}]},{"t":"p","runs":[{"text":"Expect small differences at short lengths and larger ones at long. Expect the “both” arm to win or tie, and to cost slightly more. Expect the result to differ between two models from the same vendor, which is the reason to re-run it when you upgrade."}]},{"t":"h2","id":"the-rule-of-thumb-stated-as-a-rule-of-thumb","runs":[{"text":"The rule of thumb, stated as a rule of thumb"}]},{"t":"p","runs":[{"text":"Below roughly 8,000 tokens, placement barely matters and caching should decide it. Above roughly 30,000 tokens, a bottom reminder is usually worth its tokens. Between the two, measure."}]},{"t":"p","runs":[{"text":"Those numbers are a starting point from common practice, not a finding. The point of the probe is that you can replace them with your own within a day, and then they are facts about your system rather than folklore about somebody else’s."}]}],"check":{"question":[{"text":"A system that reliably produced valid JSON at 3,000 tokens starts producing prose at 40,000 tokens. The prompt text is unchanged. What is the most likely fix?"}],"options":[{"text":[{"text":"Emphasise the format instruction with stronger wording, since the model is under-weighting it."}],"correct":false,"tempting":[{"text":"Treats a distance problem as an emphasis problem; emphasis does not shorten the gap between the instruction and generation."}]},{"text":[{"text":"Restate the output contract in a short block immediately before or after the question."}],"correct":true,"tempting":null},{"text":[{"text":"Move the retrieved passages into the system message so they inherit its weighting."}],"correct":false,"tempting":[{"text":"Puts variable, possibly untrusted material in the most authoritative position and destroys the cached prefix, for no positional gain."}]},{"text":[{"text":"Reduce "},{"text":"max_tokens","code":true},{"text":" so the model cannot produce long prose instead of JSON."}],"correct":false,"tempting":[{"text":"Constrains length rather than form; a truncated prose answer is still not JSON."}]}],"order":[0,3,2,1],"why":null},"prev":{"slug":"building-a-position-probe","title":"Measuring your own position curve","minutes":9},"next":{"slug":"repeating-the-instruction","title":"Saying it twice without saying two things","minutes":8}}}