{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"lost-in-the-middle-the-experiment","title":"The experiment that named the problem","url":"https://addaly.com/learn/context-engineering/lost-in-the-middle-the-experiment","position":31,"of":87,"minutes":9,"answer":"Holding content constant and moving only the position of the answer document produces a U-shaped accuracy curve (high at the ends, lowest in the middle.","keyIdea":"Holding content constant and moving only the position of the answer document produces a U-shaped accuracy curve (high at the ends, lowest in the middle, in the worst case below answering with no documents at all) and the size of that dip is a property of your model and task that has to be measured, not assumed."},"body":{"format":"markdown","text":"## The setup, because the details matter\n\nIn 2023 a group at Stanford, Berkeley and Samaya ran an experiment that gave the field a phrase. The design was deliberately plain.\n\nTake a question with a known answer. Assemble a context of ten, twenty or thirty documents, exactly one of which contains the answer; the rest are real passages retrieved for the same question, so they are on-topic and wrong. Then vary one thing only: **where in the sequence the answer document sits.** First, fifth, tenth, last. Ask the question, score the answer, repeat across a large question set.\n\nIf position were irrelevant, the accuracy line would be flat. It was not flat. It was a U.\n\nAccuracy was highest when the answer document was first, fell as it moved towards the middle, and rose again towards the end. The dip was not small. In the twenty-document setting the middle positions cost a large fraction of the accuracy available at the ends, and in the most striking result, accuracy with the answer buried in the middle fell *below* the model's score with no documents at all: the retrieved material had made things worse than an unaided guess.\n\n> Figure: Moving one document and changing nothing else. Line chart of answers correct, per cent against position of the answer document among twenty: Answer document at this position (1, 75) (5, 62) (10, 56) (15, 59) (20, 68); No documents at all, answered from memory (1, 58) (20, 58). Twenty documents, exactly one of which holds the answer, the other nineteen real passages retrieved for the same question. Only the position of the answer changes. The middle falls below the closed-book baseline, which is the finding that named the problem. Current models are flatter than the 2023 curves, and the shape is still a property of your model, your task and your length rather than a constant.\n\nThey repeated it with a synthetic key-value lookup task, where there is no reasoning to blame, and got the same shape. That is the part that makes it a mechanism story rather than a task story.\n\n## Why a U\n\nThree explanations, none of them complete on their own, all of them worth knowing.\n\n**The training data has the same shape.** Instructions come at the beginning of documents and requests; questions come at the end. A model trained on that distribution learns that the extremes carry the load.\n\n**Attention sinks.** Softmax attention has to allocate a total of one across all positions. When nothing in particular is relevant, models dump the excess onto the earliest tokens, which acquire a structurally privileged role. A later lesson in this module covers the mechanism; the effect here is a strong pull towards the start of the sequence.\n\n**Recency, from causal attention.** Every token attends only backwards, so tokens near the end have the shortest path to the question and to the point where generation begins.\n\nThe middle has none of these advantages and all of the competition.\n\n## What has changed since, and what has not\n\nModels have improved. Curves measured on current long-context models are flatter than the 2023 ones, and some are close to flat on easy retrieval at moderate lengths.\n\nWhat has not changed is that the curve is a property of a specific model, a specific task and a specific length, and that it flattens least on the tasks that matter most: combining several facts, resolving a contradiction, noticing an exception. The harder the use of the material, the more position matters.\n\nSo the useful conclusion from the experiment is not \"put things at the end\". It is: **position is a variable in your system, it has an effect large enough to change outcomes, and you do not know its size until you measure it.** Any team that reorders passages on the basis of a paper without re-running it is making the same mistake as a team that ignores the paper.\n\n## What it implies immediately\n\nEven before measurement, three things follow.\n\n**Fewer passages is a positional argument as well as a cost one.** With five passages there is barely a middle. The k-curve lesson gave you the cost and distraction reasons; this is the third.\n\n**The ranking your retriever produces is not automatically the order to present.** Rank decides what gets in. Order in the window is a separate decision, made for a different reason, and the ordering lesson later in this module covers the options.\n\n**A fact in the window is not a fact the model has.** This is the sentence to remember. \"It's in the context\" is a statement about your assembler, not about the answer. Verification (a citation check, a containment assertion) is what turns the first into the second.\n\n## The limitation\n\nThe original experiment used a particular kind of task: extractive question answering over retrieved passages. Generalising from it to every use of a long context is over-reading it. Summarising a whole document, checking a contract for internal consistency, or refactoring a file all use the window differently, and the position effect for those is less well characterised.\n\nWhat travels is the method rather than the number. Vary one thing, hold everything else fixed, and plot. That is the next lesson."},"check":{"question":"In the original experiment, why does repeating the finding on a synthetic key-value lookup task strengthen the conclusion?","options":[{"text":"It shows the effect appears in a task with no reasoning or language understanding to blame, so position alone accounts for it.","correct":true},{"text":"It shows the effect scales with context length, which the document task could not demonstrate.","correct":false,"tempting":"Length scaling was studied separately by varying document count; the key-value task addresses task confounds, not length."},{"text":"It rules out retrieval quality as the cause, since the key-value pairs were not retrieved.","correct":false,"tempting":"Retrieval was already ruled out in the document version, where the answer document was placed by construction rather than found."},{"text":"It demonstrates the effect on models with different tokenisers, generalising the result across families.","correct":false,"tempting":"Confuses the choice of task with the choice of model; a second task says nothing about tokenisers."}]},"prev":{"slug":"measuring-retrieval-on-its-own","title":"Measuring retrieval without the model","url":"https://addaly.com/learn/context-engineering/measuring-retrieval-on-its-own"},"next":{"slug":"ordering-and-the-middle","title":"Ordering, and the fact that models skim","url":"https://addaly.com/learn/context-engineering/ordering-and-the-middle"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The experiment that named the problem', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/lost-in-the-middle-the-experiment (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The experiment that named the problem","courseTitle":"Context Engineering","module":{"index":4,"slug":"position-and-order","title":"Position, order and what the model actually reads"},"answer":null,"keyIdea":"Holding content constant and moving only the position of the answer document produces a U-shaped accuracy curve (high at the ends, lowest in the middle, in the worst case below answering with no documents at all) and the size of that dip is a property of your model and task that has to be measured, not assumed.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 4.1","figure":{"type":"line","title":"Moving one document and changing nothing else","series":[{"label":"Answer document at this position","points":[[1,75],[5,62],[10,56],[15,59],[20,68]]},{"label":"No documents at all, answered from memory","points":[[1,58],[20,58]]}],"xLabel":"position of the answer document among twenty","yLabel":"answers correct, per cent","caption":"Twenty documents, exactly one of which holds the answer, the other nineteen real passages retrieved for the same question. Only the position of the answer changes. The middle falls below the closed-book baseline, which is the finding that named the problem. Current models are flatter than the 2023 curves, and the shape is still a property of your model, your task and your length rather than a constant."},"alt":"Figure: Moving one document and changing nothing else. Line chart of answers correct, per cent against position of the answer document among twenty: Answer document at this position (1, 75) (5, 62) (10, 56) (15, 59) (20, 68); No documents at all, answered from memory (1, 58) (20, 58). Twenty documents, exactly one of which holds the answer, the other nineteen real passages retrieved for the same question. Only the position of the answer changes. The middle falls below the closed-book baseline, which is the finding that named the problem. Current models are flatter than the 2023 curves, and the shape is still a property of your model, your task and your length rather than a constant."}},"blocks":[{"t":"h2","id":"the-setup-because-the-details-matter","runs":[{"text":"The setup, because the details matter"}]},{"t":"p","runs":[{"text":"In 2023 a group at Stanford, Berkeley and Samaya ran an experiment that gave the field a phrase. The design was deliberately plain."}]},{"t":"p","runs":[{"text":"Take a question with a known answer. Assemble a context of ten, twenty or thirty documents, exactly one of which contains the answer; the rest are real passages retrieved for the same question, so they are on-topic and wrong. Then vary one thing only: "},{"text":"where in the sequence the answer document sits.","b":true},{"text":" First, fifth, tenth, last. Ask the question, score the answer, repeat across a large question set."}]},{"t":"p","runs":[{"text":"If position were irrelevant, the accuracy line would be flat. It was not flat. It was a U."}]},{"t":"p","runs":[{"text":"Accuracy was highest when the answer document was first, fell as it moved towards the middle, and rose again towards the end. The dip was not small. In the twenty-document setting the middle positions cost a large fraction of the accuracy available at the ends, and in the most striking result, accuracy with the answer buried in the middle fell "},{"text":"below","i":true},{"text":" the model’s score with no documents at all: the retrieved material had made things worse than an unaided guess."}]},{"t":"p","runs":[{"text":"They repeated it with a synthetic key-value lookup task, where there is no reasoning to blame, and got the same shape. That is the part that makes it a mechanism story rather than a task story."}]},{"t":"h2","id":"why-a-u","runs":[{"text":"Why a U"}]},{"t":"p","runs":[{"text":"Three explanations, none of them complete on their own, all of them worth knowing."}]},{"t":"p","runs":[{"text":"The training data has the same shape.","b":true},{"text":" Instructions come at the beginning of documents and requests; questions come at the end. A model trained on that distribution learns that the extremes carry the load."}]},{"t":"p","runs":[{"text":"Attention sinks.","b":true},{"text":" Softmax attention has to allocate a total of one across all positions. When nothing in particular is relevant, models dump the excess onto the earliest tokens, which acquire a structurally privileged role. A later lesson in this module covers the mechanism; the effect here is a strong pull towards the start of the sequence."}]},{"t":"p","runs":[{"text":"Recency, from causal attention.","b":true},{"text":" Every token attends only backwards, so tokens near the end have the shortest path to the question and to the point where generation begins."}]},{"t":"p","runs":[{"text":"The middle has none of these advantages and all of the competition."}]},{"t":"h2","id":"what-has-changed-since-and-what-has-not","runs":[{"text":"What has changed since, and what has not"}]},{"t":"p","runs":[{"text":"Models have improved. Curves measured on current long-context models are flatter than the 2023 ones, and some are close to flat on easy retrieval at moderate lengths."}]},{"t":"p","runs":[{"text":"What has not changed is that the curve is a property of a specific model, a specific task and a specific length, and that it flattens least on the tasks that matter most: combining several facts, resolving a contradiction, noticing an exception. The harder the use of the material, the more position matters."}]},{"t":"p","runs":[{"text":"So the useful conclusion from the experiment is not “put things at the end”. It is: "},{"text":"position is a variable in your system, it has an effect large enough to change outcomes, and you do not know its size until you measure it.","b":true},{"text":" Any team that reorders passages on the basis of a paper without re-running it is making the same mistake as a team that ignores the paper."}]},{"t":"h2","id":"what-it-implies-immediately","runs":[{"text":"What it implies immediately"}]},{"t":"p","runs":[{"text":"Even before measurement, three things follow."}]},{"t":"p","runs":[{"text":"Fewer passages is a positional argument as well as a cost one.","b":true},{"text":" With five passages there is barely a middle. The k-curve lesson gave you the cost and distraction reasons; this is the third."}]},{"t":"p","runs":[{"text":"The ranking your retriever produces is not automatically the order to present.","b":true},{"text":" Rank decides what gets in. Order in the window is a separate decision, made for a different reason, and the ordering lesson later in this module covers the options."}]},{"t":"p","runs":[{"text":"A fact in the window is not a fact the model has.","b":true},{"text":" This is the sentence to remember. “It’s in the context” is a statement about your assembler, not about the answer. Verification (a citation check, a containment assertion) is what turns the first into the second."}]},{"t":"h2","id":"the-limitation","runs":[{"text":"The limitation"}],"role":"limit"},{"t":"p","runs":[{"text":"The original experiment used a particular kind of task: extractive question answering over retrieved passages. Generalising from it to every use of a long context is over-reading it. Summarising a whole document, checking a contract for internal consistency, or refactoring a file all use the window differently, and the position effect for those is less well characterised."}]},{"t":"p","runs":[{"text":"What travels is the method rather than the number. Vary one thing, hold everything else fixed, and plot. That is the next lesson."}]}],"check":{"question":[{"text":"In the original experiment, why does repeating the finding on a synthetic key-value lookup task strengthen the conclusion?"}],"options":[{"text":[{"text":"It shows the effect appears in a task with no reasoning or language understanding to blame, so position alone accounts for it."}],"correct":true,"tempting":null},{"text":[{"text":"It shows the effect scales with context length, which the document task could not demonstrate."}],"correct":false,"tempting":[{"text":"Length scaling was studied separately by varying document count; the key-value task addresses task confounds, not length."}]},{"text":[{"text":"It rules out retrieval quality as the cause, since the key-value pairs were not retrieved."}],"correct":false,"tempting":[{"text":"Retrieval was already ruled out in the document version, where the answer document was placed by construction rather than found."}]},{"text":[{"text":"It demonstrates the effect on models with different tokenisers, generalising the result across families."}],"correct":false,"tempting":[{"text":"Confuses the choice of task with the choice of model; a second task says nothing about tokenisers."}]}],"order":[2,0,1,3],"why":null},"prev":{"slug":"measuring-retrieval-on-its-own","title":"Measuring retrieval without the model","minutes":9},"next":{"slug":"ordering-and-the-middle","title":"Ordering, and the fact that models skim","minutes":8}}}