{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"multi-hop-and-the-single-chunk-fallacy","title":"Questions no single passage answers","url":"https://addaly.com/learn/context-engineering/multi-hop-and-the-single-chunk-fallacy","position":28,"of":87,"minutes":8,"answer":"Top-k retrieval assumes the answer sits in one passage, so route aggregates to a database, comparisons to one query per side, and only genuine chains to an iterative loop with a hop cap.","keyIdea":"Top-k retrieval assumes the answer sits in one passage, so route aggregates to a database, comparisons to one query per side, and only genuine chains to an iterative loop with a hop cap, because each hop multiplies its error rate and permanently adds its wreckage to the window."},"body":{"format":"markdown","text":"## The assumption hiding in every retrieval pipeline\n\nTop-k retrieval assumes the answer is in a passage. Rank the passages, take the best few, and the answer is among them.\n\nA large class of real questions breaks that assumption outright:\n\n- \"Which of our enterprise customers have an open critical ticket?\": a join between two systems.\n- \"Has this clause changed since the 2023 version?\": a comparison between two documents.\n- \"What did we decide about the pricing model, and why did we reject the alternative?\": a chain across a decision document and the discussion that preceded it.\n- \"How many suppliers are affected?\": an aggregate.\n\nNo single chunk contains any of these answers. The retriever will still return five passages, the model will still produce an answer, and the answer will be built from whichever fragments looked closest. This is the failure mode that makes a demo look brilliant and a deployment look unreliable, because the questions people actually ask skew towards this shape far more than benchmark questions do.\n\n## Recognise the shape before you engineer for it\n\nThree signatures, each with a different correct response:\n\n**It is an aggregate or a join.** Counts, sums, \"how many\", \"which of my\", \"all the\". The answer lives in a database, not in prose. The previous lesson's second and third doors are the fix, and retrieval is the wrong tool entirely. Recognising this saves months.\n\n**It is a comparison between two known things.** Two versions, two products, two time periods. Do not hope one search finds both: retrieve for each side separately with its own query, and assemble both into the window with clear labels. Deterministic, cheap, reliable.\n\n**It is a chain**, where what you need to look up second depends on what you found first. \"Who approved the change that caused the outage on 14 August?\" You cannot write the second query until the first has answered.\n\nOnly the third needs iterative retrieval, and it is the least common of the three.\n\n## Iterative retrieval, and the cost of a hop\n\nThe loop: retrieve, let the model read and decide what it still needs, retrieve again, up to a cap.\n\nGive it a search tool, a small budget, and an explicit instruction to state what is missing before searching again. Cap the hops at two or three.\n\nThe cap is not arbitrary. Errors compound multiplicatively: if each hop finds the right thing 80% of the time, three hops succeed 51% of the time, and the failures are invisible because the final answer is fluent. Meanwhile every hop adds its results to the context permanently, so a four-hop investigation ends with a window full of material from three abandoned lines of enquiry.\n\nTwo mitigations. Summarise each hop's findings and drop the raw passages before the next hop, so the context grows slowly. And require the model to state, in the answer, which hop supplied which fact: attribution across hops is where the reasoning breaks and it is the only place you can see it.\n\n## Graphs\n\nBuilding a knowledge graph (extracting entities and relationships, then traversing them) solves some multi-hop problems that vector search cannot touch. It is also a large, ongoing project: extraction quality caps everything, entity resolution across sources is hard, and the graph goes stale exactly as fast as your documents change.\n\nA proportionate position: if your entities and relationships already exist in a database, use the database, and no graph construction is needed. If they exist only in prose and multi-hop questions are the core of your product, a graph may be worth the investment. If neither, decomposition plus a second retrieval will get you most of the value for a fraction of the work.\n\n## The cheapest big win\n\nBefore any of this, do the boring thing: **detect the shape and route.**\n\n```python\nif looks_like_aggregate(q):      return sql_path(q)\nif looks_like_comparison(q):     return multi_query_path(q)\nreturn standard_retrieval(q)\n```\n\nA short classifier prompt or a handful of patterns handles the routing. Most systems that feel unreliable are running every question through one path, and the fix is not a better retriever but three paths and a router.\n\n## Say when you cannot\n\nWhere a question needs a join you cannot perform, the correct behaviour is to say so plainly: \"I can look up individual orders, but I cannot count across accounts.\" That is a better product than a confident number that is wrong, and it is a specific, actionable feature request from your users the moment you start counting how often it fires."},"check":{"question":"An agent answers \"how many suppliers are affected?\" with a confident number by reading five retrieved passages. The number is wrong. What is the most fundamental problem?","options":[{"text":"The passages were retrieved but not reranked, so lower-quality evidence dominated.","correct":false,"tempting":"Better ranking gives better passages, but no ranking of passages produces a count over a set none of them enumerate."},{"text":"The question is an aggregate, so no set of passages contains the answer and retrieval was the wrong mechanism.","correct":true},{"text":"The agent needed more hops to gather every affected supplier before counting.","correct":false,"tempting":"More hops raise cost and compound error while still assembling an unverifiable partial set; the answer is a query, not a search."},{"text":"The passages lacked dates, so suppliers from different incidents were merged.","correct":false,"tempting":"Names a real freshness hazard that would corrupt the set further, but the count fails even with perfectly dated passages."}]},"prev":{"slug":"structured-sources-in-context","title":"Rows, records and schemas in the window","url":"https://addaly.com/learn/context-engineering/structured-sources-in-context"},"next":{"slug":"citations-that-survive-assembly","title":"Citations you can actually verify","url":"https://addaly.com/learn/context-engineering/citations-that-survive-assembly"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Questions no single passage answers', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/multi-hop-and-the-single-chunk-fallacy (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Questions no single passage answers","courseTitle":"Context Engineering","module":{"index":3,"slug":"retrieval-as-a-pipeline","title":"Retrieval as a context pipeline"},"answer":null,"keyIdea":"Top-k retrieval assumes the answer sits in one passage, so route aggregates to a database, comparisons to one query per side, and only genuine chains to an iterative loop with a hop cap, because each hop multiplies its error rate and permanently adds its wreckage to the window.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-assumption-hiding-in-every-retrieval-pipeline","runs":[{"text":"The assumption hiding in every retrieval pipeline"}]},{"t":"p","runs":[{"text":"Top-k retrieval assumes the answer is in a passage. Rank the passages, take the best few, and the answer is among them."}]},{"t":"p","runs":[{"text":"A large class of real questions breaks that assumption outright:"}]},{"t":"ul","items":[[{"text":"“Which of our enterprise customers have an open critical ticket?”: a join between two systems."}],[{"text":"“Has this clause changed since the 2023 version?”: a comparison between two documents."}],[{"text":"“What did we decide about the pricing model, and why did we reject the alternative?”: a chain across a decision document and the discussion that preceded it."}],[{"text":"“How many suppliers are affected?”: an aggregate."}]]},{"t":"p","runs":[{"text":"No single chunk contains any of these answers. The retriever will still return five passages, the model will still produce an answer, and the answer will be built from whichever fragments looked closest. This is the failure mode that makes a demo look brilliant and a deployment look unreliable, because the questions people actually ask skew towards this shape far more than benchmark questions do."}]},{"t":"h2","id":"recognise-the-shape-before-you-engineer-for-it","runs":[{"text":"Recognise the shape before you engineer for it"}]},{"t":"p","runs":[{"text":"Three signatures, each with a different correct response:"}]},{"t":"p","runs":[{"text":"It is an aggregate or a join.","b":true},{"text":" Counts, sums, “how many”, “which of my”, “all the”. The answer lives in a database, not in prose. The previous lesson’s second and third doors are the fix, and retrieval is the wrong tool entirely. Recognising this saves months."}]},{"t":"p","runs":[{"text":"It is a comparison between two known things.","b":true},{"text":" Two versions, two products, two time periods. Do not hope one search finds both: retrieve for each side separately with its own query, and assemble both into the window with clear labels. Deterministic, cheap, reliable."}]},{"t":"p","runs":[{"text":"It is a chain","b":true},{"text":", where what you need to look up second depends on what you found first. “Who approved the change that caused the outage on 14 August?” You cannot write the second query until the first has answered."}]},{"t":"p","runs":[{"text":"Only the third needs iterative retrieval, and it is the least common of the three."}]},{"t":"h2","id":"iterative-retrieval-and-the-cost-of-a-hop","runs":[{"text":"Iterative retrieval, and the cost of a hop"}]},{"t":"p","runs":[{"text":"The loop: retrieve, let the model read and decide what it still needs, retrieve again, up to a cap."}]},{"t":"p","runs":[{"text":"Give it a search tool, a small budget, and an explicit instruction to state what is missing before searching again. Cap the hops at two or three."}]},{"t":"p","runs":[{"text":"The cap is not arbitrary. Errors compound multiplicatively: if each hop finds the right thing 80% of the time, three hops succeed 51% of the time, and the failures are invisible because the final answer is fluent. Meanwhile every hop adds its results to the context permanently, so a four-hop investigation ends with a window full of material from three abandoned lines of enquiry."}]},{"t":"p","runs":[{"text":"Two mitigations. Summarise each hop’s findings and drop the raw passages before the next hop, so the context grows slowly. And require the model to state, in the answer, which hop supplied which fact: attribution across hops is where the reasoning breaks and it is the only place you can see it."}]},{"t":"h2","id":"graphs","runs":[{"text":"Graphs"}]},{"t":"p","runs":[{"text":"Building a knowledge graph (extracting entities and relationships, then traversing them) solves some multi-hop problems that vector search cannot touch. It is also a large, ongoing project: extraction quality caps everything, entity resolution across sources is hard, and the graph goes stale exactly as fast as your documents change."}]},{"t":"p","runs":[{"text":"A proportionate position: if your entities and relationships already exist in a database, use the database, and no graph construction is needed. If they exist only in prose and multi-hop questions are the core of your product, a graph may be worth the investment. If neither, decomposition plus a second retrieval will get you most of the value for a fraction of the work."}]},{"t":"h2","id":"the-cheapest-big-win","runs":[{"text":"The cheapest big win"}]},{"t":"p","runs":[{"text":"Before any of this, do the boring thing: "},{"text":"detect the shape and route.","b":true}]},{"t":"code","lang":"python","label":"Python","text":"if looks_like_aggregate(q):      return sql_path(q)\nif looks_like_comparison(q):     return multi_query_path(q)\nreturn standard_retrieval(q)"},{"t":"p","runs":[{"text":"A short classifier prompt or a handful of patterns handles the routing. Most systems that feel unreliable are running every question through one path, and the fix is not a better retriever but three paths and a router."}]},{"t":"h2","id":"say-when-you-cannot","runs":[{"text":"Say when you cannot"}]},{"t":"p","runs":[{"text":"Where a question needs a join you cannot perform, the correct behaviour is to say so plainly: “I can look up individual orders, but I cannot count across accounts.” That is a better product than a confident number that is wrong, and it is a specific, actionable feature request from your users the moment you start counting how often it fires."}]}],"check":{"question":[{"text":"An agent answers “how many suppliers are affected?” with a confident number by reading five retrieved passages. The number is wrong. What is the most fundamental problem?"}],"options":[{"text":[{"text":"The passages were retrieved but not reranked, so lower-quality evidence dominated."}],"correct":false,"tempting":[{"text":"Better ranking gives better passages, but no ranking of passages produces a count over a set none of them enumerate."}]},{"text":[{"text":"The question is an aggregate, so no set of passages contains the answer and retrieval was the wrong mechanism."}],"correct":true,"tempting":null},{"text":[{"text":"The agent needed more hops to gather every affected supplier before counting."}],"correct":false,"tempting":[{"text":"More hops raise cost and compound error while still assembling an unverifiable partial set; the answer is a query, not a search."}]},{"text":[{"text":"The passages lacked dates, so suppliers from different incidents were merged."}],"correct":false,"tempting":[{"text":"Names a real freshness hazard that would corrupt the set further, but the count fails even with perfectly dated passages."}]}],"order":[3,0,2,1],"why":null},"prev":{"slug":"structured-sources-in-context","title":"Rows, records and schemas in the window","minutes":9},"next":{"slug":"citations-that-survive-assembly","title":"Citations you can actually verify","minutes":8}}}