{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"the-k-curve","title":"How many passages is the right number","url":"https://addaly.com/learn/context-engineering/the-k-curve","position":14,"of":87,"minutes":9,"answer":"Sweep k and measure accuracy, retrieval recall and tokens together: answer quality peaks and then falls because passages ranked just below the answer are on-topic distractors.","keyIdea":"Sweep k and measure accuracy, retrieval recall and tokens together: answer quality peaks and then falls because passages ranked just below the answer are on-topic distractors, and the flat left edge of the curve is cheaper than the peak."},"body":{"format":"markdown","text":"## The default nobody chose\n\nAlmost every retrieval-augmented system in the world uses `k=5` or `k=10`, and almost nobody measured it. The number came from a tutorial. It is sometimes right and it is never right for a reason.\n\nThree quantities move as k grows, and they do not move together.\n\n**Recall rises, with diminishing returns.** The chance that the necessary passage is somewhere in the set goes up steeply from k=1 to k=5, then flattens. Going from 10 to 40 typically adds a couple of percentage points.\n\n**Tokens and cost rise linearly.** Twenty passages of 400 tokens is 8,000 tokens, on every request, forever. At k=40 you are paying four times the retrieval budget of k=10 for those last few points of recall.\n\n**Answer accuracy rises, peaks, and falls.** This is the part people do not expect. Past some point, adding more passages makes answers worse, not merely more expensive.\n\n## Why more evidence can hurt\n\nTwo mechanisms, both worth being able to name.\n\n**Distraction.** Passages ranked 15 to 30 are, by construction, on-topic and not the answer. They are the most dangerous kind of wrong material, because they look exactly like what the model is looking for. Probability mass moves towards them. The model produces an answer that is a blend of the correct passage and three plausible neighbours.\n\n**Dilution and position.** More material means each passage holds a smaller share of attention, and a longer context puts the true passage further from the question. The position lesson later in the course covers the shape; the effect here is that the same correct passage is less likely to be used at k=40 than at k=8.\n\nThe combined result is a curve with a peak. Where the peak sits depends on your chunk size, your reranker, your model and your task, which is why the number cannot be borrowed.\n\n> Figure: Recall keeps rising after accuracy has peaked. Line chart of per cent against passages retrieved, k: Answer accuracy (1, 54) (3, 68) (5, 74) (8, 76) (12, 73) (20, 67) (40, 59); Retrieval recall at k (1, 52) (3, 74) (5, 83) (8, 88) (12, 91) (20, 93) (40, 95). One corpus, one model, one hundred questions. Recall flattens; answer accuracy peaks near eight passages and then falls, because passages ranked just below the answer are on-topic and wrong, which is the most effective kind of distractor. Tokens rise straight through: k of forty costs four times the retrieval budget of k of ten. Choose where accuracy has flattened, not where it peaks.\n\n## Measure it in an afternoon\n\n```python\nfor k in [1, 3, 5, 8, 12, 20, 40]:\n    acc, toks = [], []\n    for q, gold in eval_set:\n        ctx = retrieve(q, k=k)\n        toks.append(count_tokens(ctx))\n        acc.append(judge(answer(q, ctx), gold))\n    print(k, round(mean(acc), 3), int(mean(toks)))\n```\n\nA hundred questions with known answers is enough to see the shape. Plot accuracy against k and tokens against k on the same axis, and choose the point where accuracy has flattened rather than the point where it peaks: the peak is usually inside the noise, and the flat region's left edge is cheaper and just as good.\n\nTeams running this for the first time commonly find the answer is smaller than they were using. Going from k=10 to k=5 with a reranker in front is one of the more reliable ways to cut a bill without losing quality.\n\n## Two failures the sweep separates\n\nWhile you have the harness up, record a second number: whether the gold passage was in the retrieved set at all.\n\nThat splits every failure into two kinds.\n\n- **Not retrieved.** Fix retrieval: hybrid search, query rewriting, chunking, a better embedding model.\n- **Retrieved and not used.** Fix assembly: position, ordering, delimiters, instructions, or the fact that the passage is buried in 20 others.\n\nThe ratio between them tells you which module of this course to work in next. A system with 90% retrieval recall and 60% answer accuracy has an assembly problem, and no amount of retriever tuning will move it.\n\n## k is not one number\n\nA fixed k treats every question the same. Two refinements, both cheap:\n\n**Cut on the score gap.** Retrieve 30, then keep only those within some distance of the top score. A question with one obvious answer takes three passages; a broad question takes fifteen. This adapts the budget to the query rather than to the average query.\n\n**Cut on token budget rather than count.** Chunks vary in size. Ten chunks might be 2,000 tokens or 9,000. Fill to a token budget, which is what you actually care about, and let the count fall where it falls. The packing lesson later in this module has the code.\n\n## The limitation\n\nAll of this assumes your evaluation set is representative. A k sweep on 100 head queries will pick a k that is right for head queries and possibly wrong for the tail, where recall matters more because the relevant material is rarer. If your traffic has a long tail, and most does, sweep separately for the tail, or accept that you have optimised for the common case and say so."},"check":{"question":"A team raises k from 10 to 30. Retrieval recall improves from 88% to 94%, but end-to-end answer accuracy drops from 76% to 71%. What is the most likely mechanism?","options":[{"text":"The extra passages exceeded the model's effective context, so the additional evidence was ignored entirely.","correct":false,"tempting":"Would predict unchanged accuracy rather than a fall; ignored material does not make previously correct answers wrong."},{"text":"The reranker was trained for k=10 and produces invalid scores beyond that depth.","correct":false,"tempting":"Invents a depth limit rerankers do not have; they score each pair independently of how many pairs there are."},{"text":"Recall and accuracy measure different things, so the comparison is invalid and one of the two numbers must be wrong.","correct":false,"tempting":"Declines to explain a real and reproducible effect; the two numbers are measuring different stages and can legitimately move in opposite directions."},{"text":"Passages ranked 11 to 30 are on-topic but not the answer, so they compete with the correct passage and pull the answer towards them.","correct":true}]},"prev":{"slug":"filter-before-you-rank","title":"Filter before you rank, not after","url":"https://addaly.com/learn/context-engineering/filter-before-you-rank"},"next":{"slug":"near-duplicates-and-boilerplate","title":"The same passage three times","url":"https://addaly.com/learn/context-engineering/near-duplicates-and-boilerplate"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'How many passages is the right number', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/the-k-curve (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"How many passages is the right number","courseTitle":"Context Engineering","module":{"index":2,"slug":"what-earns-a-place","title":"Selection: what earns a place"},"answer":null,"keyIdea":"Sweep k and measure accuracy, retrieval recall and tokens together: answer quality peaks and then falls because passages ranked just below the answer are on-topic distractors, and the flat left edge of the curve is cheaper than the peak.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"line","title":"Recall keeps rising after accuracy has peaked","series":[{"label":"Answer accuracy","points":[[1,54],[3,68],[5,74],[8,76],[12,73],[20,67],[40,59]]},{"label":"Retrieval recall at k","points":[[1,52],[3,74],[5,83],[8,88],[12,91],[20,93],[40,95]]}],"xLabel":"passages retrieved, k","yLabel":"per cent","caption":"One corpus, one model, one hundred questions. Recall flattens; answer accuracy peaks near eight passages and then falls, because passages ranked just below the answer are on-topic and wrong, which is the most effective kind of distractor. Tokens rise straight through: k of forty costs four times the retrieval budget of k of ten. Choose where accuracy has flattened, not where it peaks."},"alt":"Figure: Recall keeps rising after accuracy has peaked. Line chart of per cent against passages retrieved, k: Answer accuracy (1, 54) (3, 68) (5, 74) (8, 76) (12, 73) (20, 67) (40, 59); Retrieval recall at k (1, 52) (3, 74) (5, 83) (8, 88) (12, 91) (20, 93) (40, 95). One corpus, one model, one hundred questions. Recall flattens; answer accuracy peaks near eight passages and then falls, because passages ranked just below the answer are on-topic and wrong, which is the most effective kind of distractor. Tokens rise straight through: k of forty costs four times the retrieval budget of k of ten. Choose where accuracy has flattened, not where it peaks."}},"blocks":[{"t":"h2","id":"the-default-nobody-chose","runs":[{"text":"The default nobody chose"}]},{"t":"p","runs":[{"text":"Almost every retrieval-augmented system in the world uses "},{"text":"k=5","code":true},{"text":" or "},{"text":"k=10","code":true},{"text":", and almost nobody measured it. The number came from a tutorial. It is sometimes right and it is never right for a reason."}]},{"t":"p","runs":[{"text":"Three quantities move as k grows, and they do not move together."}]},{"t":"p","runs":[{"text":"Recall rises, with diminishing returns.","b":true},{"text":" The chance that the necessary passage is somewhere in the set goes up steeply from k=1 to k=5, then flattens. Going from 10 to 40 typically adds a couple of percentage points."}]},{"t":"p","runs":[{"text":"Tokens and cost rise linearly.","b":true},{"text":" Twenty passages of 400 tokens is 8,000 tokens, on every request, forever. At k=40 you are paying four times the retrieval budget of k=10 for those last few points of recall."}]},{"t":"p","runs":[{"text":"Answer accuracy rises, peaks, and falls.","b":true},{"text":" This is the part people do not expect. Past some point, adding more passages makes answers worse, not merely more expensive."}]},{"t":"h2","id":"why-more-evidence-can-hurt","runs":[{"text":"Why more evidence can hurt"}]},{"t":"p","runs":[{"text":"Two mechanisms, both worth being able to name."}]},{"t":"p","runs":[{"text":"Distraction.","b":true},{"text":" Passages ranked 15 to 30 are, by construction, on-topic and not the answer. They are the most dangerous kind of wrong material, because they look exactly like what the model is looking for. Probability mass moves towards them. The model produces an answer that is a blend of the correct passage and three plausible neighbours."}]},{"t":"p","runs":[{"text":"Dilution and position.","b":true},{"text":" More material means each passage holds a smaller share of attention, and a longer context puts the true passage further from the question. The position lesson later in the course covers the shape; the effect here is that the same correct passage is less likely to be used at k=40 than at k=8."}]},{"t":"p","runs":[{"text":"The combined result is a curve with a peak. Where the peak sits depends on your chunk size, your reranker, your model and your task, which is why the number cannot be borrowed."}]},{"t":"h2","id":"measure-it-in-an-afternoon","runs":[{"text":"Measure it in an afternoon"}]},{"t":"code","lang":"python","label":"Python","text":"for k in [1, 3, 5, 8, 12, 20, 40]:\n    acc, toks = [], []\n    for q, gold in eval_set:\n        ctx = retrieve(q, k=k)\n        toks.append(count_tokens(ctx))\n        acc.append(judge(answer(q, ctx), gold))\n    print(k, round(mean(acc), 3), int(mean(toks)))"},{"t":"p","runs":[{"text":"A hundred questions with known answers is enough to see the shape. Plot accuracy against k and tokens against k on the same axis, and choose the point where accuracy has flattened rather than the point where it peaks: the peak is usually inside the noise, and the flat region’s left edge is cheaper and just as good."}]},{"t":"p","runs":[{"text":"Teams running this for the first time commonly find the answer is smaller than they were using. Going from k=10 to k=5 with a reranker in front is one of the more reliable ways to cut a bill without losing quality."}]},{"t":"h2","id":"two-failures-the-sweep-separates","runs":[{"text":"Two failures the sweep separates"}]},{"t":"p","runs":[{"text":"While you have the harness up, record a second number: whether the gold passage was in the retrieved set at all."}]},{"t":"p","runs":[{"text":"That splits every failure into two kinds."}]},{"t":"ul","items":[[{"text":"Not retrieved.","b":true},{"text":" Fix retrieval: hybrid search, query rewriting, chunking, a better embedding model."}],[{"text":"Retrieved and not used.","b":true},{"text":" Fix assembly: position, ordering, delimiters, instructions, or the fact that the passage is buried in 20 others."}]]},{"t":"p","runs":[{"text":"The ratio between them tells you which module of this course to work in next. A system with 90% retrieval recall and 60% answer accuracy has an assembly problem, and no amount of retriever tuning will move it."}]},{"t":"h2","id":"k-is-not-one-number","runs":[{"text":"k is not one number"}]},{"t":"p","runs":[{"text":"A fixed k treats every question the same. Two refinements, both cheap:"}]},{"t":"p","runs":[{"text":"Cut on the score gap.","b":true},{"text":" Retrieve 30, then keep only those within some distance of the top score. A question with one obvious answer takes three passages; a broad question takes fifteen. This adapts the budget to the query rather than to the average query."}]},{"t":"p","runs":[{"text":"Cut on token budget rather than count.","b":true},{"text":" Chunks vary in size. Ten chunks might be 2,000 tokens or 9,000. Fill to a token budget, which is what you actually care about, and let the count fall where it falls. The packing lesson later in this module has the code."}]},{"t":"h2","id":"the-limitation","runs":[{"text":"The limitation"}],"role":"limit"},{"t":"p","runs":[{"text":"All of this assumes your evaluation set is representative. A k sweep on 100 head queries will pick a k that is right for head queries and possibly wrong for the tail, where recall matters more because the relevant material is rarer. If your traffic has a long tail, and most does, sweep separately for the tail, or accept that you have optimised for the common case and say so."}]}],"check":{"question":[{"text":"A team raises k from 10 to 30. Retrieval recall improves from 88% to 94%, but end-to-end answer accuracy drops from 76% to 71%. What is the most likely mechanism?"}],"options":[{"text":[{"text":"The extra passages exceeded the model’s effective context, so the additional evidence was ignored entirely."}],"correct":false,"tempting":[{"text":"Would predict unchanged accuracy rather than a fall; ignored material does not make previously correct answers wrong."}]},{"text":[{"text":"The reranker was trained for k=10 and produces invalid scores beyond that depth."}],"correct":false,"tempting":[{"text":"Invents a depth limit rerankers do not have; they score each pair independently of how many pairs there are."}]},{"text":[{"text":"Recall and accuracy measure different things, so the comparison is invalid and one of the two numbers must be wrong."}],"correct":false,"tempting":[{"text":"Declines to explain a real and reproducible effect; the two numbers are measuring different stages and can legitimately move in opposite directions."}]},{"text":[{"text":"Passages ranked 11 to 30 are on-topic but not the answer, so they compete with the correct passage and pull the answer towards them."}],"correct":true,"tempting":null}],"order":[0,3,1,2],"why":null},"prev":{"slug":"filter-before-you-rank","title":"Filter before you rank, not after","minutes":8},"next":{"slug":"near-duplicates-and-boilerplate","title":"The same passage three times","minutes":8}}}