{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"retrieving-from-your-own-history","title":"Searching the transcript instead of carrying it","url":"https://addaly.com/learn/context-engineering/retrieving-from-your-own-history","position":54,"of":87,"minutes":8,"answer":"Indexing the transcript keeps context bounded however long a conversation runs, but conversational turns retrieve badly until they are segmented by topic and rewritten into standalone form at write time.","keyIdea":"Indexing the transcript keeps context bounded however long a conversation runs, but conversational turns retrieve badly until they are segmented by topic and rewritten into standalone form at write time, and retrieved history must supplement the verbatim recent turns rather than replace them."},"body":{"format":"markdown","text":"## Bounded context, unbounded conversation\n\nCompaction compresses history. The alternative is to stop carrying it at all: index the transcript, and retrieve only the parts relevant to the current message.\n\nThis inverts the cost curve. History stops growing in the window; a conversation of five hundred turns costs the same per call as one of twenty. It also crosses sessions naturally: a message from three weeks ago is as retrievable as one from three minutes ago.\n\nIt is also harder than it looks, for reasons that are specific to conversational text.\n\n## Why conversation retrieves badly\n\n**Turns are short and full of pronouns.** \"Yes, do that\" is four tokens and means nothing on its own. Embedding it produces a vector that matches nothing useful.\n\n**Meaning is distributed across turns.** A decision is often a user message and an assistant reply together; retrieving either half gives you a fragment.\n\n**Similarity across a transcript is uniformly high.** Everything in one conversation is about the same subject, so cosine similarity discriminates poorly: every turn looks somewhat relevant.\n\nThe fixes are the same ones from the retrieval module, applied to a different corpus.\n\n**Segment by topic, not by turn.** Group consecutive turns into episodes that share a subject, and index the episode. A simple heuristic (a gap in time, an explicit change of subject, a completed task) works better than a fixed turn count.\n\n**Augment each segment at write time.** The same contextual-augmentation trick: store a one-line description of what this exchange was about, in standalone language. \"The user asked whether the refund could go to a different card; agreed to keep the original method.\" That sentence retrieves; \"Yes, do that\" does not.\n\n**Resolve references before indexing.** Rewrite each user turn into a standalone form as you store it, using the same rewriting machinery as the query lesson. You are paying for the rewrite anyway on the way in.\n\n## Always keep the tail verbatim\n\nThe critical design rule: **retrieval supplements the recent window, it does not replace it.**\n\nA system that retrieves only semantically similar history and drops the last few turns will lose the thread of the immediate exchange, because the most recent turn is often not the most semantically similar to the current message. The user says \"actually, make it shorter\"; nothing in that retrieves the draft it refers to.\n\nSo the layout is additive:\n\n```\n[structured state]         always\n[retrieved episodes]       relevant older exchanges, chronologically ordered\n[last k turns]             always, verbatim\n[current message]\n```\n\n## Restore chronology, and label it\n\nRetrieved episodes come back ranked by relevance. Present them in time order, as the ordering lesson argued, because a conversation is a sequence and reading it out of order produces a wrong account of what was decided when.\n\nAnd label them as what they are:\n\n```\nEarlier in this conversation (17 August, turns 12-15):\n```\n\nWithout the label, the model treats a retrieved fragment from three weeks ago as part of the current exchange, and answers as though a resolved issue is live. With the date, it can at least reason about sequence.\n\n## When this is worth building\n\nBe honest about the threshold. For a product where conversations are under thirty turns, this machinery is not worth it: pinned head, compacted middle, verbatim tail is simpler and works.\n\nIt becomes worth it when conversations are long-running and continuous: an assistant somebody uses daily for a year, a support relationship spanning months, an agent working a task over weeks. There, the transcript is a corpus, and treating it as one is the right call.\n\n## What it costs to write\n\nIndexing a transcript is work on the write path: segmenting, rewriting each turn into standalone form, embedding, and inserting. That is a model call or two per exchange, which is a real cost on a chatty product and is paid whether or not anybody ever searches that conversation again.\n\nRun it asynchronously, after the reply has been sent, so it never sits in the user's latency path. And skip it for conversations under a handful of turns, which are the majority and which will never be retrieved from.\n\n## The evaluation is the same as any retrieval\n\nBuild a gold set of the form *given this message, which earlier exchange should be retrieved*, from real conversations where the model failed for lack of history. Measure recall@k on it. Every technique from the retrieval module applies unchanged: hybrid search matters here too, because users refer back to exact strings, names and numbers far more often than they paraphrase.\n\nThe common finding: keyword search over the transcript outperforms embeddings for the \"what did we say about X\" query, which is most of the traffic. Run both."},"check":{"question":"A system replaces its recent-turn window entirely with semantic retrieval over the transcript. Users report it loses the thread on follow-ups like \"make it shorter\". Why?","options":[{"text":"Short messages embed poorly, so the follow-up itself fails to produce a useful query.","correct":false,"tempting":"True of the query and worth fixing, but even a perfect query for \"make it shorter\" would not surface the draft by similarity."},{"text":"The retrieved episodes were returned in relevance order rather than chronological order.","correct":false,"tempting":"Ordering corrupts sequence reasoning, but the failure described is a missing turn rather than a misordered one."},{"text":"The immediately preceding turn is often not the most semantically similar one, so it is not retrieved.","correct":true},{"text":"Transcript segments were indexed without contextual augmentation, so none of them retrieve well.","correct":false,"tempting":"Augmentation improves recall generally, but the specific failure is about recency, which no augmentation supplies."}]},"prev":{"slug":"memory-across-sessions","title":"Memory between conversations","url":"https://addaly.com/learn/context-engineering/memory-across-sessions"},"next":{"slug":"the-handover-note","title":"Passing context to whoever comes next","url":"https://addaly.com/learn/context-engineering/the-handover-note"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Searching the transcript instead of carrying it', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/retrieving-from-your-own-history (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Searching the transcript instead of carrying it","courseTitle":"Context Engineering","module":{"index":6,"slug":"conversation-and-memory","title":"Conversation, memory and compaction"},"answer":null,"keyIdea":"Indexing the transcript keeps context bounded however long a conversation runs, but conversational turns retrieve badly until they are segmented by topic and rewritten into standalone form at write time, and retrieved history must supplement the verbatim recent turns rather than replace them.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"bounded-context-unbounded-conversation","runs":[{"text":"Bounded context, unbounded conversation"}]},{"t":"p","runs":[{"text":"Compaction compresses history. The alternative is to stop carrying it at all: index the transcript, and retrieve only the parts relevant to the current message."}]},{"t":"p","runs":[{"text":"This inverts the cost curve. History stops growing in the window; a conversation of five hundred turns costs the same per call as one of twenty. It also crosses sessions naturally: a message from three weeks ago is as retrievable as one from three minutes ago."}]},{"t":"p","runs":[{"text":"It is also harder than it looks, for reasons that are specific to conversational text."}]},{"t":"h2","id":"why-conversation-retrieves-badly","runs":[{"text":"Why conversation retrieves badly"}]},{"t":"p","runs":[{"text":"Turns are short and full of pronouns.","b":true},{"text":" “Yes, do that” is four tokens and means nothing on its own. Embedding it produces a vector that matches nothing useful."}]},{"t":"p","runs":[{"text":"Meaning is distributed across turns.","b":true},{"text":" A decision is often a user message and an assistant reply together; retrieving either half gives you a fragment."}]},{"t":"p","runs":[{"text":"Similarity across a transcript is uniformly high.","b":true},{"text":" Everything in one conversation is about the same subject, so cosine similarity discriminates poorly: every turn looks somewhat relevant."}]},{"t":"p","runs":[{"text":"The fixes are the same ones from the retrieval module, applied to a different corpus."}]},{"t":"p","runs":[{"text":"Segment by topic, not by turn.","b":true},{"text":" Group consecutive turns into episodes that share a subject, and index the episode. A simple heuristic (a gap in time, an explicit change of subject, a completed task) works better than a fixed turn count."}]},{"t":"p","runs":[{"text":"Augment each segment at write time.","b":true},{"text":" The same contextual-augmentation trick: store a one-line description of what this exchange was about, in standalone language. “The user asked whether the refund could go to a different card; agreed to keep the original method.” That sentence retrieves; “Yes, do that” does not."}]},{"t":"p","runs":[{"text":"Resolve references before indexing.","b":true},{"text":" Rewrite each user turn into a standalone form as you store it, using the same rewriting machinery as the query lesson. You are paying for the rewrite anyway on the way in."}]},{"t":"h2","id":"always-keep-the-tail-verbatim","runs":[{"text":"Always keep the tail verbatim"}]},{"t":"p","runs":[{"text":"The critical design rule: "},{"text":"retrieval supplements the recent window, it does not replace it.","b":true}]},{"t":"p","runs":[{"text":"A system that retrieves only semantically similar history and drops the last few turns will lose the thread of the immediate exchange, because the most recent turn is often not the most semantically similar to the current message. The user says “actually, make it shorter”; nothing in that retrieves the draft it refers to."}]},{"t":"p","runs":[{"text":"So the layout is additive:"}]},{"t":"code","lang":"","label":"Text","text":"[structured state]         always\n[retrieved episodes]       relevant older exchanges, chronologically ordered\n[last k turns]             always, verbatim\n[current message]"},{"t":"h2","id":"restore-chronology-and-label-it","runs":[{"text":"Restore chronology, and label it"}]},{"t":"p","runs":[{"text":"Retrieved episodes come back ranked by relevance. Present them in time order, as the ordering lesson argued, because a conversation is a sequence and reading it out of order produces a wrong account of what was decided when."}]},{"t":"p","runs":[{"text":"And label them as what they are:"}]},{"t":"code","lang":"","label":"Text","text":"Earlier in this conversation (17 August, turns 12-15):"},{"t":"p","runs":[{"text":"Without the label, the model treats a retrieved fragment from three weeks ago as part of the current exchange, and answers as though a resolved issue is live. With the date, it can at least reason about sequence."}]},{"t":"h2","id":"when-this-is-worth-building","runs":[{"text":"When this is worth building"}]},{"t":"p","runs":[{"text":"Be honest about the threshold. For a product where conversations are under thirty turns, this machinery is not worth it: pinned head, compacted middle, verbatim tail is simpler and works."}]},{"t":"p","runs":[{"text":"It becomes worth it when conversations are long-running and continuous: an assistant somebody uses daily for a year, a support relationship spanning months, an agent working a task over weeks. There, the transcript is a corpus, and treating it as one is the right call."}]},{"t":"h2","id":"what-it-costs-to-write","runs":[{"text":"What it costs to write"}]},{"t":"p","runs":[{"text":"Indexing a transcript is work on the write path: segmenting, rewriting each turn into standalone form, embedding, and inserting. That is a model call or two per exchange, which is a real cost on a chatty product and is paid whether or not anybody ever searches that conversation again."}]},{"t":"p","runs":[{"text":"Run it asynchronously, after the reply has been sent, so it never sits in the user’s latency path. And skip it for conversations under a handful of turns, which are the majority and which will never be retrieved from."}]},{"t":"h2","id":"the-evaluation-is-the-same-as-any-retrieval","runs":[{"text":"The evaluation is the same as any retrieval"}]},{"t":"p","runs":[{"text":"Build a gold set of the form "},{"text":"given this message, which earlier exchange should be retrieved","i":true},{"text":", from real conversations where the model failed for lack of history. Measure recall@k on it. Every technique from the retrieval module applies unchanged: hybrid search matters here too, because users refer back to exact strings, names and numbers far more often than they paraphrase."}]},{"t":"p","runs":[{"text":"The common finding: keyword search over the transcript outperforms embeddings for the “what did we say about X” query, which is most of the traffic. Run both."}]}],"check":{"question":[{"text":"A system replaces its recent-turn window entirely with semantic retrieval over the transcript. Users report it loses the thread on follow-ups like “make it shorter”. Why?"}],"options":[{"text":[{"text":"Short messages embed poorly, so the follow-up itself fails to produce a useful query."}],"correct":false,"tempting":[{"text":"True of the query and worth fixing, but even a perfect query for “make it shorter” would not surface the draft by similarity."}]},{"text":[{"text":"The retrieved episodes were returned in relevance order rather than chronological order."}],"correct":false,"tempting":[{"text":"Ordering corrupts sequence reasoning, but the failure described is a missing turn rather than a misordered one."}]},{"text":[{"text":"The immediately preceding turn is often not the most semantically similar one, so it is not retrieved."}],"correct":true,"tempting":null},{"text":[{"text":"Transcript segments were indexed without contextual augmentation, so none of them retrieve well."}],"correct":false,"tempting":[{"text":"Augmentation improves recall generally, but the specific failure is about recency, which no augmentation supplies."}]}],"order":[3,0,1,2],"why":null},"prev":{"slug":"memory-across-sessions","title":"Memory between conversations","minutes":9},"next":{"slug":"the-handover-note","title":"Passing context to whoever comes next","minutes":8}}}