{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"logging-a-request-you-can-replay","title":"Logging enough to rebuild the request","url":"https://addaly.com/learn/context-engineering/logging-a-request-you-can-replay","position":84,"of":87,"minutes":9,"answer":"Log the inputs, the configuration version and the prefix hash rather than the rendered prompt, because a pure assembler can reconstruct the exact request from those three while storing the prompt itself is expensive.","keyIdea":"Log the inputs, the configuration version and the prefix hash rather than the rendered prompt, because a pure assembler can reconstruct the exact request from those three while storing the prompt itself is expensive, full of personal data, and still not enough to explain a bill."},"body":{"format":"markdown","text":"## The question you will be asked\n\nA user complains about an answer from Tuesday. Someone asks what happened.\n\nAnswering requires knowing what was in the window. Not approximately: exactly. Which passages, in which order, under which prompt version, with which settings. Without that, every diagnosis is a guess, and the usual outcome is that somebody tries the question again today, gets a reasonable answer, and the ticket is closed with nothing learned.\n\n## Log the inputs, not the output of the assembler\n\nThe instinct is to store the rendered prompt. Resist it. A rendered prompt is tens of kilobytes per request, it is dominated by material you already have in your corpus, and it is a dense concentration of personal data in a log store built for operational text.\n\nIf the assembler is a pure function of its inputs (the design from the first module, now paying for itself again) then logging the inputs reconstructs the request exactly:\n\n```json\n{\n  \"request_id\": \"r_8812\",\n  \"config_version\": \"ctx-v37\",\n  \"model\": \"as returned by the API, not as requested\",\n  \"prefix_sha\": \"9c1f...\",\n  \"query_raw\": \"<redacted or referenced>\",\n  \"query_rewritten\": \"...\",\n  \"retrieved\": [{\"id\": \"doc14#3\", \"score\": 0.81, \"rank\": 1, \"trust\": \"untrusted\"}],\n  \"packed\": [\"doc14#3\", \"doc02#7\"],\n  \"tokens\": {\"input\": 6120, \"cached_read\": 5400, \"cache_write\": 0, \"output\": 380},\n  \"latency_ms\": {\"ttft\": 640, \"total\": 3120},\n  \"tools_called\": [\"search_orders\"],\n  \"finish_reason\": \"end_turn\",\n  \"has_untrusted\": true\n}\n```\n\nA few hundred bytes. Everything needed to rebuild the request, plus the operational numbers you need for everything else.\n\n## The fields that earn their place\n\n**`config_version`.** Without it you cannot attribute anything. Every response should also carry it back to the client, so a screenshot in a bug report identifies the version that produced it.\n\n**`prefix_sha`.** A hash of everything above the cache breakpoint. When cache hit rate drops, this tells you instantly whether the prefix changed and when, which is otherwise a long and frustrating investigation.\n\n**`retrieved` versus `packed`.** Two different lists. What the retriever returned and what actually made it into the window after dedup and budget. The gap between them is where a surprising number of \"the model ignored the document\" reports turn out to live: the document was retrieved, ranked fourth, and dropped at packing.\n\n**`model` as returned.** Not the string you sent. If you requested an alias, the response tells you what you actually got, and that is the value that matters six weeks later.\n\n**The four token counts.** Input, cached read, cache write, output. Anything less and you cannot reconstruct the bill or the cache hit rate.\n\n**`has_untrusted` plus tools called.** The security module's alerting signature, available only if you log both.\n\n## Personal data needs its own handling\n\nThe user's text is the one field that is sensitive and necessary for debugging. Treat it differently from the rest of the record.\n\nPut it in a separate store with a short retention period (thirty or ninety days) while the operational record, which is not personal, can live much longer. Reference it by id from the main log. That way a retention sweep or an erasure request touches one store, and your ability to compute last year's cache hit rate is unaffected.\n\nThis connects directly to the deletion lesson in the conversation module: your log store is one of the ten places user text ends up, and it is the one most often forgotten in an erasure implementation. The legal requirements differ by country and remain unsettled in several; the engineering requirement (knowing every store and being able to delete from each) does not.\n\n## Sampling, done properly\n\nAt volume you cannot keep everything. Sample, but not uniformly, because uniform sampling systematically discards the interesting requests.\n\nKeep every error, every request over a latency threshold, every request where a write tool fired, every request with untrusted content and an action, and a small random sample of the rest. This is tail-based sampling and it keeps the requests worth having while dropping the overwhelming majority of the boring ones.\n\n## Replay is the point\n\nThe payoff is a command that takes a request id and rebuilds the exact request:\n\n```bash\nreplay r_8812 --config ctx-v37        # what they got\nreplay r_8812 --config ctx-v38        # what they would get now\n```\n\nThe first tells you whether the problem reproduces at all. The second tells you whether you have already fixed it. Both take seconds, and between them they resolve most incident tickets before anybody has to form a theory.\n\nOne caveat to keep in view: replay reconstructs the request faithfully, and it cannot reconstruct the model. If the provider has moved the version behind your alias, or changed anything about serving, the same request may produce a different answer today. That is not a flaw in the logging; it is the subject of the next lesson."},"check":{"question":"Why log the assembler's inputs and a prefix hash rather than the rendered prompt?","options":[{"text":"Rendered prompts cannot be stored, since providers treat the final serialisation as proprietary.","correct":false,"tempting":"You render it yourself and may store it freely; the reasons not to are cost and data handling, not permission."},{"text":"A pure assembler rebuilds the exact request from them, at a fraction of the size and sensitivity.","correct":true},{"text":"Hashes compress better than text, so a hash of the prefix preserves the same information more efficiently.","correct":false,"tempting":"A hash is one-way and preserves nothing of the content; it identifies a prefix rather than storing it."},{"text":"Rendered prompts change between library versions, so a stored copy would not match what runs today.","correct":false,"tempting":"True of templates and an argument for logging inputs, but a stored render is still an accurate record of that day, which is what a log is for."}]},"prev":{"slug":"judging-with-a-model","title":"Using a model to mark the homework","url":"https://addaly.com/learn/context-engineering/judging-with-a-model"},"next":{"slug":"shipping-a-context-change","title":"Rolling out a prompt like a deployment","url":"https://addaly.com/learn/context-engineering/shipping-a-context-change"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Logging enough to rebuild the request', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/logging-a-request-you-can-replay (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Logging enough to rebuild the request","courseTitle":"Context Engineering","module":{"index":9,"slug":"shipping-and-measuring","title":"Shipping context, and proving it got better"},"answer":null,"keyIdea":"Log the inputs, the configuration version and the prefix hash rather than the rendered prompt, because a pure assembler can reconstruct the exact request from those three while storing the prompt itself is expensive, full of personal data, and still not enough to explain a bill.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-question-you-will-be-asked","runs":[{"text":"The question you will be asked"}]},{"t":"p","runs":[{"text":"A user complains about an answer from Tuesday. Someone asks what happened."}]},{"t":"p","runs":[{"text":"Answering requires knowing what was in the window. Not approximately: exactly. Which passages, in which order, under which prompt version, with which settings. Without that, every diagnosis is a guess, and the usual outcome is that somebody tries the question again today, gets a reasonable answer, and the ticket is closed with nothing learned."}]},{"t":"h2","id":"log-the-inputs-not-the-output-of-the-assembler","runs":[{"text":"Log the inputs, not the output of the assembler"}]},{"t":"p","runs":[{"text":"The instinct is to store the rendered prompt. Resist it. A rendered prompt is tens of kilobytes per request, it is dominated by material you already have in your corpus, and it is a dense concentration of personal data in a log store built for operational text."}]},{"t":"p","runs":[{"text":"If the assembler is a pure function of its inputs (the design from the first module, now paying for itself again) then logging the inputs reconstructs the request exactly:"}]},{"t":"code","lang":"json","label":"JSON","text":"{\n  \"request_id\": \"r_8812\",\n  \"config_version\": \"ctx-v37\",\n  \"model\": \"as returned by the API, not as requested\",\n  \"prefix_sha\": \"9c1f...\",\n  \"query_raw\": \"<redacted or referenced>\",\n  \"query_rewritten\": \"...\",\n  \"retrieved\": [{\"id\": \"doc14#3\", \"score\": 0.81, \"rank\": 1, \"trust\": \"untrusted\"}],\n  \"packed\": [\"doc14#3\", \"doc02#7\"],\n  \"tokens\": {\"input\": 6120, \"cached_read\": 5400, \"cache_write\": 0, \"output\": 380},\n  \"latency_ms\": {\"ttft\": 640, \"total\": 3120},\n  \"tools_called\": [\"search_orders\"],\n  \"finish_reason\": \"end_turn\",\n  \"has_untrusted\": true\n}"},{"t":"p","runs":[{"text":"A few hundred bytes. Everything needed to rebuild the request, plus the operational numbers you need for everything else."}]},{"t":"h2","id":"the-fields-that-earn-their-place","runs":[{"text":"The fields that earn their place"}]},{"t":"p","runs":[{"text":"config_version","b":true,"code":true},{"text":".","b":true},{"text":" Without it you cannot attribute anything. Every response should also carry it back to the client, so a screenshot in a bug report identifies the version that produced it."}]},{"t":"p","runs":[{"text":"prefix_sha","b":true,"code":true},{"text":".","b":true},{"text":" A hash of everything above the cache breakpoint. When cache hit rate drops, this tells you instantly whether the prefix changed and when, which is otherwise a long and frustrating investigation."}]},{"t":"p","runs":[{"text":"retrieved","b":true,"code":true},{"text":" versus ","b":true},{"text":"packed","b":true,"code":true},{"text":".","b":true},{"text":" Two different lists. What the retriever returned and what actually made it into the window after dedup and budget. The gap between them is where a surprising number of “the model ignored the document” reports turn out to live: the document was retrieved, ranked fourth, and dropped at packing."}]},{"t":"p","runs":[{"text":"model","b":true,"code":true},{"text":" as returned.","b":true},{"text":" Not the string you sent. If you requested an alias, the response tells you what you actually got, and that is the value that matters six weeks later."}]},{"t":"p","runs":[{"text":"The four token counts.","b":true},{"text":" Input, cached read, cache write, output. Anything less and you cannot reconstruct the bill or the cache hit rate."}]},{"t":"p","runs":[{"text":"has_untrusted","b":true,"code":true},{"text":" plus tools called.","b":true},{"text":" The security module’s alerting signature, available only if you log both."}]},{"t":"h2","id":"personal-data-needs-its-own-handling","runs":[{"text":"Personal data needs its own handling"}]},{"t":"p","runs":[{"text":"The user’s text is the one field that is sensitive and necessary for debugging. Treat it differently from the rest of the record."}]},{"t":"p","runs":[{"text":"Put it in a separate store with a short retention period (thirty or ninety days) while the operational record, which is not personal, can live much longer. Reference it by id from the main log. That way a retention sweep or an erasure request touches one store, and your ability to compute last year’s cache hit rate is unaffected."}]},{"t":"p","runs":[{"text":"This connects directly to the deletion lesson in the conversation module: your log store is one of the ten places user text ends up, and it is the one most often forgotten in an erasure implementation. The legal requirements differ by country and remain unsettled in several; the engineering requirement (knowing every store and being able to delete from each) does not."}]},{"t":"h2","id":"sampling-done-properly","runs":[{"text":"Sampling, done properly"}]},{"t":"p","runs":[{"text":"At volume you cannot keep everything. Sample, but not uniformly, because uniform sampling systematically discards the interesting requests."}]},{"t":"p","runs":[{"text":"Keep every error, every request over a latency threshold, every request where a write tool fired, every request with untrusted content and an action, and a small random sample of the rest. This is tail-based sampling and it keeps the requests worth having while dropping the overwhelming majority of the boring ones."}]},{"t":"h2","id":"replay-is-the-point","runs":[{"text":"Replay is the point"}]},{"t":"p","runs":[{"text":"The payoff is a command that takes a request id and rebuilds the exact request:"}]},{"t":"code","lang":"bash","label":"Bash","text":"replay r_8812 --config ctx-v37        # what they got\nreplay r_8812 --config ctx-v38        # what they would get now"},{"t":"p","runs":[{"text":"The first tells you whether the problem reproduces at all. The second tells you whether you have already fixed it. Both take seconds, and between them they resolve most incident tickets before anybody has to form a theory."}]},{"t":"p","runs":[{"text":"One caveat to keep in view: replay reconstructs the request faithfully, and it cannot reconstruct the model. If the provider has moved the version behind your alias, or changed anything about serving, the same request may produce a different answer today. That is not a flaw in the logging; it is the subject of the next lesson."}]}],"check":{"question":[{"text":"Why log the assembler’s inputs and a prefix hash rather than the rendered prompt?"}],"options":[{"text":[{"text":"Rendered prompts cannot be stored, since providers treat the final serialisation as proprietary."}],"correct":false,"tempting":[{"text":"You render it yourself and may store it freely; the reasons not to are cost and data handling, not permission."}]},{"text":[{"text":"A pure assembler rebuilds the exact request from them, at a fraction of the size and sensitivity."}],"correct":true,"tempting":null},{"text":[{"text":"Hashes compress better than text, so a hash of the prefix preserves the same information more efficiently."}],"correct":false,"tempting":[{"text":"A hash is one-way and preserves nothing of the content; it identifies a prefix rather than storing it."}]},{"text":[{"text":"Rendered prompts change between library versions, so a stored copy would not match what runs today."}],"correct":false,"tempting":[{"text":"True of templates and an argument for logging inputs, but a stored render is still an accurate record of that day, which is what a log is for."}]}],"order":[3,1,0,2],"why":null},"prev":{"slug":"judging-with-a-model","title":"Using a model to mark the homework","minutes":9},"next":{"slug":"shipping-a-context-change","title":"Rolling out a prompt like a deployment","minutes":9}}}