{"course":{"slug":"agents-and-automation","title":"Agents and Automation","url":"https://addaly.com/learn/agents-and-automation","level":"intermediate","track":"build"},"lesson":{"slug":"what-it-keeps-and-what-it-must-forget","title":"Retention: the duty to keep and the duty to delete","url":"https://addaly.com/learn/agents-and-automation/what-it-keeps-and-what-it-must-forget","position":71,"of":72,"minutes":9,"answer":"Keep a decision record rather than the full payload, so the duty to explain what happened stops competing with the duty to delete personal data.","keyIdea":"Keep a decision record rather than the full payload, so the duty to explain what happened stops competing with the duty to delete personal data."},"body":{"format":"markdown","text":"## Two duties that pull against each other\n\nYou must be able to explain what the system did, months later, to an auditor or an affected person. And you must be able to delete personal data: on request, or when the retention period ends.\n\nTeams treat this as a contradiction and resolve it by keeping everything, which is the wrong answer and the common one.\n\n## The resolution is scoping\n\nThe record that satisfies the first duty is not the same as the data that triggers the second. Separate them deliberately:\n\n**The decision record**, kept long: a reference to the input rather than its contents, a hash of the input, the decision, the config version, the timestamp, whether a person reviewed it, and the outcome. This is small, it is what an audit actually asks for, and it usually contains little or no personal data beyond an identifier you already hold elsewhere.\n\n**The full payload**, kept briefly: the document text, the transcript, the tool results. Retained long enough to debug (thirty days is often plenty) then deleted automatically.\n\nWith that split, an erasure request removes the payload and the identifier link while the decision record survives as a countable event. You can still say \"on 4 March, 1,204 claims were assessed, 87 routed to review, under config version 14\" without holding anybody's claim text.\n\n> Figure: One duty to explain, one duty to delete. The decision record, kept for years: A reference to the input, and a hash of it, rather than its contents; The decision, and the config version in force; Whether a person reviewed it, and who; The timestamp, and the outcome. The full payload, kept about thirty days: The document text and the whole transcript; Every tool result the agent read; The error report that captured the payload which failed; The evaluation fixture somebody copied a real record into during a busy week. With the split, an erasure request removes the payload and the identifier link while the record survives as a countable event: on 4 March, 1,204 claims assessed, 87 routed to review, under config version 14.\n\nDesign this on day one. Retrofitting it means discovering that the only record of what happened is the transcript you are being asked to delete.\n\n## Deletion has to reach every copy\n\nMake the list, because it is longer than anybody's first guess:\n\n- The primary database.\n- The trace and log store, which holds everything the agent read.\n- Blob storage for the original documents.\n- The vector index, where the embedding of a deleted chunk otherwise remains and remains retrievable.\n- Error reports, which capture the payload that failed.\n- Evaluation fixtures: real records somebody copied into a test set during a busy week, and the copy nobody thinks about.\n- Anything you sent to a provider, under whatever retention their terms specify.\n- Backups.\n\nBackups are the hard one, and the honest answer is not \"we delete from backups\". You cannot rewrite immutable snapshots, and nobody sensibly restores a six-month-old backup to remove a row. The workable approach, and the one regulators broadly accept when it is documented, is: backups age out on a stated schedule, and deletions are re-applied on any restore. Write that down as a procedure. An undocumented gap is a finding; a documented, reasoned approach is a conversation.\n\n## Test the deletion path\n\nOnce a quarter, take a test record, exercise the deletion, and verify across every store on the list. Record the result with a date.\n\nAn untested deletion path does not work. That is not cynicism: the failure is nearly always the store somebody added six months after the path was written: a new cache, a new index, a reporting copy. Only the drill finds it.\n\n## Retention is a decision, not a default\n\nFor every store, write down how long and why. \"Ninety days, because that is the window in which disputes arise\" is a reason. \"Indefinitely, because storage is cheap\" is not, and it converts a data asset into a growing liability that somebody will eventually have to explain.\n\nThen enforce it with a scheduled job that actually deletes, and alert if that job does not run. A retention policy that exists only in a document is the same shape as a budget that exists only in a document, and it fails the same way.\n\n## The narrow exception\n\nSometimes a duty to preserve overrides deletion: litigation, a regulatory investigation, or an obligation to retain and report certain material. That exception has to be narrow, explicit and logged: a specific record, a stated reason, an expiry, and an owner.\n\nThe system this course is published on does exactly this: content removed for a small set of legally reportable categories is preserved in a separate hold that survives both content deletion and account deletion, precisely because the retention obligation and the erasure right point in opposite directions and the conflict is resolved once, deliberately, in code. Build your exception the same way: as a named path with a reason attached, never as a general reluctance to delete things."},"check":{"question":"An automation must be able to explain decisions to auditors for seven years and must also honour deletion requests for personal data. How is that resolved without keeping everything?","options":[{"text":"Encrypt the stored payloads and destroy the key on request, so the data is unrecoverable but the record remains.","correct":false,"tempting":"Crypto-shredding is a legitimate technique in some settings, but it leaves the ciphertext in place and its status as deletion is contested and jurisdiction-dependent."},{"text":"Keep a small decision record (input reference and hash, decision, config version, reviewer, timestamp) for the long period, and delete the full payload on a short schedule.","correct":true},{"text":"Anonymise the stored payloads by removing names, so they can be retained for the full seven years.","correct":false,"tempting":"Stripping names leaves quasi-identifiers that still identify people, so the retained data remains personal data with the same duties attached."},{"text":"Retain everything under the audit obligation, since a legal duty to keep records overrides a deletion request.","correct":false,"tempting":"Preservation duties are narrow and specific; treating a general audit expectation as a blanket override is the reasoning that produces the finding."}]},"prev":{"slug":"rules-that-differ-by-country","title":"The rules, where they agree and where they do not","url":"https://addaly.com/learn/agents-and-automation/rules-that-differ-by-country"},"next":{"slug":"retiring-an-automation","title":"Switching it off, on purpose","url":"https://addaly.com/learn/agents-and-automation/retiring-an-automation"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Retention: the duty to keep and the duty to delete', in Agents and Automation. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/agents-and-automation/what-it-keeps-and-what-it-must-forget (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Retention: the duty to keep and the duty to delete","courseTitle":"Agents and Automation","module":{"index":8,"slug":"living-with-it","title":"The year after it ships"},"answer":null,"keyIdea":"Keep a decision record rather than the full payload, so the duty to explain what happened stops competing with the duty to delete personal data.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"left":{"title":"The decision record, kept for years","items":["A reference to the input, and a hash of it, rather than its contents","The decision, and the config version in force","Whether a person reviewed it, and who","The timestamp, and the outcome"]},"type":"compare","right":{"title":"The full payload, kept about thirty days","items":["The document text and the whole transcript","Every tool result the agent read","The error report that captured the payload which failed","The evaluation fixture somebody copied a real record into during a busy week"]},"title":"One duty to explain, one duty to delete","caption":"With the split, an erasure request removes the payload and the identifier link while the record survives as a countable event: on 4 March, 1,204 claims assessed, 87 routed to review, under config version 14."},"alt":"Figure: One duty to explain, one duty to delete. The decision record, kept for years: A reference to the input, and a hash of it, rather than its contents; The decision, and the config version in force; Whether a person reviewed it, and who; The timestamp, and the outcome. The full payload, kept about thirty days: The document text and the whole transcript; Every tool result the agent read; The error report that captured the payload which failed; The evaluation fixture somebody copied a real record into during a busy week. With the split, an erasure request removes the payload and the identifier link while the record survives as a countable event: on 4 March, 1,204 claims assessed, 87 routed to review, under config version 14."}},"blocks":[{"t":"h2","id":"two-duties-that-pull-against-each-other","runs":[{"text":"Two duties that pull against each other"}]},{"t":"p","runs":[{"text":"You must be able to explain what the system did, months later, to an auditor or an affected person. And you must be able to delete personal data: on request, or when the retention period ends."}]},{"t":"p","runs":[{"text":"Teams treat this as a contradiction and resolve it by keeping everything, which is the wrong answer and the common one."}]},{"t":"h2","id":"the-resolution-is-scoping","runs":[{"text":"The resolution is scoping"}]},{"t":"p","runs":[{"text":"The record that satisfies the first duty is not the same as the data that triggers the second. Separate them deliberately:"}]},{"t":"p","runs":[{"text":"The decision record","b":true},{"text":", kept long: a reference to the input rather than its contents, a hash of the input, the decision, the config version, the timestamp, whether a person reviewed it, and the outcome. This is small, it is what an audit actually asks for, and it usually contains little or no personal data beyond an identifier you already hold elsewhere."}]},{"t":"p","runs":[{"text":"The full payload","b":true},{"text":", kept briefly: the document text, the transcript, the tool results. Retained long enough to debug (thirty days is often plenty) then deleted automatically."}]},{"t":"p","runs":[{"text":"With that split, an erasure request removes the payload and the identifier link while the decision record survives as a countable event. You can still say “on 4 March, 1,204 claims were assessed, 87 routed to review, under config version 14” without holding anybody’s claim text."}]},{"t":"p","runs":[{"text":"Design this on day one. Retrofitting it means discovering that the only record of what happened is the transcript you are being asked to delete."}]},{"t":"h2","id":"deletion-has-to-reach-every-copy","runs":[{"text":"Deletion has to reach every copy"}]},{"t":"p","runs":[{"text":"Make the list, because it is longer than anybody’s first guess:"}]},{"t":"ul","items":[[{"text":"The primary database."}],[{"text":"The trace and log store, which holds everything the agent read."}],[{"text":"Blob storage for the original documents."}],[{"text":"The vector index, where the embedding of a deleted chunk otherwise remains and remains retrievable."}],[{"text":"Error reports, which capture the payload that failed."}],[{"text":"Evaluation fixtures: real records somebody copied into a test set during a busy week, and the copy nobody thinks about."}],[{"text":"Anything you sent to a provider, under whatever retention their terms specify."}],[{"text":"Backups."}]]},{"t":"p","runs":[{"text":"Backups are the hard one, and the honest answer is not “we delete from backups”. You cannot rewrite immutable snapshots, and nobody sensibly restores a six-month-old backup to remove a row. The workable approach, and the one regulators broadly accept when it is documented, is: backups age out on a stated schedule, and deletions are re-applied on any restore. Write that down as a procedure. An undocumented gap is a finding; a documented, reasoned approach is a conversation."}]},{"t":"h2","id":"test-the-deletion-path","runs":[{"text":"Test the deletion path"}]},{"t":"p","runs":[{"text":"Once a quarter, take a test record, exercise the deletion, and verify across every store on the list. Record the result with a date."}]},{"t":"p","runs":[{"text":"An untested deletion path does not work. That is not cynicism: the failure is nearly always the store somebody added six months after the path was written: a new cache, a new index, a reporting copy. Only the drill finds it."}]},{"t":"h2","id":"retention-is-a-decision-not-a-default","runs":[{"text":"Retention is a decision, not a default"}]},{"t":"p","runs":[{"text":"For every store, write down how long and why. “Ninety days, because that is the window in which disputes arise” is a reason. “Indefinitely, because storage is cheap” is not, and it converts a data asset into a growing liability that somebody will eventually have to explain."}]},{"t":"p","runs":[{"text":"Then enforce it with a scheduled job that actually deletes, and alert if that job does not run. A retention policy that exists only in a document is the same shape as a budget that exists only in a document, and it fails the same way."}]},{"t":"h2","id":"the-narrow-exception","runs":[{"text":"The narrow exception"}]},{"t":"p","runs":[{"text":"Sometimes a duty to preserve overrides deletion: litigation, a regulatory investigation, or an obligation to retain and report certain material. That exception has to be narrow, explicit and logged: a specific record, a stated reason, an expiry, and an owner."}]},{"t":"p","runs":[{"text":"The system this course is published on does exactly this: content removed for a small set of legally reportable categories is preserved in a separate hold that survives both content deletion and account deletion, precisely because the retention obligation and the erasure right point in opposite directions and the conflict is resolved once, deliberately, in code. Build your exception the same way: as a named path with a reason attached, never as a general reluctance to delete things."}]}],"check":{"question":[{"text":"An automation must be able to explain decisions to auditors for seven years and must also honour deletion requests for personal data. How is that resolved without keeping everything?"}],"options":[{"text":[{"text":"Encrypt the stored payloads and destroy the key on request, so the data is unrecoverable but the record remains."}],"correct":false,"tempting":[{"text":"Crypto-shredding is a legitimate technique in some settings, but it leaves the ciphertext in place and its status as deletion is contested and jurisdiction-dependent."}]},{"text":[{"text":"Keep a small decision record (input reference and hash, decision, config version, reviewer, timestamp) for the long period, and delete the full payload on a short schedule."}],"correct":true,"tempting":null},{"text":[{"text":"Anonymise the stored payloads by removing names, so they can be retained for the full seven years."}],"correct":false,"tempting":[{"text":"Stripping names leaves quasi-identifiers that still identify people, so the retained data remains personal data with the same duties attached."}]},{"text":[{"text":"Retain everything under the audit obligation, since a legal duty to keep records overrides a deletion request."}],"correct":false,"tempting":[{"text":"Preservation duties are narrow and specific; treating a general audit expectation as a blanket override is the reasoning that produces the finding."}]}],"order":[3,2,1,0],"why":null},"prev":{"slug":"rules-that-differ-by-country","title":"The rules, where they agree and where they do not","minutes":10},"next":{"slug":"retiring-an-automation","title":"Switching it off, on purpose","minutes":9}}}