{"course":{"slug":"cloud-and-shipping","title":"Shipping It","url":"https://addaly.com/learn/cloud-and-shipping","level":"intermediate","track":"build"},"lesson":{"slug":"data-you-must-delete","title":"Data you must keep, and data you must delete","url":"https://addaly.com/learn/cloud-and-shipping/data-you-must-delete","position":41,"of":74,"minutes":9,"answer":"Every store of personal data is a liability with a lifetime, so decide the number for each one and enforce it with a job rather than a policy document.","keyIdea":"Every store of personal data is a liability with a lifetime, so decide the number for each one and enforce it with a job rather than a policy document."},"body":{"format":"markdown","text":"## You are collecting more than you think\n\nA live site accumulates personal data whether anyone designed for it or not:\n\n- IP addresses and user agents in web server logs.\n- Email addresses, obviously, and also in support threads, in error reports, in a CSV somebody exported.\n- Uploaded images with EXIF metadata, which routinely contains GPS coordinates and a device serial number.\n- Session records, password reset tokens, login history.\n- Whatever your analytics provider set in a cookie.\n\nThe first move is not a policy. It is to write down what you have, where it lives and why. That list is usually shorter than feared and longer than expected, and you cannot manage what you have not enumerated.\n\n## Collect less\n\nThe cheapest way to not leak something is to not have it.\n\nYou do not need a date of birth to run a forum. You do not need a full address to send an email. You do not need to store the raw IP when a truncated one or a daily-rotated hash answers the same question. Every field you decline to collect is a field that cannot appear in a breach, cannot be subpoenaed, and cannot be asked about in a data request.\n\n## Retention is a number, enforced by a job\n\n\"We keep data as long as necessary\" is not a retention policy; it means forever. Pick a number for each store and make something delete it:\n\n- Web server and application logs: 14 to 30 days.\n- Anything request-level containing an IP or user ID: shorter, 7 days is common.\n- Backups: 30 to 90 days, on a rolling schedule.\n- Soft-deleted records: purged after 30 days.\n\nThen write the job, schedule it, and alert if it stops running. A retention policy that lives only in a document is a claim you are making publicly and not honouring, which is materially worse than not making it.\n\n## Deletion has to reach every copy\n\nWhen someone asks to be deleted, the row in your primary database is the easy part. The copies are:\n\n- Read replicas: they follow, so this is fine.\n- **Backups.** The one everybody forgets.\n- The search index.\n- The cache.\n- The CDN, if user content is served through it.\n- Your analytics and error-tracking providers, which are separate systems with their own deletion APIs.\n- The CSV someone exported to a laptop last March.\n\nThe honest position on backups is that you cannot surgically remove one person from a backup archive without destroying its integrity. The accepted answer is to state the retention window plainly: deleted from live systems immediately, gone from backups within N days as the backups age out. Write that N in your policy. A specific number is both honest and defensible; silence is neither.\n\n## Soft delete, hard delete, and the lawful exception\n\nSoft delete (a `deleted_at` column) keeps referential integrity, keeps replies from orphaning, and keeps an audit trail. It is the right default for content.\n\nIt does not satisfy \"erase my personal data\", because the data is still there. So you need a real erasure path: anonymise the identity, replace the display name and email with placeholders, drop the avatar, and keep the content rows if they must stay for others' sake.\n\nAnd there are lawful exceptions worth being precise about rather than vague. This project keeps a `legal_holds` record for content it is legally obliged to preserve and report; that record deliberately survives deletion of the account, and its author reference is intentionally not a foreign key so that erasing the identity does not remove it. That is retention under a legal obligation, and it is a recognised exemption from erasure.\n\nThe lesson generalises: **name your exceptions in your policy.** \"We delete everything on request, except records we are legally required to retain, which are X and kept for Y\" is honest and correct. \"We delete everything\" while quietly keeping something is neither.\n\n## The rules of thumb, without a lawyer\n\nYou do not need legal advice to be broadly right:\n\n- Under the GDPR you have **one month** to answer an access or erasure request, and **72 hours** to notify a supervisory authority of a personal data breach. India's DPDP Act 2023 is different in detail and similar in shape.\n- Tell people what you collect, why, how long, and who else sees it. In plain language, on a page anyone can reach.\n- Give them a working button, not an email address that goes to somebody who left.\n\nThat is most of it. The failure mode for small teams is not usually the wrong retention period; it is a privacy page that describes a system nobody built.\n\n## Never log the secret\n\nRedact at the logging layer, not by remembering: `Authorization` headers, cookies, passwords, tokens, OTP codes, full card numbers. Configure the redaction in one place, because \"we will be careful\" fails the first time somebody logs an entire request object during a debugging session and it goes to production."},"check":{"question":"A user asks you to delete their account. You delete their rows from the primary database. Which part of a correct answer are you most likely to have missed?","options":[{"text":"The read replicas, which may still hold their rows","correct":false,"tempting":"Replicas apply the same deletion automatically through replication; they are the copy you do not have to think about."},{"text":"The soft-delete flag, which leaves the row present in the table","correct":false,"tempting":"A real distinction, but the question states the rows were deleted, and soft delete is a design choice you would already know you had made."},{"text":"Their session records, which will expire on their own","correct":false,"tempting":"Expiring sessions do clear themselves, and they are among the least consequential copies."},{"text":"The backups, the search index and the third-party services, none of which are reached by deleting the row, and the backup window is the number your policy should state","correct":true}]},"prev":{"slug":"the-security-you-actually-need","title":"The short list that actually matters","url":"https://addaly.com/learn/cloud-and-shipping/the-security-you-actually-need"},"next":{"slug":"the-things-that-expire","title":"Everything expires, and nothing tells you","url":"https://addaly.com/learn/cloud-and-shipping/the-things-that-expire"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Data you must keep, and data you must delete', in Shipping It. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/cloud-and-shipping/data-you-must-delete (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Data you must keep, and data you must delete","courseTitle":"Shipping It","module":{"index":5,"slug":"staying-alive","title":"Staying alive: the bad day and the day after"},"answer":null,"keyIdea":"Every store of personal data is a liability with a lifetime, so decide the number for each one and enforce it with a job rather than a policy document.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"you-are-collecting-more-than-you-think","runs":[{"text":"You are collecting more than you think"}]},{"t":"p","runs":[{"text":"A live site accumulates personal data whether anyone designed for it or not:"}]},{"t":"ul","items":[[{"text":"IP addresses and user agents in web server logs."}],[{"text":"Email addresses, obviously, and also in support threads, in error reports, in a CSV somebody exported."}],[{"text":"Uploaded images with EXIF metadata, which routinely contains GPS coordinates and a device serial number."}],[{"text":"Session records, password reset tokens, login history."}],[{"text":"Whatever your analytics provider set in a cookie."}]]},{"t":"p","runs":[{"text":"The first move is not a policy. It is to write down what you have, where it lives and why. That list is usually shorter than feared and longer than expected, and you cannot manage what you have not enumerated."}]},{"t":"h2","id":"collect-less","runs":[{"text":"Collect less"}]},{"t":"p","runs":[{"text":"The cheapest way to not leak something is to not have it."}]},{"t":"p","runs":[{"text":"You do not need a date of birth to run a forum. You do not need a full address to send an email. You do not need to store the raw IP when a truncated one or a daily-rotated hash answers the same question. Every field you decline to collect is a field that cannot appear in a breach, cannot be subpoenaed, and cannot be asked about in a data request."}]},{"t":"h2","id":"retention-is-a-number-enforced-by-a-job","runs":[{"text":"Retention is a number, enforced by a job"}]},{"t":"p","runs":[{"text":"“We keep data as long as necessary” is not a retention policy; it means forever. Pick a number for each store and make something delete it:"}]},{"t":"ul","items":[[{"text":"Web server and application logs: 14 to 30 days."}],[{"text":"Anything request-level containing an IP or user ID: shorter, 7 days is common."}],[{"text":"Backups: 30 to 90 days, on a rolling schedule."}],[{"text":"Soft-deleted records: purged after 30 days."}]]},{"t":"p","runs":[{"text":"Then write the job, schedule it, and alert if it stops running. A retention policy that lives only in a document is a claim you are making publicly and not honouring, which is materially worse than not making it."}]},{"t":"h2","id":"deletion-has-to-reach-every-copy","runs":[{"text":"Deletion has to reach every copy"}]},{"t":"p","runs":[{"text":"When someone asks to be deleted, the row in your primary database is the easy part. The copies are:"}]},{"t":"ul","items":[[{"text":"Read replicas: they follow, so this is fine."}],[{"text":"Backups.","b":true},{"text":" The one everybody forgets."}],[{"text":"The search index."}],[{"text":"The cache."}],[{"text":"The CDN, if user content is served through it."}],[{"text":"Your analytics and error-tracking providers, which are separate systems with their own deletion APIs."}],[{"text":"The CSV someone exported to a laptop last March."}]]},{"t":"p","runs":[{"text":"The honest position on backups is that you cannot surgically remove one person from a backup archive without destroying its integrity. The accepted answer is to state the retention window plainly: deleted from live systems immediately, gone from backups within N days as the backups age out. Write that N in your policy. A specific number is both honest and defensible; silence is neither."}]},{"t":"h2","id":"soft-delete-hard-delete-and-the-lawful-exception","runs":[{"text":"Soft delete, hard delete, and the lawful exception"}]},{"t":"p","runs":[{"text":"Soft delete (a "},{"text":"deleted_at","code":true},{"text":" column) keeps referential integrity, keeps replies from orphaning, and keeps an audit trail. It is the right default for content."}]},{"t":"p","runs":[{"text":"It does not satisfy “erase my personal data”, because the data is still there. So you need a real erasure path: anonymise the identity, replace the display name and email with placeholders, drop the avatar, and keep the content rows if they must stay for others’ sake."}]},{"t":"p","runs":[{"text":"And there are lawful exceptions worth being precise about rather than vague. This project keeps a "},{"text":"legal_holds","code":true},{"text":" record for content it is legally obliged to preserve and report; that record deliberately survives deletion of the account, and its author reference is intentionally not a foreign key so that erasing the identity does not remove it. That is retention under a legal obligation, and it is a recognised exemption from erasure."}]},{"t":"p","runs":[{"text":"The lesson generalises: "},{"text":"name your exceptions in your policy.","b":true},{"text":" “We delete everything on request, except records we are legally required to retain, which are X and kept for Y” is honest and correct. “We delete everything” while quietly keeping something is neither."}]},{"t":"h2","id":"the-rules-of-thumb-without-a-lawyer","runs":[{"text":"The rules of thumb, without a lawyer"}]},{"t":"p","runs":[{"text":"You do not need legal advice to be broadly right:"}]},{"t":"ul","items":[[{"text":"Under the GDPR you have "},{"text":"one month","b":true},{"text":" to answer an access or erasure request, and "},{"text":"72 hours","b":true},{"text":" to notify a supervisory authority of a personal data breach. India’s DPDP Act 2023 is different in detail and similar in shape."}],[{"text":"Tell people what you collect, why, how long, and who else sees it. In plain language, on a page anyone can reach."}],[{"text":"Give them a working button, not an email address that goes to somebody who left."}]]},{"t":"p","runs":[{"text":"That is most of it. The failure mode for small teams is not usually the wrong retention period; it is a privacy page that describes a system nobody built."}]},{"t":"h2","id":"never-log-the-secret","runs":[{"text":"Never log the secret"}]},{"t":"p","runs":[{"text":"Redact at the logging layer, not by remembering: "},{"text":"Authorization","code":true},{"text":" headers, cookies, passwords, tokens, OTP codes, full card numbers. Configure the redaction in one place, because “we will be careful” fails the first time somebody logs an entire request object during a debugging session and it goes to production."}]}],"check":{"question":[{"text":"A user asks you to delete their account. You delete their rows from the primary database. Which part of a correct answer are you most likely to have missed?"}],"options":[{"text":[{"text":"The read replicas, which may still hold their rows"}],"correct":false,"tempting":[{"text":"Replicas apply the same deletion automatically through replication; they are the copy you do not have to think about."}]},{"text":[{"text":"The soft-delete flag, which leaves the row present in the table"}],"correct":false,"tempting":[{"text":"A real distinction, but the question states the rows were deleted, and soft delete is a design choice you would already know you had made."}]},{"text":[{"text":"Their session records, which will expire on their own"}],"correct":false,"tempting":[{"text":"Expiring sessions do clear themselves, and they are among the least consequential copies."}]},{"text":[{"text":"The backups, the search index and the third-party services, none of which are reached by deleting the row, and the backup window is the number your policy should state"}],"correct":true,"tempting":null}],"order":[2,0,3,1],"why":null},"prev":{"slug":"the-security-you-actually-need","title":"The short list that actually matters","minutes":10},"next":{"slug":"the-things-that-expire","title":"Everything expires, and nothing tells you","minutes":8}}}