Deleting what you kept

Context makes copies, and copies outlive the window

A context window is transient. The infrastructure around it is not. By the time a user’s sentence has been through a typical pipeline, it exists in more places than anybody has counted:

• The conversation store.
• The assembled-request log, if you log the rendered prompt, which the measurement module will recommend.
• The trace or observability system.
• The vector index, as an embedding, plus the chunk text stored beside it.
• The keyword index.
• The retrieval cache and any response cache.
• The memory store.
• An evaluation set somebody built from real conversations.
• The provider’s own retention, for whatever period their terms state.
• Your database backups, for their retention period.

Ten places. A deletion request that reaches two of them has not deleted anything; it has made the data harder to find while leaving it there.

Write the inventory before you need it

The single most useful artefact here is a list: every store that can contain user text, who owns it, what its retention is, and how deletion is performed on it.

Write it now, while there are six entries. It is a page. Writing it during an incident or a regulator’s enquiry, when there are eleven entries and two of them are somebody’s notebook, is a different exercise entirely.

And give each entry a retention period chosen on purpose. “Forever, because nobody set one” is the default and it is the wrong answer for every store on that list except possibly the transcript.

Deletion has to be a tested operation

Treat it like any other critical path: it needs a function, a test, and an assertion.

def erase_user(user_id):
    for store in ALL_STORES:          # the inventory, as code
        store.delete_for_user(user_id)
    for store in ALL_STORES:
        assert store.count_for_user(user_id) == 0, store.name

Having ALL_STORES as a list in code, rather than as a paragraph in a document, is what stops the eleventh store from being forgotten when somebody adds it in March. A new store that is not in the list fails the test.

The assertion matters more than the deletion. Plenty of systems have a delete path that silently misses a store, and nobody notices, because a successful deletion looks exactly like a deletion that did nothing.

The parts that are hard

Three honest difficulties, none of which has a tidy answer.

Embeddings. A vector derived from a sentence is not the sentence, but it is derived from it, and research on inverting embeddings back into approximate text has shown it is far more feasible than most people assumed. Treat vectors as personal data and delete them with everything else, rather than arguing that they are anonymous.

Derived artefacts. A summary, a memory entry, a generated chunk context, a fine-tuning dataset. Each contains the user’s information in transformed form. The summary is deletable; a model trained on the data is not, in any practical sense, which is a strong argument for not training on production conversations without a clear basis and a clear consent path.

Backups. You cannot selectively delete from an immutable snapshot. The common practice is a documented backup retention window after which the data ages out, plus a suppression list so restored data is re-deleted. Whether that satisfies a given regulator is exactly the kind of question that needs a lawyer rather than an engineer.

The law

Rights to erasure and correction exist in the EU’s GDPR, India’s DPDP Act, California’s regime and many others. They are broadly similar in shape and differ in the details that matter: what counts as personal data, which exceptions apply, how long you have, whether inferred data is covered, and what a legal-obligation exception permits you to retain.

The application of all of this to AI systems is actively contested and moving. There are open questions (about training data, about model weights as a form of retention, about automated inference) where practitioners, regulators and courts do not yet agree, and where the answer differs by country.

So this lesson does not tell you what the law requires of you, and could not. It tells you what to build so that whatever your lawyers conclude is implementable: an inventory, a per-store retention period, a tested erase function, and no derived artefact whose provenance you cannot trace.

Forgetting as a feature

One last framing, because it is more useful than the compliance one. A user asking “forget that” is asking for a product behaviour, not filing a legal request. Support it directly: a delete button next to each memory, a way to clear a conversation, an ephemeral mode that retains nothing.

Systems that make forgetting easy get used more freely, because a user who cannot delete something learns to be careful about what they say. That caution costs you the context that would have made the product work.