{"course":{"slug":"machine-learning-foundations","title":"Machine Learning, Foundations","url":"https://addaly.com/learn/machine-learning-foundations","level":"intermediate","track":"start"},"lesson":{"slug":"what-you-owe-the-people-in-the-data","title":"What you owe the people in the data","url":"https://addaly.com/learn/machine-learning-foundations/what-you-owe-the-people-in-the-data","position":92,"of":92,"minutes":9,"answer":"A model card recording the target definition, training population, per-slice performance and out-of-scope uses costs an hour and outlives everyone involved; and the legal questions around copyright.","keyIdea":"A model card recording the target definition, training population, per-slice performance and out-of-scope uses costs an hour and outlives everyone involved; and the legal questions around copyright, erasure and automated decisions are genuinely unresolved and differ by country."},"body":{"format":"markdown","text":"## The rows are people\n\nEvery row in a customer table is a person who did not choose to be a training example. That fact does not stop the work, and it does shape a small number of obligations that are cheap to meet and awkward to retrofit.\n\n## Write the model card\n\nA model card is one page recording what the model is and is not. Mitchell and colleagues proposed the format in 2019 and it has become standard practice, including on Hugging Face where many published models carry one.\n\nWhat it contains:\n\n- **What it predicts**, in one sentence, with the exact target definition and the moment of prediction.\n- **Who it was trained on**: time period, geography, population, and how the rows were selected.\n- **Performance overall and per slice**, with sample sizes for each slice.\n- **Intended use**, and explicitly **out-of-scope use**. \"This was trained on applicants aged 21 to 65 in three states and should not be used outside that population.\"\n- **Known limitations**, including which groups it performs worse on and by how much.\n- **The fairness criterion chosen**, from module 5, and what it costs.\n- **When it was trained and when it should be reviewed.**\n\nThis takes an hour. It is the difference between a model that someone can use responsibly in two years and one whose limits live only in the memory of a person who has left.\n\nA data card does the same for a dataset (how it was collected, what consent was given, known gaps) and is at least as valuable, because datasets outlive models.\n\n## Data minimisation\n\nCollect what you need for the prediction, not everything available.\n\nThis is a legal requirement under GDPR and in various forms elsewhere, and it is also good engineering: fewer fields means less to maintain, less to secure, less to leak, and fewer routes for a protected attribute to enter the model by proxy.\n\nThe question to ask of each field is not \"could this help\" but \"does this help enough to justify holding it\". A model that improves by 0.002 AUC because it knows people's precise location is a trade you should make deliberately.\n\n## Deletion, and what it means for a model\n\nA person exercises their right to erasure. You delete their rows. The model trained on those rows still exists and still encodes some information about them.\n\nWhether a trained model constitutes personal data is unsettled, and it differs by jurisdiction. Regulators have taken different positions, and there is no settled answer to point at. Some organisations retrain on a schedule that bounds how long any individual's influence persists; some treat the model as derived, aggregated output; some do neither and have not been tested.\n\nThe research area is called machine unlearning, and it is early: exact unlearning is expensive and approximate methods are hard to verify. What you can do now: retrain regularly enough that deletion propagates within a stated period, and write down what that period is.\n\nAnd note that legal holds cut the other way. When retention is required by law (a fraud investigation, a regulatory obligation) that obligation can override an erasure request. Both rules exist and they conflict, and which wins is a legal question rather than an engineering one.\n\n## Privacy attacks are real and often overstated\n\nBe accurate about this rather than alarming.\n\n**Membership inference** (determining whether a specific record was in the training set) works better against overfitted models and against those trained on small datasets. It is a genuine risk and a well-regularised model on a large dataset leaks much less.\n\n**Model inversion** (reconstructing training examples) has been demonstrated, most convincingly against large generative models that can reproduce memorised training text or images. Reconstructing a row of tabular training data from a gradient boosting model is far harder.\n\n**Differential privacy** provides a formal guarantee by adding calibrated noise during training. It costs accuracy, sometimes a lot, and it is the right answer when the guarantee is required. `Opacus` for PyTorch is the free implementation. It is not a routine default and should not be presented as one.\n\n## The law is unsettled, and saying so is the honest position\n\nThree questions with no agreed answer across jurisdictions, and this course will not pretend otherwise:\n\n- **Whether training on copyrighted material is permitted.** Different countries take different views, several major cases are unresolved, and the fair-use and text-and-data-mining arguments have not been settled by courts in a way that transfers internationally.\n- **Who owns a model's output**, and whether it can be copyrighted at all.\n- **What counts as a decision requiring human review** under provisions like GDPR Article 22, and how far automated support goes before it becomes automated decision-making.\n\nThese differ by country and are moving. If your work touches any of them, that is a question for a lawyer in your jurisdiction and not for a course. What a course can do is tell you the questions exist, so that you notice you are inside one.\n\n## The last thing\n\nThe most common ethical failure in this field is not a bad decision. It is an undocumented one: a threshold set without a reason recorded, a fairness criterion chosen by default, a population excluded without anyone noticing.\n\nWrite down what you chose and why. It costs an hour, it survives your departure, and it is what turns a model from a black box somebody inherited into a decision somebody can be accountable for."},"check":{"question":"A user exercises their right to erasure and the team deletes their rows from the warehouse. What remains unresolved?","options":[{"text":"Nothing: deleting the source rows satisfies the request completely under every framework.","correct":false,"tempting":"Treats the question as settled when regulators and jurisdictions differ on whether a trained model derived from those rows is itself in scope."},{"text":"The backup copies of the warehouse, which must also be purged before the request is complete.","correct":false,"tempting":"Backup handling is a real operational requirement, but it is a well-understood one rather than the genuinely unsettled question here."},{"text":"Whether the model trained on those rows must also be retrained, since whether a trained model constitutes personal data is disputed and differs by jurisdiction.","correct":true},{"text":"Whether the deletion breaks the model's calibration, which would require recalibration on the remaining data.","correct":false,"tempting":"Removing one person's rows has no measurable effect on calibration and is a technical rather than a legal concern."}]},"prev":{"slug":"feedback-loops","title":"When the model changes the data it will be trained on","url":"https://addaly.com/learn/machine-learning-foundations/feedback-loops"},"next":null,"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What you owe the people in the data', in Machine Learning, Foundations. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/machine-learning-foundations/what-you-owe-the-people-in-the-data (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What you owe the people in the data","courseTitle":"Machine Learning, Foundations","module":{"index":9,"slug":"into-the-world","title":"Into the world, and keeping it honest"},"answer":null,"keyIdea":"A model card recording the target definition, training population, per-slice performance and out-of-scope uses costs an hour and outlives everyone involved; and the legal questions around copyright, erasure and automated decisions are genuinely unresolved and differ by country.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-rows-are-people","runs":[{"text":"The rows are people"}]},{"t":"p","runs":[{"text":"Every row in a customer table is a person who did not choose to be a training example. That fact does not stop the work, and it does shape a small number of obligations that are cheap to meet and awkward to retrofit."}]},{"t":"h2","id":"write-the-model-card","runs":[{"text":"Write the model card"}]},{"t":"p","runs":[{"text":"A model card is one page recording what the model is and is not. Mitchell and colleagues proposed the format in 2019 and it has become standard practice, including on Hugging Face where many published models carry one."}]},{"t":"p","runs":[{"text":"What it contains:"}]},{"t":"ul","items":[[{"text":"What it predicts","b":true},{"text":", in one sentence, with the exact target definition and the moment of prediction."}],[{"text":"Who it was trained on","b":true},{"text":": time period, geography, population, and how the rows were selected."}],[{"text":"Performance overall and per slice","b":true},{"text":", with sample sizes for each slice."}],[{"text":"Intended use","b":true},{"text":", and explicitly "},{"text":"out-of-scope use","b":true},{"text":". “This was trained on applicants aged 21 to 65 in three states and should not be used outside that population.”"}],[{"text":"Known limitations","b":true},{"text":", including which groups it performs worse on and by how much."}],[{"text":"The fairness criterion chosen","b":true},{"text":", from module 5, and what it costs."}],[{"text":"When it was trained and when it should be reviewed.","b":true}]]},{"t":"p","runs":[{"text":"This takes an hour. It is the difference between a model that someone can use responsibly in two years and one whose limits live only in the memory of a person who has left."}]},{"t":"p","runs":[{"text":"A data card does the same for a dataset (how it was collected, what consent was given, known gaps) and is at least as valuable, because datasets outlive models."}]},{"t":"h2","id":"data-minimisation","runs":[{"text":"Data minimisation"}]},{"t":"p","runs":[{"text":"Collect what you need for the prediction, not everything available."}]},{"t":"p","runs":[{"text":"This is a legal requirement under GDPR and in various forms elsewhere, and it is also good engineering: fewer fields means less to maintain, less to secure, less to leak, and fewer routes for a protected attribute to enter the model by proxy."}]},{"t":"p","runs":[{"text":"The question to ask of each field is not “could this help” but “does this help enough to justify holding it”. A model that improves by 0.002 AUC because it knows people’s precise location is a trade you should make deliberately."}]},{"t":"h2","id":"deletion-and-what-it-means-for-a-model","runs":[{"text":"Deletion, and what it means for a model"}]},{"t":"p","runs":[{"text":"A person exercises their right to erasure. You delete their rows. The model trained on those rows still exists and still encodes some information about them."}]},{"t":"p","runs":[{"text":"Whether a trained model constitutes personal data is unsettled, and it differs by jurisdiction. Regulators have taken different positions, and there is no settled answer to point at. Some organisations retrain on a schedule that bounds how long any individual’s influence persists; some treat the model as derived, aggregated output; some do neither and have not been tested."}]},{"t":"p","runs":[{"text":"The research area is called machine unlearning, and it is early: exact unlearning is expensive and approximate methods are hard to verify. What you can do now: retrain regularly enough that deletion propagates within a stated period, and write down what that period is."}]},{"t":"p","runs":[{"text":"And note that legal holds cut the other way. When retention is required by law (a fraud investigation, a regulatory obligation) that obligation can override an erasure request. Both rules exist and they conflict, and which wins is a legal question rather than an engineering one."}]},{"t":"h2","id":"privacy-attacks-are-real-and-often-overstated","runs":[{"text":"Privacy attacks are real and often overstated"}]},{"t":"p","runs":[{"text":"Be accurate about this rather than alarming."}]},{"t":"p","runs":[{"text":"Membership inference","b":true},{"text":" (determining whether a specific record was in the training set) works better against overfitted models and against those trained on small datasets. It is a genuine risk and a well-regularised model on a large dataset leaks much less."}]},{"t":"p","runs":[{"text":"Model inversion","b":true},{"text":" (reconstructing training examples) has been demonstrated, most convincingly against large generative models that can reproduce memorised training text or images. Reconstructing a row of tabular training data from a gradient boosting model is far harder."}]},{"t":"p","runs":[{"text":"Differential privacy","b":true},{"text":" provides a formal guarantee by adding calibrated noise during training. It costs accuracy, sometimes a lot, and it is the right answer when the guarantee is required. "},{"text":"Opacus","code":true},{"text":" for PyTorch is the free implementation. It is not a routine default and should not be presented as one."}]},{"t":"h2","id":"the-law-is-unsettled-and-saying-so-is-the-honest-position","runs":[{"text":"The law is unsettled, and saying so is the honest position"}]},{"t":"p","runs":[{"text":"Three questions with no agreed answer across jurisdictions, and this course will not pretend otherwise:"}]},{"t":"ul","items":[[{"text":"Whether training on copyrighted material is permitted.","b":true},{"text":" Different countries take different views, several major cases are unresolved, and the fair-use and text-and-data-mining arguments have not been settled by courts in a way that transfers internationally."}],[{"text":"Who owns a model’s output","b":true},{"text":", and whether it can be copyrighted at all."}],[{"text":"What counts as a decision requiring human review","b":true},{"text":" under provisions like GDPR Article 22, and how far automated support goes before it becomes automated decision-making."}]]},{"t":"p","runs":[{"text":"These differ by country and are moving. If your work touches any of them, that is a question for a lawyer in your jurisdiction and not for a course. What a course can do is tell you the questions exist, so that you notice you are inside one."}]},{"t":"h2","id":"the-last-thing","runs":[{"text":"The last thing"}]},{"t":"p","runs":[{"text":"The most common ethical failure in this field is not a bad decision. It is an undocumented one: a threshold set without a reason recorded, a fairness criterion chosen by default, a population excluded without anyone noticing."}]},{"t":"p","runs":[{"text":"Write down what you chose and why. It costs an hour, it survives your departure, and it is what turns a model from a black box somebody inherited into a decision somebody can be accountable for."}]}],"check":{"question":[{"text":"A user exercises their right to erasure and the team deletes their rows from the warehouse. What remains unresolved?"}],"options":[{"text":[{"text":"Nothing: deleting the source rows satisfies the request completely under every framework."}],"correct":false,"tempting":[{"text":"Treats the question as settled when regulators and jurisdictions differ on whether a trained model derived from those rows is itself in scope."}]},{"text":[{"text":"The backup copies of the warehouse, which must also be purged before the request is complete."}],"correct":false,"tempting":[{"text":"Backup handling is a real operational requirement, but it is a well-understood one rather than the genuinely unsettled question here."}]},{"text":[{"text":"Whether the model trained on those rows must also be retrained, since whether a trained model constitutes personal data is disputed and differs by jurisdiction."}],"correct":true,"tempting":null},{"text":[{"text":"Whether the deletion breaks the model’s calibration, which would require recalibration on the remaining data."}],"correct":false,"tempting":[{"text":"Removing one person’s rows has no measurable effect on calibration and is a technical rather than a legal concern."}]}],"order":[0,3,2,1],"why":null},"prev":{"slug":"feedback-loops","title":"When the model changes the data it will be trained on","minutes":9},"next":null}}