What you owe the people in the data

The rows are people

Every row in a customer table is a person who did not choose to be a training example. That fact does not stop the work, and it does shape a small number of obligations that are cheap to meet and awkward to retrofit.

Write the model card

A model card is one page recording what the model is and is not. Mitchell and colleagues proposed the format in 2019 and it has become standard practice, including on Hugging Face where many published models carry one.

What it contains:

• What it predicts, in one sentence, with the exact target definition and the moment of prediction.
• Who it was trained on: time period, geography, population, and how the rows were selected.
• Performance overall and per slice, with sample sizes for each slice.
• Intended use, and explicitly out-of-scope use. “This was trained on applicants aged 21 to 65 in three states and should not be used outside that population.”
• Known limitations, including which groups it performs worse on and by how much.
• The fairness criterion chosen, from module 5, and what it costs.
• When it was trained and when it should be reviewed.

This takes an hour. It is the difference between a model that someone can use responsibly in two years and one whose limits live only in the memory of a person who has left.

A data card does the same for a dataset (how it was collected, what consent was given, known gaps) and is at least as valuable, because datasets outlive models.

Data minimisation

Collect what you need for the prediction, not everything available.

This is a legal requirement under GDPR and in various forms elsewhere, and it is also good engineering: fewer fields means less to maintain, less to secure, less to leak, and fewer routes for a protected attribute to enter the model by proxy.

The question to ask of each field is not “could this help” but “does this help enough to justify holding it”. A model that improves by 0.002 AUC because it knows people’s precise location is a trade you should make deliberately.

Deletion, and what it means for a model

A person exercises their right to erasure. You delete their rows. The model trained on those rows still exists and still encodes some information about them.

Whether a trained model constitutes personal data is unsettled, and it differs by jurisdiction. Regulators have taken different positions, and there is no settled answer to point at. Some organisations retrain on a schedule that bounds how long any individual’s influence persists; some treat the model as derived, aggregated output; some do neither and have not been tested.

The research area is called machine unlearning, and it is early: exact unlearning is expensive and approximate methods are hard to verify. What you can do now: retrain regularly enough that deletion propagates within a stated period, and write down what that period is.

And note that legal holds cut the other way. When retention is required by law (a fraud investigation, a regulatory obligation) that obligation can override an erasure request. Both rules exist and they conflict, and which wins is a legal question rather than an engineering one.

Privacy attacks are real and often overstated

Be accurate about this rather than alarming.

Membership inference (determining whether a specific record was in the training set) works better against overfitted models and against those trained on small datasets. It is a genuine risk and a well-regularised model on a large dataset leaks much less.

Model inversion (reconstructing training examples) has been demonstrated, most convincingly against large generative models that can reproduce memorised training text or images. Reconstructing a row of tabular training data from a gradient boosting model is far harder.

Differential privacy provides a formal guarantee by adding calibrated noise during training. It costs accuracy, sometimes a lot, and it is the right answer when the guarantee is required. Opacus for PyTorch is the free implementation. It is not a routine default and should not be presented as one.

The law is unsettled, and saying so is the honest position

Three questions with no agreed answer across jurisdictions, and this course will not pretend otherwise:

• Whether training on copyrighted material is permitted. Different countries take different views, several major cases are unresolved, and the fair-use and text-and-data-mining arguments have not been settled by courts in a way that transfers internationally.
• Who owns a model’s output, and whether it can be copyrighted at all.
• What counts as a decision requiring human review under provisions like GDPR Article 22, and how far automated support goes before it becomes automated decision-making.

These differ by country and are moving. If your work touches any of them, that is a question for a lawyer in your jurisdiction and not for a course. What a course can do is tell you the questions exist, so that you notice you are inside one.

The last thing

The most common ethical failure in this field is not a bad decision. It is an undocumented one: a threshold set without a reason recorded, a fairness criterion chosen by default, a population excluded without anyone noticing.

Write down what you chose and why. It costs an hour, it survives your departure, and it is what turns a model from a black box somebody inherited into a decision somebody can be accountable for.