Why you cannot have all three

A theorem, not an opinion

The previous lesson gave three fairness criteria. This one gives the result that makes choosing between them unavoidable.

If two groups have different base rates for the outcome, then no classifier can simultaneously satisfy calibration within groups and equal false positive and false negative rates, except a perfect classifier, or one that has no predictive power at all.

This was proved independently by Kleinberg, Mullainathan and Raghavan, and by Chouldechova, both in 2016. It is a mathematical result. It does not depend on the algorithm, the data quality, the features, or how careful you are.

Seeing why

The intuition takes a paragraph.

Calibration says: among everyone the model scored at 0.7, the same fraction are actually positive, in both groups. Now suppose group A has a 30% base rate and group B has 10%.

To be calibrated for group B, whose members are less likely to be positive on average, the model has to spread B’s scores differently: a given score has to be reachable by the same fraction of true positives in each group. Doing that while also holding the false positive rate equal across groups is over-determined. The base rate appears in both requirements, pulling in different directions, and the only ways out are a model that gets everything right (no errors to distribute) or a model that predicts nothing (nothing to distribute).

You can verify this yourself with a spreadsheet: fix two base rates, fix a false positive rate, and try to satisfy calibration at every score band. It does not close.

COMPAS, again

This is exactly what happened in the recidivism argument. ProPublica showed unequal false positive rates. Northpointe showed calibration within groups. Base rates in the recorded data differed between groups.

Given that difference, both findings had to be true simultaneously. The theorem says so. Neither party was making an error, and neither could have satisfied the other without abandoning their own criterion.

What the debate should have been about (and largely became, afterwards) is which criterion the situation demands, and whether the recorded base rate difference reflects the world or reflects policing patterns. The second question is not a statistical one at all.

What this means for your work

Three consequences, and they are practical.

You must choose, and the choice is a value judgement. There is no technically superior answer. There is a decision about which harm matters more in this context, and it belongs to the people accountable for the decision, not to the person writing the code. Your job is to make the trade-off legible so they can make it knowingly.

Write down the choice and its cost. “We equalised false negative rates across groups, which raised the false positive rate for group B from 8% to 13%. We chose this because a missed diagnosis is worse than an unnecessary follow-up.” That sentence is what accountability looks like in this field.

Beware anyone selling a solution. A tool that claims to make your model fair has picked a definition on your behalf. Ask which one. If the answer is not immediate and specific, the tool is doing something you cannot audit.

The escape that is not a trick

The theorem’s condition is different base rates. If the base rates are equal, the conflict disappears.

So the question worth asking is why they differ. Sometimes the difference is real. Often it is an artefact of measurement: the outcome you recorded is not the outcome you care about. Recorded rearrest is not reoffending; it is reoffending filtered through where police patrol. Recorded default is not inability to pay; it is inability to pay filtered through who was offered what terms.

When the measured base rate difference comes substantially from the measurement, the right intervention is upstream: fix the label, or change what you are predicting. That is slower and more disruptive than adjusting a threshold, and it is the only move that addresses the cause rather than distributing the consequence.

Why this is in a foundations course

Because it is the clearest case in machine learning where the mathematics tells you something a technique cannot fix. Most of this course teaches methods. This lesson teaches a limit, and the limit is more durable than any method in the syllabus: it will still hold when every algorithm here has been replaced.