Where the rows came from, and who is missing

The assumption underneath every model

Every method in this course assumes your training rows are drawn from the same distribution as the rows you will predict on. When that fails, no amount of modelling repairs it, and the failure does not appear in any validation score, because the validation set is drawn from the same flawed sample as the training set.

This is the one class of problem that a held-out set cannot detect. It is worth sitting with for a moment, because everything else in this course is a variation on “hold data out and measure”.

Four ways the sample goes wrong

Selection bias. Your data records only the cases that reached you. A loan model trained on approved applicants has no rows for anyone rejected, so it learns to predict default among people the previous process approved, and it cannot tell you anything about the people it was designed to help you evaluate. The formal name is the rejection inference problem, it is well known in credit, and it has no clean solution. The partial ones are expensive: approve a random small fraction regardless of score, and accept the losses as the price of knowing.

Survivorship bias. Only the survivors are in the table. Analysing customer behaviour using currently active customers excludes everyone who left, which is precisely the group a churn model needs. Abraham Wald’s wartime example remains the clearest: armour the parts of returning aircraft with no bullet holes, because planes hit there did not return.

Measurement differences between groups. The same quantity measured differently across subsets. A hospital that records blood pressure in a triage queue and another that records it in a quiet room produce different numbers for the same patient. A model trained mostly on one will misread the other.

Feedback loops. The model’s own predictions change the data it is later trained on. Predict high crime in a district, send more officers, record more incidents, retrain, predict higher crime. Nothing about the district changed. This is covered again in module 9 because it is a production problem, but it starts here, in how the next dataset is generated.

Two questions, asked out loud

Before you build anything:

Who is in this data, and who is not?

What had to happen for a row to exist?

The second is the sharper one. For a customer support dataset: someone had a problem, knew the support channel existed, had the time and language to use it, and completed the form. Everyone who gave up before completing it is absent, and they may be the group you most need to understand.

For Addaly’s own data, the same question has an uncomfortable answer: rows exist for people with a working internet connection, a device, and enough literacy to use the site. That is a real limit on what the data can tell us about learners in general, and stating it is better than discovering it later.

What you can actually do

Reweight, when you know the target population’s composition. If your sample is 70% urban and the population is 35%, weight rural rows up. This corrects the marginal distribution and does nothing about groups with zero rows.

Collect deliberately. Oversample small groups so you have enough rows to measure per-group performance, which is different from making the model treat them equally, and is a prerequisite for the fairness work in module 5.

Randomise a slice. In selection-bias settings, the only reliable fix is a small random exploration policy: approve 2% at random, show a random result to 1% of searches. It costs money and it is the only way to get unbiased data about the region your policy currently excludes.

Document the limits. Write down who the model was trained on and who it should not be used for. This is what a model card is, and module 9 builds one.

Distribution shift after deployment

The sample can also be right on day one and wrong by month six. Three named forms, because the distinction guides the fix:

• Covariate shift: the input distribution moves, the relationship holds. Your users get younger; age still predicts what it did. Reweighting can help.
• Label shift: the base rate moves. Fraud rises from 1% to 4%. Recalibration helps; the ranking may still be fine.
• Concept drift: the relationship itself changes. Fraudsters adapt to your model. Nothing helps except new labelled data and retraining.

They are detected differently, they are fixed differently, and calling all three “drift” is how teams end up applying the wrong remedy for a quarter.

The position

There is no statistical test that tells you your sample is unrepresentative of a population you did not sample. You can compare your data against a census, against another source, against what a domain expert expects, and those comparisons are worth doing. But the definitive check does not exist, and the appropriate response is to write down what you know about how the rows came to be, and to say the limits out loud in whatever document accompanies the model.