The number your model has to beat before anyone should care

94% accuracy is often a failure

A fraud model reports 94% accuracy and the room applauds. Then someone asks what happens if you predict “not fraud” for every single transaction. The answer is 96%, because 4% of transactions are fraudulent. The model is worse than a single line of code that ignores the data.

This is not a rare embarrassment. It is the default outcome when nobody computes a baseline, and it is the cheapest quality gate in machine learning: one function, five minutes, run before you train anything.

The four baselines worth having

1. The constant. For classification, always predict the most common class. For regression, always predict the mean, or, if the target is skewed, the median, which minimises absolute error.

from sklearn.dummy import DummyClassifier, DummyRegressor
DummyClassifier(strategy="most_frequent").fit(X_tr, y_tr).score(X_te, y_te)
DummyRegressor(strategy="median").fit(X_tr, y_tr)

2. Yesterday. For anything with a time axis, predict that tomorrow equals today. This is the persistence baseline and it is brutal. In demand forecasting, sales prediction and traffic estimation it regularly beats models that took three weeks to build, because most series are dominated by their own recent value.

3. The rule the business already uses. Someone in operations has a rule of thumb: flag any transaction over ₹50,000 from a new account. Implement it, score it on your test set, and you now know what your model is replacing. This is the only baseline your stakeholders truly care about, and it is the one most often skipped.

4. One good feature. Fit a model on the single most obvious predictor and nothing else. If “days since last purchase” alone gets you to 0.78 AUC and your 60-feature gradient boosting model gets 0.80, that is worth knowing before you build a feature pipeline that has to be maintained for three years.

Report the gap, not the score

A score alone is uninterpretable. “RMSE 4,300” means nothing until you know the median baseline scores 6,100; then your model explains about 30% of the gap you had a chance to close, and that is a sentence a non-technical person can act on.

Most honest one-line summary in ML reporting:

Baseline (predict last month) RMSE 6,100. Model RMSE 4,300. Improvement 30%, measured on 14 months of held-out data.

Compare that with “our model achieves an RMSE of 4,300”. Same run, wildly different amount of information.

Baselines catch bugs, not just weak models

The second use of a baseline is diagnostic, and it is the reason experienced people compute one before every project rather than for the report at the end.

• The model does much worse than the constant baseline. Something is broken. Labels shuffled, features and targets misaligned by one row, a transformation fitted on the wrong data.
• **The model does far better than any baseline, immediately, with no tuning.** Almost always leakage. A 0.99 AUC on a problem humans find hard is a bug report, not a result. Look for a column recorded after the outcome: the classic is account_closed_reason in a churn dataset.
• The model exactly matches the baseline. It has learned to predict the constant. Check whether the features reach the model at all; a pipeline that drops every column silently is a real and common failure.

Ceilings matter as much as floors

The baseline is the floor. There is also a ceiling, and it is rarely 100%.

Give two experienced radiologists the same 200 scans and they will disagree on some fraction of them. That disagreement rate is roughly the point past which a model’s extra accuracy stops being measurable, because your labels themselves contain that much noise. The same applies far outside medicine: if two support agents label the same ticket differently one time in six, a model scoring 85% on that task may already be at the ceiling of what the labels can express.

So when you can, measure human agreement on a sample of your own data. Two people, a hundred rows, an afternoon. Between the baseline and the human agreement rate is the entire range in which your work can make a difference, and it is often much narrower than anyone assumed.

What this costs you

About twenty minutes. There is a version of this course that skips this lesson and gets to gradient boosting sooner, and the learner who takes that version will one day present a number to a room that has no idea whether it is good. The baseline is what turns a score into a claim.