{"course":{"slug":"machine-learning-foundations","title":"Machine Learning, Foundations","url":"https://addaly.com/learn/machine-learning-foundations","level":"intermediate","track":"start"},"lesson":{"slug":"the-baseline-you-must-beat","title":"The number your model has to beat before anyone should care","url":"https://addaly.com/learn/machine-learning-foundations/the-baseline-you-must-beat","position":5,"of":92,"minutes":8,"answer":"Compute the constant, persistence and existing-rule baselines before training, and report the gap between baseline and model rather than the model's score alone.","keyIdea":"Compute the constant, persistence and existing-rule baselines before training, and report the gap between baseline and model rather than the model's score alone."},"body":{"format":"markdown","text":"## 94% accuracy is often a failure\n\nA fraud model reports 94% accuracy and the room applauds. Then someone asks what happens if you predict \"not fraud\" for every single transaction. The answer is 96%, because 4% of transactions are fraudulent. The model is worse than a single line of code that ignores the data.\n\nThis is not a rare embarrassment. It is the default outcome when nobody computes a baseline, and it is the cheapest quality gate in machine learning: one function, five minutes, run before you train anything.\n\n## The four baselines worth having\n\n**1. The constant.** For classification, always predict the most common class. For regression, always predict the mean, or, if the target is skewed, the median, which minimises absolute error.\n\n```python\nfrom sklearn.dummy import DummyClassifier, DummyRegressor\nDummyClassifier(strategy=\"most_frequent\").fit(X_tr, y_tr).score(X_te, y_te)\nDummyRegressor(strategy=\"median\").fit(X_tr, y_tr)\n```\n\n**2. Yesterday.** For anything with a time axis, predict that tomorrow equals today. This is the *persistence* baseline and it is brutal. In demand forecasting, sales prediction and traffic estimation it regularly beats models that took three weeks to build, because most series are dominated by their own recent value.\n\n**3. The rule the business already uses.** Someone in operations has a rule of thumb: flag any transaction over ₹50,000 from a new account. Implement it, score it on your test set, and you now know what your model is replacing. This is the only baseline your stakeholders truly care about, and it is the one most often skipped.\n\n**4. One good feature.** Fit a model on the single most obvious predictor and nothing else. If \"days since last purchase\" alone gets you to 0.78 AUC and your 60-feature gradient boosting model gets 0.80, that is worth knowing before you build a feature pipeline that has to be maintained for three years.\n\n## Report the gap, not the score\n\nA score alone is uninterpretable. \"RMSE 4,300\" means nothing until you know the median baseline scores 6,100; then your model explains about 30% of the gap you had a chance to close, and that is a sentence a non-technical person can act on.\n\n> Figure: RMSE 4,300 of what. Bars (RMSE on the same 14 months of held-out demand): Constant: predict the median 6900; Persistence: predict last month 6100; One good feature alone 4900; Full model, 60 features 4300. The model beats the persistence baseline by 30 per cent, measured on 14 months of held-out data. Reported alone, 4,300 is a number. Reported beside the baselines it is a claim someone can act on; and the single-feature model shows that most of the gap was closed by one column.\n\nMost honest one-line summary in ML reporting:\n\n> Baseline (predict last month) RMSE 6,100. Model RMSE 4,300. Improvement 30%, measured on 14 months of held-out data.\n\nCompare that with \"our model achieves an RMSE of 4,300\". Same run, wildly different amount of information.\n\n## Baselines catch bugs, not just weak models\n\nThe second use of a baseline is diagnostic, and it is the reason experienced people compute one before every project rather than for the report at the end.\n\n- **The model does much worse than the constant baseline.** Something is broken. Labels shuffled, features and targets misaligned by one row, a transformation fitted on the wrong data.\n- **The model does *far* better than any baseline, immediately, with no tuning.** Almost always leakage. A 0.99 AUC on a problem humans find hard is a bug report, not a result. Look for a column recorded after the outcome: the classic is `account_closed_reason` in a churn dataset.\n- **The model exactly matches the baseline.** It has learned to predict the constant. Check whether the features reach the model at all; a pipeline that drops every column silently is a real and common failure.\n\n## Ceilings matter as much as floors\n\nThe baseline is the floor. There is also a ceiling, and it is rarely 100%.\n\nGive two experienced radiologists the same 200 scans and they will disagree on some fraction of them. That disagreement rate is roughly the point past which a model's extra accuracy stops being measurable, because your labels themselves contain that much noise. The same applies far outside medicine: if two support agents label the same ticket differently one time in six, a model scoring 85% on that task may already be at the ceiling of what the labels can express.\n\nSo when you can, measure human agreement on a sample of your own data. Two people, a hundred rows, an afternoon. Between the baseline and the human agreement rate is the entire range in which your work can make a difference, and it is often much narrower than anyone assumed.\n\n## What this costs you\n\nAbout twenty minutes. There is a version of this course that skips this lesson and gets to gradient boosting sooner, and the learner who takes that version will one day present a number to a room that has no idea whether it is good. The baseline is what turns a score into a claim."},"check":{"question":"A team builds a model to predict next week's electricity demand and reports RMSE of 210 MW, calling it a strong result. What single check would most change how that number should be read?","options":[{"text":"Compute a confidence interval on the RMSE using the bootstrap.","correct":false,"tempting":"Quantifies uncertainty in the number, which is useful, but tells you nothing about whether 210 is good or bad in the first place."},{"text":"Retrain with cross-validation instead of a single split.","correct":false,"tempting":"Improves the reliability of the estimate but leaves the same interpretive vacuum: a more trustworthy 210 is still meaningless without a reference point."},{"text":"Check whether demand is normally distributed so that RMSE is an appropriate metric.","correct":false,"tempting":"Applies a distributional worry that does not bear on interpretation; RMSE is well defined regardless and the question is what to compare it against."},{"text":"Score a persistence baseline that predicts next week equals this week, on exactly the same held-out period.","correct":true}]},"prev":{"slug":"what-a-loss-function-encodes","title":"The loss is where you state what a mistake costs","url":"https://addaly.com/learn/machine-learning-foundations/what-a-loss-function-encodes"},"next":{"slug":"prediction-is-not-explanation","title":"Why a model that predicts well cannot tell you what to change","url":"https://addaly.com/learn/machine-learning-foundations/prediction-is-not-explanation"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The number your model has to beat before anyone should care', in Machine Learning, Foundations. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/machine-learning-foundations/the-baseline-you-must-beat (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The number your model has to beat before anyone should care","courseTitle":"Machine Learning, Foundations","module":{"index":1,"slug":"what-learning-is","title":"What learning is, and what it is not"},"answer":null,"keyIdea":"Compute the constant, persistence and existing-rule baselines before training, and report the gap between baseline and model rather than the model’s score alone.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"RMSE on the same 14 months of held-out demand","title":"RMSE 4,300 of what","series":[{"label":"Constant: predict the median","value":6900},{"label":"Persistence: predict last month","value":6100},{"label":"One good feature alone","value":4900},{"label":"Full model, 60 features","value":4300,"highlight":true}],"caption":"The model beats the persistence baseline by 30 per cent, measured on 14 months of held-out data. Reported alone, 4,300 is a number. Reported beside the baselines it is a claim someone can act on; and the single-feature model shows that most of the gap was closed by one column."},"alt":"Figure: RMSE 4,300 of what. Bars (RMSE on the same 14 months of held-out demand): Constant: predict the median 6900; Persistence: predict last month 6100; One good feature alone 4900; Full model, 60 features 4300. The model beats the persistence baseline by 30 per cent, measured on 14 months of held-out data. Reported alone, 4,300 is a number. Reported beside the baselines it is a claim someone can act on; and the single-feature model shows that most of the gap was closed by one column."}},"blocks":[{"t":"h2","id":"94-accuracy-is-often-a-failure","runs":[{"text":"94% accuracy is often a failure"}]},{"t":"p","runs":[{"text":"A fraud model reports 94% accuracy and the room applauds. Then someone asks what happens if you predict “not fraud” for every single transaction. The answer is 96%, because 4% of transactions are fraudulent. The model is worse than a single line of code that ignores the data."}]},{"t":"p","runs":[{"text":"This is not a rare embarrassment. It is the default outcome when nobody computes a baseline, and it is the cheapest quality gate in machine learning: one function, five minutes, run before you train anything."}]},{"t":"h2","id":"the-four-baselines-worth-having","runs":[{"text":"The four baselines worth having"}]},{"t":"p","runs":[{"text":"1. The constant.","b":true},{"text":" For classification, always predict the most common class. For regression, always predict the mean, or, if the target is skewed, the median, which minimises absolute error."}]},{"t":"code","lang":"python","label":"Python","text":"from sklearn.dummy import DummyClassifier, DummyRegressor\nDummyClassifier(strategy=\"most_frequent\").fit(X_tr, y_tr).score(X_te, y_te)\nDummyRegressor(strategy=\"median\").fit(X_tr, y_tr)"},{"t":"p","runs":[{"text":"2. Yesterday.","b":true},{"text":" For anything with a time axis, predict that tomorrow equals today. This is the "},{"text":"persistence","i":true},{"text":" baseline and it is brutal. In demand forecasting, sales prediction and traffic estimation it regularly beats models that took three weeks to build, because most series are dominated by their own recent value."}]},{"t":"p","runs":[{"text":"3. The rule the business already uses.","b":true},{"text":" Someone in operations has a rule of thumb: flag any transaction over ₹50,000 from a new account. Implement it, score it on your test set, and you now know what your model is replacing. This is the only baseline your stakeholders truly care about, and it is the one most often skipped."}]},{"t":"p","runs":[{"text":"4. One good feature.","b":true},{"text":" Fit a model on the single most obvious predictor and nothing else. If “days since last purchase” alone gets you to 0.78 AUC and your 60-feature gradient boosting model gets 0.80, that is worth knowing before you build a feature pipeline that has to be maintained for three years."}]},{"t":"h2","id":"report-the-gap-not-the-score","runs":[{"text":"Report the gap, not the score"}]},{"t":"p","runs":[{"text":"A score alone is uninterpretable. “RMSE 4,300” means nothing until you know the median baseline scores 6,100; then your model explains about 30% of the gap you had a chance to close, and that is a sentence a non-technical person can act on."}]},{"t":"p","runs":[{"text":"Most honest one-line summary in ML reporting:"}]},{"t":"quote","runs":[{"text":"Baseline (predict last month) RMSE 6,100. Model RMSE 4,300. Improvement 30%, measured on 14 months of held-out data."}]},{"t":"p","runs":[{"text":"Compare that with “our model achieves an RMSE of 4,300”. Same run, wildly different amount of information."}]},{"t":"h2","id":"baselines-catch-bugs-not-just-weak-models","runs":[{"text":"Baselines catch bugs, not just weak models"}]},{"t":"p","runs":[{"text":"The second use of a baseline is diagnostic, and it is the reason experienced people compute one before every project rather than for the report at the end."}]},{"t":"ul","items":[[{"text":"The model does much worse than the constant baseline.","b":true},{"text":" Something is broken. Labels shuffled, features and targets misaligned by one row, a transformation fitted on the wrong data."}],[{"text":"**The model does "},{"text":"far","i":true},{"text":" better than any baseline, immediately, with no tuning.** Almost always leakage. A 0.99 AUC on a problem humans find hard is a bug report, not a result. Look for a column recorded after the outcome: the classic is "},{"text":"account_closed_reason","code":true},{"text":" in a churn dataset."}],[{"text":"The model exactly matches the baseline.","b":true},{"text":" It has learned to predict the constant. Check whether the features reach the model at all; a pipeline that drops every column silently is a real and common failure."}]]},{"t":"h2","id":"ceilings-matter-as-much-as-floors","runs":[{"text":"Ceilings matter as much as floors"}]},{"t":"p","runs":[{"text":"The baseline is the floor. There is also a ceiling, and it is rarely 100%."}]},{"t":"p","runs":[{"text":"Give two experienced radiologists the same 200 scans and they will disagree on some fraction of them. That disagreement rate is roughly the point past which a model’s extra accuracy stops being measurable, because your labels themselves contain that much noise. The same applies far outside medicine: if two support agents label the same ticket differently one time in six, a model scoring 85% on that task may already be at the ceiling of what the labels can express."}]},{"t":"p","runs":[{"text":"So when you can, measure human agreement on a sample of your own data. Two people, a hundred rows, an afternoon. Between the baseline and the human agreement rate is the entire range in which your work can make a difference, and it is often much narrower than anyone assumed."}]},{"t":"h2","id":"what-this-costs-you","runs":[{"text":"What this costs you"}]},{"t":"p","runs":[{"text":"About twenty minutes. There is a version of this course that skips this lesson and gets to gradient boosting sooner, and the learner who takes that version will one day present a number to a room that has no idea whether it is good. The baseline is what turns a score into a claim."}]}],"check":{"question":[{"text":"A team builds a model to predict next week’s electricity demand and reports RMSE of 210 MW, calling it a strong result. What single check would most change how that number should be read?"}],"options":[{"text":[{"text":"Compute a confidence interval on the RMSE using the bootstrap."}],"correct":false,"tempting":[{"text":"Quantifies uncertainty in the number, which is useful, but tells you nothing about whether 210 is good or bad in the first place."}]},{"text":[{"text":"Retrain with cross-validation instead of a single split."}],"correct":false,"tempting":[{"text":"Improves the reliability of the estimate but leaves the same interpretive vacuum: a more trustworthy 210 is still meaningless without a reference point."}]},{"text":[{"text":"Check whether demand is normally distributed so that RMSE is an appropriate metric."}],"correct":false,"tempting":[{"text":"Applies a distributional worry that does not bear on interpretation; RMSE is well defined regardless and the question is what to compare it against."}]},{"text":[{"text":"Score a persistence baseline that predicts next week equals this week, on exactly the same held-out period."}],"correct":true,"tempting":null}],"order":[1,3,2,0],"why":null},"prev":{"slug":"what-a-loss-function-encodes","title":"The loss is where you state what a mistake costs","minutes":9},"next":{"slug":"prediction-is-not-explanation","title":"Why a model that predicts well cannot tell you what to change","minutes":9}}}