{"course":{"slug":"ai-for-small-business","title":"AI for a Small Business","url":"https://addaly.com/learn/ai-for-small-business","level":"beginner","track":"work"},"lesson":{"slug":"the-naive-baseline","title":"The number any forecast has to beat","url":"https://addaly.com/learn/ai-for-small-business/the-naive-baseline","position":71,"of":88,"minutes":9,"answer":"A forecast is only worth having if it beats the simplest possible guess on data it has never seen, so you state the naive baseline and the holdout period before you build anything.","keyIdea":"A forecast is only worth having if it beats the simplest possible guess on data it has never seen, so you state the naive baseline and the holdout period before you build anything."},"body":{"format":"markdown","text":"## The comparison that decides everything\n\nSomebody sells you demand forecasting. It predicts next week's sales within 8%. Is that good?\n\nThere is no way to answer without knowing what the stupidest possible method scores. For most small businesses, \"same week last year, adjusted for growth\" or \"last week\" lands within 10 to 15% without any modelling at all. An 8% forecast beating a 12% baseline is a real, modest improvement. An 8% forecast against a 7% baseline is a worse method sold with a better number.\n\nSo: name the baseline first, in writing, before anything is built or bought.\n\n## The three baselines worth computing\n\nThey take ten minutes in a spreadsheet, and one of them will be hard to beat.\n\n**Last period.** Next week equals this week. Surprisingly strong for anything without much seasonality.\n\n**Same period last year.** Next week equals the same week last year, times your annual growth. Strong for anything seasonal, which is most retail and hospitality.\n\n**The recent average.** Next week equals the average of the last four weeks. Strong when your demand is noisy but stable.\n\nCompute all three over the last year, measure how far each was wrong, and keep the best. That number is now the bar.\n\n> Figure: Is eight per cent any good. Bars (average error over the last year (%)): Next week equals this week 13.5; Same week last year, plus growth 11.2; Average of the last four weeks 12.4; The forecasting product being sold to you 8. Eight against eleven is a real but modest improvement; eight against seven is a worse method sold with a better number. Compute the three baselines first, on weeks the method has never seen, and split by date rather than at random.\n\n## Measuring \"wrong\" without kidding yourself\n\nTake the absolute difference between forecast and actual, as a percentage of actual, and average it across weeks. That is mean absolute percentage error, and it is the ordinary measure. For a small shop, 10 to 20% is typical and 5% is suspicious.\n\nTwo cautions. Averaging signed errors gives you nearly zero, because overs cancel unders: always take the absolute value. And percentage error explodes on near-zero weeks, so if you have closed weeks or very low-volume lines, use the plain absolute error in units instead and compare like with like.\n\n## The holdout, and the mistake everyone makes once\n\nYou must measure on data the method has never seen. Otherwise you are measuring memory.\n\nFor anything with a time dimension, the split is by time and only by time: build on everything up to the end of June, test on July to September. Never shuffle randomly. A random split lets a method learn from August while predicting July, which it will never be able to do in real life, and the resulting score will be excellent and meaningless.\n\nThis is a specific form of the leak covered two lessons from here, and it is the version that catches careful people.\n\n## What actually drives small-business demand\n\nBefore reaching for a model, write down the things you already know move your numbers, because a baseline that includes them often beats a model that does not.\n\nDay of week, which is usually the largest single effect. Pay dates. School terms and holidays. Festivals and their movement: Diwali, Eid and Easter shift against the calendar each year, so \"same week last year\" quietly misaligns unless you align on the festival rather than the week number. Weather, for anything sold outdoors or eaten cold. Your own promotions, which is the one people forget: a week with a half-price offer is not evidence about ordinary demand and will poison both your history and your forecast if it is left unmarked.\n\nKeep a simple events register: a date, a label, one line of description. It costs a minute a month and it is what makes your history interpretable two years later.\n\n## When forecasting is worth it at all\n\nOnly when a number changes a decision. How much to order, how many staff to roster, whether to open on a bank holiday.\n\nIf the answer is the same whether next week is 800 or 950, the forecast has no value at any accuracy. A surprising number of forecasting projects fail this test and are built anyway, because the question \"what would I do differently?\" is asked after the work rather than before it.\n\n## The position for most readers\n\nA spreadsheet holding last year's weekly figures, marked up with your events, plus a growth adjustment, is a decent forecast for most small businesses. It costs nothing, everybody in the business can read it, and it beats a bought model often enough that you should make the vendor prove otherwise on your own held-out weeks.\n\nAsk them for that. A vendor who will not run their method on your data and show the comparison against your baseline is selling the 8% rather than the improvement."},"check":{"question":"A bakery's supplier offers a forecasting add-on and demonstrates 9% error on the bakery's own last twelve months. Why is this not yet evidence the tool is worth buying?","options":[{"text":"Because twelve months is too short a history for any forecasting method to learn seasonal patterns reliably, so the result will not hold as more data accumulates.","correct":false,"tempting":"Twelve months is thin for seasonality, but the fault in the demonstration is more basic than the amount of history."},{"text":"Because mean error hides the weeks that matter, and a method can score well on average while being badly wrong on the peak weeks where ordering decisions actually bite.","correct":false,"tempting":"A real weakness of averaged error worth checking, and it does not address whether 9% is an improvement at all."},{"text":"Because nobody has computed what \"same week last year\" scores on the same twelve months, so there is no way to tell whether 9% is better than free.","correct":true},{"text":"Because the demonstration was run by the supplier rather than the bakery, and the bakery cannot verify that the figures came from its own export rather than a comparable one.","correct":false,"tempting":"Reasonable commercial caution about the provenance of a demo, but even a scrupulously honest 9% means nothing without a baseline."}]},"prev":{"slug":"the-analysis-that-shows-its-working","title":"Make it compute, do not let it estimate","url":"https://addaly.com/learn/ai-for-small-business/the-analysis-that-shows-its-working"},"next":{"slug":"seasonality-and-the-events-register","title":"Patterns you can name, and the promotion that poisons the history","url":"https://addaly.com/learn/ai-for-small-business/seasonality-and-the-events-register"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The number any forecast has to beat', in AI for a Small Business. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-small-business/the-naive-baseline (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The number any forecast has to beat","courseTitle":"AI for a Small Business","module":{"index":8,"slug":"your-own-numbers","title":"Learning from your own numbers"},"answer":null,"keyIdea":"A forecast is only worth having if it beats the simplest possible guess on data it has never seen, so you state the naive baseline and the holdout period before you build anything.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"average error over the last year (%)","title":"Is eight per cent any good","series":[{"label":"Next week equals this week","value":13.5},{"label":"Same week last year, plus growth","value":11.2},{"label":"Average of the last four weeks","value":12.4},{"label":"The forecasting product being sold to you","value":8,"highlight":true}],"caption":"Eight against eleven is a real but modest improvement; eight against seven is a worse method sold with a better number. Compute the three baselines first, on weeks the method has never seen, and split by date rather than at random."},"alt":"Figure: Is eight per cent any good. Bars (average error over the last year (%)): Next week equals this week 13.5; Same week last year, plus growth 11.2; Average of the last four weeks 12.4; The forecasting product being sold to you 8. Eight against eleven is a real but modest improvement; eight against seven is a worse method sold with a better number. Compute the three baselines first, on weeks the method has never seen, and split by date rather than at random."}},"blocks":[{"t":"h2","id":"the-comparison-that-decides-everything","runs":[{"text":"The comparison that decides everything"}]},{"t":"p","runs":[{"text":"Somebody sells you demand forecasting. It predicts next week’s sales within 8%. Is that good?"}]},{"t":"p","runs":[{"text":"There is no way to answer without knowing what the stupidest possible method scores. For most small businesses, “same week last year, adjusted for growth” or “last week” lands within 10 to 15% without any modelling at all. An 8% forecast beating a 12% baseline is a real, modest improvement. An 8% forecast against a 7% baseline is a worse method sold with a better number."}]},{"t":"p","runs":[{"text":"So: name the baseline first, in writing, before anything is built or bought."}]},{"t":"h2","id":"the-three-baselines-worth-computing","runs":[{"text":"The three baselines worth computing"}]},{"t":"p","runs":[{"text":"They take ten minutes in a spreadsheet, and one of them will be hard to beat."}]},{"t":"p","runs":[{"text":"Last period.","b":true},{"text":" Next week equals this week. Surprisingly strong for anything without much seasonality."}]},{"t":"p","runs":[{"text":"Same period last year.","b":true},{"text":" Next week equals the same week last year, times your annual growth. Strong for anything seasonal, which is most retail and hospitality."}]},{"t":"p","runs":[{"text":"The recent average.","b":true},{"text":" Next week equals the average of the last four weeks. Strong when your demand is noisy but stable."}]},{"t":"p","runs":[{"text":"Compute all three over the last year, measure how far each was wrong, and keep the best. That number is now the bar."}]},{"t":"h2","id":"measuring-wrong-without-kidding-yourself","runs":[{"text":"Measuring “wrong” without kidding yourself"}]},{"t":"p","runs":[{"text":"Take the absolute difference between forecast and actual, as a percentage of actual, and average it across weeks. That is mean absolute percentage error, and it is the ordinary measure. For a small shop, 10 to 20% is typical and 5% is suspicious."}]},{"t":"p","runs":[{"text":"Two cautions. Averaging signed errors gives you nearly zero, because overs cancel unders: always take the absolute value. And percentage error explodes on near-zero weeks, so if you have closed weeks or very low-volume lines, use the plain absolute error in units instead and compare like with like."}]},{"t":"h2","id":"the-holdout-and-the-mistake-everyone-makes-once","runs":[{"text":"The holdout, and the mistake everyone makes once"}]},{"t":"p","runs":[{"text":"You must measure on data the method has never seen. Otherwise you are measuring memory."}]},{"t":"p","runs":[{"text":"For anything with a time dimension, the split is by time and only by time: build on everything up to the end of June, test on July to September. Never shuffle randomly. A random split lets a method learn from August while predicting July, which it will never be able to do in real life, and the resulting score will be excellent and meaningless."}]},{"t":"p","runs":[{"text":"This is a specific form of the leak covered two lessons from here, and it is the version that catches careful people."}]},{"t":"h2","id":"what-actually-drives-small-business-demand","runs":[{"text":"What actually drives small-business demand"}]},{"t":"p","runs":[{"text":"Before reaching for a model, write down the things you already know move your numbers, because a baseline that includes them often beats a model that does not."}]},{"t":"p","runs":[{"text":"Day of week, which is usually the largest single effect. Pay dates. School terms and holidays. Festivals and their movement: Diwali, Eid and Easter shift against the calendar each year, so “same week last year” quietly misaligns unless you align on the festival rather than the week number. Weather, for anything sold outdoors or eaten cold. Your own promotions, which is the one people forget: a week with a half-price offer is not evidence about ordinary demand and will poison both your history and your forecast if it is left unmarked."}]},{"t":"p","runs":[{"text":"Keep a simple events register: a date, a label, one line of description. It costs a minute a month and it is what makes your history interpretable two years later."}]},{"t":"h2","id":"when-forecasting-is-worth-it-at-all","runs":[{"text":"When forecasting is worth it at all"}]},{"t":"p","runs":[{"text":"Only when a number changes a decision. How much to order, how many staff to roster, whether to open on a bank holiday."}]},{"t":"p","runs":[{"text":"If the answer is the same whether next week is 800 or 950, the forecast has no value at any accuracy. A surprising number of forecasting projects fail this test and are built anyway, because the question “what would I do differently?” is asked after the work rather than before it."}]},{"t":"h2","id":"the-position-for-most-readers","runs":[{"text":"The position for most readers"}]},{"t":"p","runs":[{"text":"A spreadsheet holding last year’s weekly figures, marked up with your events, plus a growth adjustment, is a decent forecast for most small businesses. It costs nothing, everybody in the business can read it, and it beats a bought model often enough that you should make the vendor prove otherwise on your own held-out weeks."}]},{"t":"p","runs":[{"text":"Ask them for that. A vendor who will not run their method on your data and show the comparison against your baseline is selling the 8% rather than the improvement."}]}],"check":{"question":[{"text":"A bakery’s supplier offers a forecasting add-on and demonstrates 9% error on the bakery’s own last twelve months. Why is this not yet evidence the tool is worth buying?"}],"options":[{"text":[{"text":"Because twelve months is too short a history for any forecasting method to learn seasonal patterns reliably, so the result will not hold as more data accumulates."}],"correct":false,"tempting":[{"text":"Twelve months is thin for seasonality, but the fault in the demonstration is more basic than the amount of history."}]},{"text":[{"text":"Because mean error hides the weeks that matter, and a method can score well on average while being badly wrong on the peak weeks where ordering decisions actually bite."}],"correct":false,"tempting":[{"text":"A real weakness of averaged error worth checking, and it does not address whether 9% is an improvement at all."}]},{"text":[{"text":"Because nobody has computed what “same week last year” scores on the same twelve months, so there is no way to tell whether 9% is better than free."}],"correct":true,"tempting":null},{"text":[{"text":"Because the demonstration was run by the supplier rather than the bakery, and the bakery cannot verify that the figures came from its own export rather than a comparable one."}],"correct":false,"tempting":[{"text":"Reasonable commercial caution about the provenance of a demo, but even a scrupulously honest 9% means nothing without a baseline."}]}],"order":[1,3,0,2],"why":null},"prev":{"slug":"the-analysis-that-shows-its-working","title":"Make it compute, do not let it estimate","minutes":9},"next":{"slug":"seasonality-and-the-events-register","title":"Patterns you can name, and the promotion that poisons the history","minutes":8}}}