The number any forecast has to beat

The comparison that decides everything

Somebody sells you demand forecasting. It predicts next week’s sales within 8%. Is that good?

There is no way to answer without knowing what the stupidest possible method scores. For most small businesses, “same week last year, adjusted for growth” or “last week” lands within 10 to 15% without any modelling at all. An 8% forecast beating a 12% baseline is a real, modest improvement. An 8% forecast against a 7% baseline is a worse method sold with a better number.

So: name the baseline first, in writing, before anything is built or bought.

The three baselines worth computing

They take ten minutes in a spreadsheet, and one of them will be hard to beat.

Last period. Next week equals this week. Surprisingly strong for anything without much seasonality.

Same period last year. Next week equals the same week last year, times your annual growth. Strong for anything seasonal, which is most retail and hospitality.

The recent average. Next week equals the average of the last four weeks. Strong when your demand is noisy but stable.

Compute all three over the last year, measure how far each was wrong, and keep the best. That number is now the bar.

Measuring “wrong” without kidding yourself

Take the absolute difference between forecast and actual, as a percentage of actual, and average it across weeks. That is mean absolute percentage error, and it is the ordinary measure. For a small shop, 10 to 20% is typical and 5% is suspicious.

Two cautions. Averaging signed errors gives you nearly zero, because overs cancel unders: always take the absolute value. And percentage error explodes on near-zero weeks, so if you have closed weeks or very low-volume lines, use the plain absolute error in units instead and compare like with like.

The holdout, and the mistake everyone makes once

You must measure on data the method has never seen. Otherwise you are measuring memory.

For anything with a time dimension, the split is by time and only by time: build on everything up to the end of June, test on July to September. Never shuffle randomly. A random split lets a method learn from August while predicting July, which it will never be able to do in real life, and the resulting score will be excellent and meaningless.

This is a specific form of the leak covered two lessons from here, and it is the version that catches careful people.

What actually drives small-business demand

Before reaching for a model, write down the things you already know move your numbers, because a baseline that includes them often beats a model that does not.

Day of week, which is usually the largest single effect. Pay dates. School terms and holidays. Festivals and their movement: Diwali, Eid and Easter shift against the calendar each year, so “same week last year” quietly misaligns unless you align on the festival rather than the week number. Weather, for anything sold outdoors or eaten cold. Your own promotions, which is the one people forget: a week with a half-price offer is not evidence about ordinary demand and will poison both your history and your forecast if it is left unmarked.

Keep a simple events register: a date, a label, one line of description. It costs a minute a month and it is what makes your history interpretable two years later.

When forecasting is worth it at all

Only when a number changes a decision. How much to order, how many staff to roster, whether to open on a bank holiday.

If the answer is the same whether next week is 800 or 950, the forecast has no value at any accuracy. A surprising number of forecasting projects fail this test and are built anyway, because the question “what would I do differently?” is asked after the work rather than before it.

The position for most readers

A spreadsheet holding last year’s weekly figures, marked up with your events, plus a growth adjustment, is a decent forecast for most small businesses. It costs nothing, everybody in the business can read it, and it beats a bought model often enough that you should make the vendor prove otherwise on your own held-out weeks.

Ask them for that. A vendor who will not run their method on your data and show the comparison against your baseline is selling the 8% rather than the improvement.