Demand forecasting is one of the older applications of machine learning and one of the most frequently disappointing. The model is rarely the limitation. The limitation is that the history contains events nobody recorded, and a model trained on that history learns the consequences without ever seeing the causes.
AI demand forecasting works when the data is honest about what happened. It fails quietly when it is not, and the failure looks like a confident number.
Your history probably lies to you
Sales data records what you sold, which is not what customers wanted. Where you were out of stock, demand looks like zero. Where you ran a promotion, demand looks structurally higher. Where a competitor closed for six months, your baseline shifted for reasons no column captures.
A model trained on that will forecast low for products that stock out, which causes them to stock out again. That feedback loop is the single most common failure in retail forecasting and it is invisible unless somebody goes looking for it.
What you need before starting
- At least two full seasonal cycles, so annual patterns can be distinguished from noise.
- A record of stock-outs, so unmet demand can be separated from absent demand.
- Promotion and price history, since these explain a large share of variation.
- Any known one-off events: a closure, a supply failure, an unusual bulk order.
The last two are almost never in the data warehouse and almost always in somebody's memory. Extracting them is unglamorous and it is usually where the accuracy comes from.
The biggest accuracy gain on most forecasting projects comes from a spreadsheet of past promotions somebody kept privately. Not from changing the model.
Ethan Caldwell, CTO & Co-Founder, Engineered With AI
Backtesting flatters you if you let it
Evaluating a forecast on historical data is essential and easy to do in a way that produces meaningless results. Randomly splitting time series data lets the model see the future, and the resulting accuracy is fiction.
Split by time, train on the earlier period and test on the later, and repeat the split at several points. A model that performs well in one window and poorly in another is telling you something important about stability that a single average conceals.
Compare against the boring baseline
Always test against the simplest possible forecast: last period repeated, or the same week last year. A surprising number of sophisticated models fail to beat that, and knowing so early saves months. It also gives everyone a shared reference point for whether an improvement is worth the complexity.
Forecast a range, not a number
A single number invites false confidence and gives the planner nothing to work with. A range with a stated confidence lets them make an actual decision, because the cost of being short and the cost of being long are rarely equal.
Where holding extra stock is cheap and stocking out loses a customer, the right decision sits at the high end of the range. Where stock is perishable, the reverse. The forecast should inform that judgement rather than replace it.
Some things are not forecastable
A new product with no history, a genuinely novel event, an item selling two units a month: no model resolves these, and pretending otherwise wastes effort and credibility.
Segment the catalogue by how forecastable each part is and apply effort accordingly. High-volume, stable items justify sophistication. The long tail is usually better served by a simple rule and a safety buffer, and saying so plainly is what earns trust in the parts that do work.
Measure it where the decision happens
Forecast accuracy is easy to measure and easy to optimise in ways that help nobody. The business outcomes are stock-outs avoided, holding cost, and waste, and those are what the project should be judged on.
Track how often the forecast was actually used, too. A forecast that planners override every week is either wrong or untrusted, and the fix differs completely depending on which. More on measuring this kind of work is in how to measure automation ROI.
Forecasts nobody trusts?
We will look at your history and tell you honestly what is forecastable and what is not.