Demand planning is a good application for machine learning because the measure of success – the forecast accuracy – is clear. To learn, an application needs a clear measure of success. Having a clear measure of success sounds easy.
But often, defining success in not easy. Consider a situation where a manufacturer learns of a shortage of a key component. Customers have already been promised products that depend upon that key production input. The supply chain planning engine needs to be rerun to generate a new supply plan.
But in this case the objective can become much more subjective and hard to define:
- Which customers get the full order on time in full?
- Should he company take a margin hit by expediting a shipment or risk a dissatisfied customer’s future business?
- How far out do we push the promise date for a customer?
- Which customers do we short and by how much?
It turns out many problems are hard to define precisely enough to allow for machine learning.
Some suppliers of supply chain planning are looking to “solve” this problem by using pattern recognition to see how planners resolved similar problems in the past and then suggest a similar resolution the next time the problem arises. The idea is that the solution will suggest a resolution, but people make the choices. But this type of solution may be difficult to achieve. The more at bats a system has, the faster it gets smarter. If you are doing daily demand plans, and every day you have new data on how well yesterday’s forecast did, you have a system that can improve quickly.
In the replanning problem described above, significant component shortages may only occur a couple times a year. And the resolution that was used in June may be completely different than the one in December. It may take hundreds, but more likely, thousands of at bats before the machine can begin to unravel why a planner did one thing in one situation and a different thing in the next.
Measures of Success Need to Be Clear, but Don’t Have to be Perfect
As previously mentioned, machine learning is used to improve forecasts. A demand forecast is made, a machine learning engine ingests data on how accurate that forecast was, and then the machine autonomously applies better math to improve the next forecast.
But ideally a forecast is based on demand, not sales. In the consumer goods supply chain, that means the forecast needs to understand both the sales and the lost sales. Lost sales are the sales that could have been achieved if inventory had been in stock on the store shelf. The goal is to stock to true demand – or as close to it as we can get – so the retailer doesn’t miss out on sales.
Some suppliers – Manhattan Associates is one – have a solution for calculating lost sales. But the lost sales calculation is made based on a store forecast. The store forecast tells you what the sales should have been; if the sales don’t occur, store systems check to make sure the inventory was available to complete the sale. If the inventory was not available, it was a lost sale.
But this is circular reasoning. A demand forecast is made, and then to check the accuracy of that forecast one must rely, in part, on that same forecast.
Actually, the problem is even worse than that (see The Peril of Circular Reasoning in Machine Learning and Forecasting).
In short, the objective measure of success – which is critical to improving forecasts – is not as true a measurement as we might like in demand planning. But in this case, even a less than perfect objective measure still leads to improved predictions. In short, while a clear measure of success is needed for machine learning, these measures don’t have to be perfect. If their use improves the predictive power of an application, that is what really counts.
In conclusion, in working on the newly published Supply Chain Planning market study, I talked to several suppliers about their use, or plans to use, machine learning to improve supply chain planning. I’d like to thank everybody that talked to me.
Leave a Reply