There was a “let’s make a deal” brain teaser that appeared in the “Ask Marilyn” column in Parade magazine several years ago. Readers were asked to suppose they were a contestant on the television game show “Let’s Make A Deal.” The host tells you that there is a car behind one of three large doors on the stage. The other two doors have either a small gift or a goat behind them. You get to choose a door, and keep whatever is behind it.
After you make your choice, the host reveals what is behind one of the two doors you didn’t choose. It isn’t the car. The car, therefore, is behind the door you chose or the other unopened door. The host gives you the option of switching your choice. Should you switch?
Most people, including me, might assume that your odds of winning the car will be the same whether you stick with your original selection or switch. The answer, which is totally non-intuitive, is that you should always switch. Your odds of winning will increase significantly.
I will explain the answer to this riddle later, but first I want to explain how this applies to Demand Management. This game show scenario is the best-known example of the Bayesian approach to probability and forecasting. Bayes asked the question: Does your expectation of a future event change because of what you know about prior events? He mathematically proved that it does.
He wrote about prior and posterior probability and his approach to probability is known as the Bayesian Approach. Basically, the Bayesian approach involves creating a model composed of components that are given different weights. Posterior probability leads to changes in the weightings of certain components of the model. Prior probability is the situation the contestant was first confronted with. Posterior probability occurred when the host opened up a door and showed that the car was not behind it.
Most demand management solutions could be loosely described as “learning” applications. There are multiple statistical forecasting algorithms, from the very simple to the complex. No single algorithm is good or bad- they all have a use and are good for certain history patterns. Most demand management software vendors use at least 15 algorithms. Demand is a moving target. Over time, demand patterns will change and the forecasting algorithms need to change as well.
Several years ago, demand applications became self-grading. They offered alerts that, in effect, said “I’m not performing well.” In the first generation of automated forecasting, the application created automated alerts when the forecast error passed a preset threshold. Users then needed to manually figure out what algorithm would work better for a certain SKU or SKU by location.
In the second generation, the Demand Management applications had automated alerts, but they also automated the selection of the new algorithm. This “best fit automation” combines your new forecasting data with the historical data points and runs it against all the algorithms a vendor offers (usually at least 10) and then measures the forecast error from each and picks the algorithm with the lowest forecast error. However, this automation had unintended consequences. This functionality proved to be too “nervous” in many instances-i.e., the solution could flip-flop the best algorithm for a particular SKU many times a year. As a result, production and procurement personnel are faced with unusually large surges or troughs in what is required of them.
While most Demand Management suppliers continue to use best fit, there is now a third-generation solution of forecasting automation.
There are different ways to approach the “nervousness” problem, some less automated than others. Infor, for example, approaches it through the use of a Bayesian model, which requires less manual intervention. Their model looks at seasonality, level, trend, and variance as completely separate components of the forecast. Each of these components has a weight, or in Bayesian terms, a probability associated with them. As new data comes in, the solution does not completely throw out one algorithm and replace it with another. Instead, it changes the weights associated with the algorithms that are forecasting the various components of the demand plan. This makes the solution less “noisy” but still allows it to automate the forecasting process. How much weight is given to new data points can be preset, which affects the “nervousness” of the solution. Infor claims this technique provides the best forecast results for their customers, with some customers even reporting that forecast accuracy straight from the system is more accurate than when their users have intervened through collaborative efforts.
When it comes to Demand Management, the way we think and talk about these applications has changed substantially over the last five years. We went through a period where suppliers bragged that their algorithms and forecasting models were better than their competitors. In time, we came to the realization that the best statistical forecast was not necessarily the best demand plan, nor does the best demand plan lead to the best results if it does not include inputs from key collaborative stakeholders. This led to a focus on “collaborative demand management”.
However, I am beginning to wonder whether the pendulum has swung too far. Support for collaborative processes, like Sales and Operations Planning, still matters. But forecast accuracy and process automation are still important too.
So what is the solution to the brain teaser? To increase the probability that you’ll get the car, you should switch doors. This seems counter-intuitive because it seems that no matter what the host has revealed, you are left with two doors and no way to know which one has the car. The probability would seem to be 50 percent for either. It isn’t. If you stick with your original choice, prior probability applies, and your odds are one in three. If you switch, posterior probability applies, and your odds are one in two.
Think of it this way: What if there were a thousand doors instead of three? And what if the host eliminated 900 doors instead of one? When you chose your door, you had a one in a thousand chance of being right. The remaining 99 doors, other than the one you picked, have a probability of 99 in 1000 of being the best choice, about 10 percent. A 10 percent chance is much better than a one in a thousand chance