demand sensing Archives

Machine Learning in the Digital Supply Chain isn’t New

Demand Signal Repositories | By Steve Banker • 10/23/2017

Machine learning has become hot this year and suppliers are investing research and development into using machine learning in the supply chain. But in a conversation with Michael Farlekas and John Lash – the CEO and VP of Product Marketing at E2open respectively – I was reminded of the fact that machine learning in the supply chain is not a new technology. Last year E2open acquired Terra Technology. Terra has been using machine learning to power their demand management and demand sensing applications since 2004. Their customers include Procter & Gamble, Unilever, AkzoNobel, and several other global, multinational companies.

For machine learning to work well, it needs to be a big data application. In this case in addition to doing forecasting based on historical sales, companies leverage other data sets such as their retail customer’s point of sale, recent shipments of products from their warehouses to their stores, the retailer’s orders, syndicated data, and store inventory. Many of these data sets are accessed daily, or even several times a day, so the dynamic nature of demand is captured to a much higher degree than traditional forecasting techniques.

E2open also acquired Orchestro last year. This is a demand signal repository solution which harmonizes retail Point of Sale (POS), syndicated, internal ERP, and 3rd party data into a common view of demand. In other words, the Orchestro solution makes it much easier to access demand sensing retail data sets.

Machine Learning for Demand Management

Machine learning for demand management works this way. The engine is making many forecasts simultaneously in different planning horizons. So, there can be a forecast for demand for liquid detergent in a 100-ounce container to the Walmart store in Tuscaloosa tomorrow, one week out, and one month out. There can also be a forecast for how much of that detergent will be needed at the distribution center in Cullman, Alabama tomorrow, one week, and one month from now. Other forecasts are being done for other big retail customers and channels.

Machine Learning is Used for Demand Sensing in the E2open Solution

For the forecast of what is needed in the Tuscaloosa store tomorrow, it may be that the data source with the highest influence factor is point of sale and the algorithm weights it accordingly. For a forecast of how much of that detergent will be needed in the Cullman warehouse in a month, a different algorithm is used that weights the data sets such that the statistical forecast is most important, recent shipments the second most important; customer orders third, and so forth.

Then every day, all of these forecasts at all locations and all time horizons are done again.

When this was first explained to me a few years ago, my reaction was “even if you can make all these forecasts, why would you? How could you operationalize this?”

The answer is that different forecasts are used in different time horizons by different groups in the company. So, if a factory has a frozen two-week production schedule, and the new schedule starts on October 16th, the factory planner will use the aggregated daily forecasts of the actual 14 days from October 16th through the 30th to create the forecast for demand over this specific time period and based on the projected inventory, determine what products they need to produce for the next two weeks. All other forecasts are ignored until the next production run needs to begin, when they once again use the most recent forecast pertinent to their needs.

But other departments use different forecasts in different time buckets. Thus, on that same Day 0 – October 16th – the transportation department might find it very helpful to know projected shipments to all the distribution centers on October 19th to create their private fleet plan for the 19th. In this case, there is a deployment lead time of 3 days so the daily forecasts between Oct 16th and the 19th are aggregated to determine how much inventory will be at the DCs on Oct 19th and how much will need to be shipped on the 19th to maintain service commitments.

It should also be pointed out, that the machine learning engine does not rest on its laurels. It continues to monitor the accuracy of all forecasts, continues to reweight which data sets are used at all locations in all time horizons. Mr. Farlekas says that E2open has strong evidence that the more humans are allowed to change forecasts, the worse those forecasts become. In short, this is a black box solution.

This is, perhaps, why I had a hard time remembering that the E2open solution was based on machine learning. Machine learning has only become sexy in the last year or two, before that the folks I talked to at the old Terra Technology seemed a bit reluctant to explain exactly how they did what they did, because “black box” solutions were definitely not sexy.

The same dynamic is playing out in the supply planning area for the process manufacturing industry. A leading supplier recently explained to me how they use machine learning to solve a particular planning problem, but then asked me to keep that information confidential because solutions not based on “first principles” in that industry are considered inferior. They have been using this machine learning solution since 2002, but you would never know that from reading their website.

One final point, the E2open solution is a productized solution. Mr. Farlekas said that their solution is “commercial grade, our customers don’t need data scientists and the solution does not require training.”

However, as we begin to look at adding unstructured data sets to our demand models – like social media sentiment – it is a tough problem to analyze that data and transform it into ‘scored’ data that can be analyzed by the demand planning engine. I talked to Jeff Bodenstab, the VP of Marketing at the ToolsGroup, on this topic. They have just released a product called Social Sensing to solve this problem, but he admits it is early stage. They are just going through their first implementations right now. This data set will be most useful for new product introductions and promotions forecasting.

In conclusion, machine learning is a hot new topic for supply chain executives. But it’s not new.

What Does Demand Sensing Teach Us About IoT Projects?

Business Intelligence | By Steve Banker • 12/12/2016

Perhaps this year’s biggest catch phrase was the “Internet of Things” (IoT). IoT is centered on the idea that there is a tremendous amount of sensor data that could be used to run value chains more effectively. Within demand management, however, there are already huge amounts of data that too few consumer goods and food & beverage companies fully leverage – Point of Sale (POS) data. In my conversations with product managers and executives from supply chain software companies, most estimate that less than 20 percent of their customers have made the transition to improving their forecasts with demand sensing data. The consumer goods companies that do leverage POS data tend to be very large multinationals that serve as category captains for the largest retailers. Our discipline should be doing a much better job when it comes to demand sensing.

Yet we know that this data can improve forecasting, particularly short term forecasting that is used to improve replenishment planning. Jeff Bodenstab, a Vice President of Marketing at the ToolsGroup, showed me a slide I like a lot, but which I did modify a bit.

Stages in Demand Management (Numbers indicate required data points and are meant to be explanatory rather than exact)

When companies are at the beginning of their journey their forecasts are not based on granular data. A company might start by working with aggregate data on how many products, at the regional and product family level, are apt to be sold in a given period. Then a supply chain manager uses “splitting formulas” – formulas generally based upon historical sales – to disaggregate, for example, 80,000 cases of cola that will be sold in the Northeast, into specific stock keeping units (SKUs) to be shipped to particular warehouses. SKUs represent the individual products that make up a product family. So a product family might be sodas – the SKUs represent the different flavors of cola and the different sizes and container types those products are sold in.

The next stage would be to forecast at the SKU level at the distribution center level. This is a bottom up forecast where historical SKU shipment data from each DC is used for replenishment planning.

Even more data is needed to plan at the “ship to” level. Here instead of forecasting how much of a SKU will ship in total from a DC, a company is using historical data to forecast how much of a SKU will ship from that DC to every store location in that region that is supported by the DC.

“Each stage requires roughly an order of magnitude more data,” Mr. Bodenstab explained. But by the next stage – the use of POS data for demand sensing – we are getting into Big Data territory. Here, for example, a company might download all the consumption data on which of the company’s SKUs have been sold in the last week or even the preceding day by a given store. There are different ways this forecast can be made, but the more common way is to understand a store’s reordering policies, and the on hand inventory at the store and local DC, and then forecast what the store will order next week. “When forecasts are expressed in daily time-buckets,” Mr. Bodenstab pointed out, “the system can make use of ‘daily profiles’ that more precisely describe customer behavior patterns, within the week or the month.”

When it comes to using POS data, many demand management professionals will tell you it is not easy. Companies that embrace demand sensing will need to implement a Demand Signal Repository to store all this POS and market syndicate data; inaccurate and incomplete data needs to be “cleaned,” and the demand forecasting systems have to scale to be able to handle the data volumes.

But there is good ROI in these projects. Demand sensing leads to better replenishment planning (the right inventory in the right locations), makes it more likely that the products customers are looking for will actually be on the shelf (and the customer won’t try a competitor’s product and end up having a new preference), and improves promotional planning.

If demand management teaches us anything about learning to leverage the Big Data generated by sensors, it is that leveraging IoT data is likely to be a lengthy maturation process.

Demand Management vs. Demand Sensing

Consumer Packaged Goods (CPG) | By Steve Banker • 05/04/2009

On April 22^nd RedPrairie issued a press release announcing that it was adding “flowcasting” to its product suite. Despite reading the press release carefully, I couldn’t fully grasp what “flowcasting” was. Fortunately, RedPrairie’s management team visited us the next day and we discussed this new solution.

It turns out that “flowcasting” is synonymous with “demand sensing” and “sense and respond”. The industry has not settled yet on a single term to describe this emerging class of solutions, but I will use “demand sensing” because it is used more frequently.

What is the difference between demand sensing and traditional demand management? In traditional demand management, you start by summing historical sales and shipment data across all customers over some time period. This historical demand allows you to make baseline revenue and unit forecasts. The forecast is done at different levels of aggregation. Typically, the most granular forecast is by SKU by the manufacturer’s distribution centers (DCs). Usually, several DCs are supplied by a particular factory, and there is also a factory-level forecast. If several factories can manufacture a SKU, there would probably also be a regional or worldwide forecast. So while we talk about “forecast” in the singular, this is only true at the highest level where we have summed the forecast across all products and factories for some period of time. That global forecast is disaggregated into multiple “forecasts” for particular factories, DCs, SKUs.

The forecast is then shared with sales and marketing, and possibly customers, and is adjusted based on upcoming promotions, new product introductions, and so forth. While the forecast is used by various groups in the company, the primary driver for the forecast is to create manufacturing plans, often on a monthly schedule (e.g., SKU by factory by month). At many manufacturing companies, the production plan, ideally in combination with DC-level forecasts, then drives replenishment planning for deliveries from a factory to a DC.

In contrast, demand sensing is based on more granular demand data. In the case of RedPrairie’s solution, demand data is based on a customer’s historical POS and in-store inventory data. Then, as is the case for traditional demand management, users collaborate to adjust the forecast for promotions and new product introductions. And the primary purpose of this forecast is not manufacturing planning, but to drive better replenishment planning between the factory and the DCs.

While there are some differences in how individual manufacturers go about their forecasting process, and in particular Demand Management and Sensing solutions, in general the table below highlights the differences as I see them.

Demand Management vs. Demand Sensing (click to enlarge image)

RedPrairie showed us the results achieved by a very large Food manufacturer in supplying one of its largest retail customers. This food company was having problems creating accurate replenishment plans, particularly around the holidays. In 2008, as compared to 2007, their holiday period store sales increased by 20 percent, DC out-of-stocks decreased by 86 percent, and DC Inventory turns increased by 10 percent.

While it’s nice to hear this success story from RedPrairie, I’d prefer to hear it directly from the food manufacturer. RedPrairie will be presenting its new flowcasting product next week at its upcoming user conference and they are hopeful a supply chain executive from the food company will co-present with them. My colleague Adrian Gonzalez is attending the conference (he’s speaking on “Is Green Good for Business During a Recession?”), so if this food manufacturer presents more details about their success with flowcasting, you’ll hear about it here first.

New Distribution Optimization Software Solutions

Logistics Technologies | By Steve Banker • 01/27/2009

I recently completed a study of the Supply Chain Planning (SCP)software market. In doing this study, I came across solutions that could not be cleanly classified as either Supply Chain Planning or Supply Chain Execution.

First, a quick background on supply chain planning and execution solutions. SCP solutions are typically based on optimization algorithms. I will spare you a detailed definition of optimization (it might make your eyes glaze over), but in the simplest terms, think of optimization as being very advanced math. By definition, Supply Chain Planning solutions focus on a “planning horizon”-i.e., on putting together optimal and feasible plans to be executed in the coming weeks and months. The primary way companies save money using SCP solutions is through reduced inventory (e.g., reductions in raw materials, work-in-process, or finished goods inventories).

Two types of Supply Chain Execution (SCE) solutions are Warehouse Management Systems (WMS) and Manufacturing Execution Systems (MES). Both are, in some ways, the opposite of SCP solutions. While some WMS solutions have math associated with them (e.g., labor management and slotting), they are not true optimization solutions. WMS and MES operate in an “execution” time frame, focusing on what needs to get done today, in the next few hours, or even in the next few minutes, to operate the warehouse or factory efficiently. The primary payback of high-end WMS and MES comes in the form of improved labor efficiencies.

“Demand sensing” solutions are an emerging hybrid solution that are part SCP, part SCE. One such solution is offered by Terra Technology. Terra is a private company with several consumer goods and food and beverage manufacturing clients. Like SCP, their solution is based on advanced optimization. Terra’s demand sensing solution is designed to take daily orders and shipment data, combine it with the weekly shipment forecast, and then create daily SKU forecasts at the DC level. The daily forecasts extend out over 40 days and are refreshed daily. These short-term forecasts are considerably more accurate than what is generated by demand planning solutions. Further, like SCP solutions, the payback is based on reducing inventory. Procter & Gamble (P&G), for example, publicly stated that its short-term forecasts were improved by over 30 percent, allowing it to reduce DC safety stock by more than 10 percent.

This solution, however, is used in an execution time frame. The primary users of this solution are replenishment planners that plan shipments from the factory to the DC. The planners are not part of the demand planning staff. This solution is generally not used to change the demand forecast, but to improve replenishment processes that occur several times a week, an execution time horizon.

Procter & Gamble also uses a company called Transportation Warehouse Optimization (TWO) that offers another hybrid optimization/execution style solution. TWO’s solutions take the retail DC orders and delivery schedule and create a lean/pull type environment that is used to facilitate both full truckload shipments and efficient labor utilization in the warehouse.

First, advanced math (but not true optimization) is used to build mixed-SKU pallets and to efficiently build truck loads that respect weigh out, cube out, and stop sequencing constraints. In building the mixed SKU pallets, logic is utilized to insure that case picking proceeds efficiently. In other words, you don’t want your pickers moving from one side of the DC to the other to build a particular pallet. The pallet-building logic needs to respect weight/size/crush constraints, while not resulting in excessive travel time for pickers. Finally, the solution has Distributed Order Management style functionality that looks at orders daily, or in near real time, to determine which DC makes the most sense to ship from.

Thus, like supply chain planning, this type of solution contains very advanced math that could be loosely labeled “optimization.” But it can also be labeled an execution solution because it operates in an execution time frame and one of its main payback buckets is efficient labor utilization in the DC.

Transportation Management Systems (TMS) have long been labeled a supply chain execution solution, but they’re really hybrid applications that contain elements of optimization occurring in “execution” timeframes. Now, new SCE solutions that that involve distribution operations have emerged with the same characteristics.

We traditionally believed that you could not optimize across both warehousing and transportation operations simultaneously (at least not in practice). Most companies typically optimize their transportation operations first, and then “sub-optimize” their warehousing operations, instead of conducting a “holistic” optimization that jointly considers both transportation and warehousing constraints to arrive at an optimal plan. In other words, we argued that WMS integrated to TMS did not mean you were achieving true logistics optimization. That argument is starting to breaking down.

A complete analysis of logistics trends, technologies, and services

Archives

Login