It was recently announced that Walmart has expanded its use of autonomous mobile robots (AMRs) used for real-time, on-shelf product data. Walmart will put these AMRs in 350 stores. The robots come from Bossa Nova Robotics.
I recently had the chance to talk to Sarjoun Skaff, the cofounder and chief technology officer at this San Francisco based company.
I had seen AMRs with cameras designed to track inventory in stores before. The robots I saw were designed to track on-shelf inventory. But they did it badly. They could not really count the number of units of a stock keeping unit in a slot because they could not see the units behind the item in front. They also had a hard time identifying products that were twisted on the shelf.
Mr. Skaff explained that is not what their robots were designed to do. They identify whether a slot is empty or full. They identify which stock keeping unit (SKU) is in a slot and the price associated with that slot.
This is not trivial. Having items incorrectly priced is illegal if the goods have been promoted at a lower price in advertisements or online. It is also poor customer service. Knowing that a stock keeping unit is not on the shelf allows workers to stock the item and capture sales they would otherwise lose.
Recent ARC survey-based research shows average front of store inventory accuracy is 88%. That is abysmal when you compare it to the 99.9%accuracy you can get in a warehouse using a warehouse management system and scanning. The poor inventory performance not only leads to lost sales, it lowers demand forecast accuracy. If the store system thinks an item is in-stock, when it is not, and the product is not selling, the forecasting engine can decide that a fast-moving item is a slow mover. A rough rule of thumb is that a 1% forecast improvement leads to a 2.5% reduction in the amount of inventory that needs to be held. That is big money for retail chains! Poor in-store inventory errors can proliferate and cascade in a manner that leads a retailer’s supply chain to get jerked around. The result is increasing replenishment costs.
On shelf availability can be a metric that improves the inventory accuracy at stores, although it clearly does not get you all the way to kind of inventory accuracy the supply chain team would like to see. But Mr. Skaff also points out that increasingly mass merchant retailers are keeping surplus inventory not just in back rooms, but on the top shelf above the slotted products on the shelves. They redesigned their robots to be able to identify whether these SKUs were present or not.
I asked how their robots integrated to a store’s planogram. A planogram is a store shelf map that shows where every slot on a shelf is, how big it is supposed to be, and what product is supposed to be in that slot. I was surprised to learn that their solution does not integrate to planograms. It provides a “Realogram, real-time visibility to the truth on the ground.” When they started, they were trying to match what their robots detected to the planogram. But that approach, “just didn’t work.”
The company started working with Walmart in 2014. “We figured things out as we went. I pinch myself that we were able to work with Walmart. That they took a chance on such a small company. I’ve been impressed with their ability to collaboratively work with us, take risks, and be patient while we figured things out.”
The problem was that planograms just changed too often, and Bossa Nova would not even know it was working off the wrong data. “We went back to the drawing board.” It is up to their customers to take the Bossa Nova data and match it to the planogram. More sophisticated retailers can load the Bossa Nova data into their planogram analytics solution and that system can reveal whether there are exceptions or not.
The accuracy of planograms is suspect, but the problem of data accuracy around product packaging is a magnitude worse. Consumer goods manufacturers are changing packaging all the time. The brand owner might add a ribbon that says “sodium free” for example and it would not be immediately clear if this is a new SKU or new packaging for an existing product. Products from the same company may have subtle differences from one SKU to the next – regular Diet Coke for example vs. Raspberry Diet Coke. Mr. Skaff said that conventional computer vision approaches required manual labeling of product images so they can be recognized on the shelves.
Having robots being able to autonomously navigate through a store without relying on 2D tags or other markers is an impressive technical feat. But it was not the toughest technical problem the robotics company had to solve. Accurately identifying products was a far more difficult technical task! Bossa Nova couldn’t adequately solve this problem until they bought HawXeye last July. HasXeye was a spin off that came out of Carnegie Mellon University’s Biometrics lab. HawXeye and the Biometrics lab were using artificial intelligence to do facial recognition. It turned out this was just the solution for identifying products, even when turned sideways. Now the robots can rumble up an aisle and within two minutes identify all the SKUs in the aisle as well as the exceptions (new SKUs or products whose label changes make them unidentifiable).
Business logic is used to drive the navigation. Instead of just navigating all the way through a store, the robots may go down aisles with fast moving products or high margin products several times a day. Other aisles might only be visited a couple times a week.
You could have a small army of store personnel checking the 150,000 products stocked in a 250,000 square supercenter and still have very poor inventory accuracy. This is a problem that needs automation, even if the automation only takes a retailer part way to the finish line of perfect in-store inventory accuracy.