While “Big Data” has been getting all the attention, analysis of all that data can be improved if it is married with “Small Data.”
In the emerging world of an industrial Internet of Things (IoT) – IoT applications used by industry – more and more smart devices with sensors are using Internet protocols to transfer data to other software applications to improve operations. This has led to an explosive growth in data – the “Big Data” problem.
In an industrial setting, smart devices are often embedded in capital equipment: trucks, assembly machines, conveyors, forklifts, etc.
Small data (a subset of big data) refers to the set of data coming from a specific device that retains the structure for assessing that unique piece of equipment.
Preventing credit card fraud, a commonly used example of a Big Data application, fits this “small data within Big Data” approach. The information for all credit card holders becomes Big Data. The transactions for a specific card holder represents small data. The company monitors the activity of a card number and compares each transaction in near real time with the owner’s purchase history. If an individual has a history of purchases originating in the US and the bank becomes aware of one made, a flag is raised and the account may be frozen (to avoid this, just call the credit card company before your trip). Analytics on the small data assesses the condition of the credit card and raises a flag when they detect an outlier.
The small data concept can be used in the supply chain realm to improve the performance of several types of logistic assets. In other words, a firm’s trucks, warehouses, and factories can operate at a high level of performance without having to pay too much in maintenance.
As my colleague Steve Banker wrote, Shell, who runs one of the world’s most sophisticated production control towers, will tell you that when Big Data becomes truly massive, the small data format for alerting has to be built into an even larger analytical and business process format to become useful (Shell Oil Runs One of the World’s Most Advanced Control Towers).
Ralph Rio is ARC Advisory Group’s expert on Enterprise Asset Management and Field Service Management.