Stop throwing away your Big Data!

Most people in business have seen a chart similar to the one below. The chart is designed to impress on you on the big data explosion.

You may have heard some big data prognosticator spout a statistic similar to the following:

• 90% of the data in the world today has been created in the last two years alone
• Humans create more data in just two days than was created in all of history up until the year 2003!
• World's data will grow by 50X in next decade

I am not going to defend or deny any of these claims. However, I have had multiple conversations with executives at Fortune 500 companies that deny that this phenomenon is happening to them. They assert that this big data explosion may be happening in Silicon Valley to companies like Google or Facebook, but it isn't happening in their mundane world of retail, insurance, manufacturing, healthcare, etc.

I tend to agree with them. The data isn't growing that fast. Instead, the data is already there, and those companies are simply throwing it away. They are already awash in big data, and they simply do not take advantage of it.

The big data explosion for traditional Fortune 500 companies is not about the growth of the data. The big data explosion is about using the data that has already exploded instead of throwing it away.

The big data explosion has already happened!

Let me give you a simple example. It happens to nearly everyone reading this post. It happened the last time you drove your car.

When you drove to work this morning, you probably barely looked at your dashboard to read and remember your speed. Granted, you would have had to put down your phone to stop reading your morning email while you drive (please don't do that) to record these values. The entire time you drove your speedometer was telling you the velocity of the vehicle. You didn't care, so you "threw that data away" as you thought it wasn't very valuable at the time.

Just like the tree that falls in the woods will make a sound, your speedometer tells you the speed of your vehicle even if you do not need the information. That data is constantly flowing to you, but you don't use it. In fact, in a recurring drive on familiar roads, you may never look at the speedometer during an entire commute! But it is still doing its job.

Now enter Google Maps. It is likely that you have Google Maps on your smartphone. It is also likely that you allow Google Maps to communicate your position to Google. As you drive down the road not looking at your speedometer, Google is gathering your GPS coordinates from your smartphone and calculating your speed (essentially, looking at your speedometer). Google is combining that information with its historical database of other cars on that road at that time over the years, as well as cars that are in front of you and beside you on the road. It is combining that information together (blending it) and telling the cars behind you that the traffic is slowing down or is normally flowing. It does this in the form of an analytic, a red or green line overlaid on the map of the highway.

I will say this a different way. Google is taking data that you barely care about (the reading of your speedometer) and using big data analysis to answer a question for its users. It is displaying that data in the form of an analytic that essentially answers one, and only one, question. "What is the traffic like on my route?"

Was new data created in this analysis? Absolutely not. The data was constantly being displayed to each and every driver as the vehicles went down the road. Most of that data was thrown away as the individual drivers thought that it was useless data. Google thought differently.

• For the manufacturer, are you saving all of the sensor data on all of your manufacturing lines? Do you know the ID of every worker that worked on every manufacturing process for the past ten years?
• For the transportation company, do you monitor the location and speed of all of your trucks, trains, and planes? Do you know the weight of every package for any given day?
• For the hospital, do you know the heart rate, blood pressure, and blood oxygen level for every patient for the past ten years? Do you know the ID of every caregiver that worked on every patient?
• For the retailer, do you know the price of every similar product that your customer didn't buy? If she bought one brand of detergent but didn't buy another, what were the price deltas, shelf locations, and availability of coupons of each of her detergent purchases for the past ten years?

The big data explosion has already happened. It isn't about the amount of data; it is about if you are gathering that data and using that data to make intelligent decisions.

Are you effectively using the data that is exploding in your organization to make decisions?

In your organization, how much data are you throwing away each and every day? Are you okay with that?

Data Never Sleeps image source

Speedometer image source

Data Explosion image source

Trash can image source