Big Data: Is The Hype Finally Over?

I saw a headline a few weeks back that read: “Big Data is Out; Machine Learning is In.” The post was based on Gartner’s “hype cycle” that showed while big data had big hype in 2014, it has all but fallen off the “hype map” for 2015.

 Does that mean I should start looking for a new job?

Hardly.

As the article itself pointed out, perhaps the reason the hype around big data is fizzling is because the technologies that were all the big news last year are already in use and no longer big news. (In fact, the technologies that did make the 2015 hype chart — including autonomous vehicles, machine learning, and the Internet of Things — all produce and rely on large quantities of data.)

This isn’t the first time that the media has proclaimed the premature demise of big data. After an article in the journal Science last year showed that Google’s much-touted big data experiment to predict flu trends was massively overestimating the number of flu cases, headlines popped up all over declaring that big data was a big mistake.

But let’s be clear: big data is not responsible for bad analytics and big data is here to stay.

In its purest form, big data is just that: data.  

As I pointed out in another recent article about some of the mind-boggling statistics around big data, we are positively swimming in data, and that’s not going to change:

• More data has been created in the past two years than in the entire previous history of the human race.
• By the year 2020, about 1.7 megabytes of new information will be created every second for every human being on the planet.
• By then, our accumulated digital knowledge will be around 44 zettabytes, or 44 trillion gigabytes, up from just 4.4 zettabytes today.

The problem is that our ability to properly analyze that data and draw conclusions from it is not keeping pace with the rate at which we create it.

And that’s OK.  We’ll catch up.  Because I’ve seen what big data and the associated data analytics can do for business, I know that the talent and resources will be poured into this field and our abilities will improve at a prodigious rate.

That’s why I’m not worried about whether the hype around big data is on the rise or the decline.

Remember the buzz around e-commerce or business intelligence? They all received overinflated valuations and made promises they couldn’t keep, but the phenomena themselves became very real and today are part of our daily lives and business.

So it will be — so it already is — with big data.  It may be simply that the hype is dying down because vast quantities of data are becoming commonplace. Even the smallest businesses run by a single person can access and analyze data about their business that wasn’t available to them even five years ago.

At this point, only 0.5 percent of the data we create is ever analyzed and used.  Imagine the potential that exists in even another fraction of that information?  By some estimates, just a 10 percent increase in data accessibility will result in more than $65 million additional net income for a typical Fortune 1000 company.

Those aren’t numbers any company can afford to ignore.  And that’s why I believe that big data will persist, hyped up or not.

The data isn’t going anywhere, and someone is going to figure out how to put it to good use.  The only question is which businesses will get there first to reap the most reward.

As always, let me know your thoughts using the comment function below. Also, Here at LinkedIn and at Forbes I regularly write about management, technology and the mega-trend that is Big Data. If you would like to read my regular posts then please click 'Follow' and feel free to also connect via Twitter, Facebook and The Advanced Performance Institute.

You might also be interested in my new big data case study collection, which you can download for free from here: Big Data Case Study Collection: 7 Amazing Companies That Really Get Big Data.

My current book is Big Data: Using Smart Big Data, Analytics and Metrics To Make Better Decisions and Improve Performance' and my brand new books (available to pre-order now) are Key Business Analytics: The 60+ Business Analysis Tools Every Manager Needs To Know and Big Data in Practice.

Finally, here are some other posts from my Data Informed column:

• The Hype May Be Over, bug Big Data Isn't
• The Difference Between Big Data and a Lot of Data
• The 5 Scariest Ways Big Data Is Used Today

Photo:  Shutterstock.com