I bet my Data is Bigger than your Big Data

For every Dutch citizen there about 132 types of personal data characteristics available to buy via a commercial data provider. This allows marketers to pinpoint us based on our religion, age, political preference, literary likings, favorite movies, sexual orientation and even our next holiday destination. These are not just random pieces of data, they form a comprehensive contextual personal profile of every individual. We have entered an age of massive information gathering on our customers and citizens. With the introduction of personal sensors, wearables and smart meters we are faced with a massive gathering of personal micro-information.

Massive data gathering is feeding the very hyped term “Big Data”. Technology providers are tumbling over each other to demonstrate their solutions for storing huge amounts of data, gaining faster access and delivering more complex analytics. This is leading to an almost obscene compulsion to gather and store data about every movement, micro transaction and systems interaction. With the sole purpose of find the “holy grail” of Big Data: “Get the answer to questions you have never envisioned to ask before”.

Modern companies cannot ignore this revolution. Their worst nightmare is to miss unforeseen insight in their customers due to the lack of data. This fear is reinforced by mythical examples of Big Data being the discovery of new and very profitable customers, products and services. Almost all aspects of our personal life are currently registered; travel movements, payment history, communication patterns, reading preference, media usage, consumption habits, shopping behavior (both online and in real life) and all our movements in the public space. So we are filling our own digital profile.

The problem with Big Data is embedded in the name. It is primarily focused gathering as much data as possible. Every IT manager is eager match ranks with his peers stating his big data plans. And he would definitely rise in prestige when he can talk about the “massive size” of his Big Data.

“The Big Data race seems to be something like a race to the top. Comparing sizes to determine who is going to be King of the Big Data Hill”

Don’t forget the final goal of Big Data: information

The major issue with a lot of Big Data systems is to produce actual information. To distinguish the interesting parts of data from the boring parts. The key to the success of Big Data lies in the usage of analytics to find the gold nuggets in this massive pool of Big Data-dirt. Big data is about gathering meaningful insights, not storing loads of data. Smart analytics and data science are the instruments to creating this information. So my advice about Big Data is to be Smart and not just Big.

The next time you see an IT manager showing off the size of his Big Data you can ask him if his data is also smarter; or is it just bigger.

If you are an IT worker I’d advise you to develop you analysis skills more and spend less time on skills to store massive amounts of data. Since storing will become a commodity and analysis will be the crucial competence in this world.

I hope you draw the same conclusion as I do after writing this piece:

In Big Data size does not matter, intelligence does. Quite similar to the “real world” after all…

Thanks to Mr. Pierre Hessler for inspiration on the title.

------------------------------------

Robbrecht van Amerongenis Business Innovation Manager at AMIS.
If you like this post please share via one of the buttons below or press the thumbs-up icon.