Big Data - Uncovering New Sources Of Value

Executive Summary

“Big data” is a terminology that is being excessively used on our discussions and conversations lately but it could be barely understood well by few! Even though, it is one of the next big things as stated in Gartner Technology Hype diagram but overstating this terminology on our discussions and conversations reflects our high level of expectations for the impact that we suspect from this terminology.

This post was written to explain the inception and evolution of "Big Data" over the years and it concludes the real meaning and key characteristics of “Big Data”.

The real story about big data…

Chronologically, the story has been started with the structured data era where we have captured lots of data for almost all transactions that are being automated. Millions of dollars were spent on capturing, integrating, exposing and building data warehouses to be able to manage all the captured structured data. As a quick example, online payment systems capture your personal information, financial information, and transaction information. All these data are linked into structured fixed predefined format where your phone number, email, nationality, date of birth, etc.; are captured and stored in a structured manner. 

Overnight, things changed dramatically and we found ourselves in the middle of unstructured data era! Unstructured data such as images, videos and text were governing the scene. On a yearly basis, it is estimated that a worker writes about a book’s worth email! People talk about events on twitter, blogging, and Facebook and producing massive amount of unstructured data! Every single photo that is being posted on Facebook contains people, location, date, mood, status and much more! 

Market leaders in various industries exploit this opportunity to create a unique competitive differentiation through analyzing the unstructured data to get better insight into it & allow decision makers to take the right decisions by tailoring their products and services to the level that would fulfill the expectation of their customers. As example, e-commerce industry leaders such as Booking.com, Amazon and Souq.com exploit this opportunity to give us more visibility over how we should spend our money, do more saving and proposing alternatives! Social network industry leaders such as LinkedIn, Twitter, Facebook, etc.; give us more visibility over things that are on our zone of interests, link us to people that we may know and much more! For market leaders, having more satisfied and loyal clients and generating more revenue were part of the positive outcomes from exploiting this opportunity.

In parallel with unstructured data era, sensor data era has started! Devices are collecting massive amount of data. Starting from your smart devices\phones that capture your motion and movements paired with GPS (Geo-location data) till Boeing 787 Dreamliner that generates half a terabyte of senor data every flight! 

What is “Big Data”?

The massive amount of structured and unstructured data that are coming from multiple data sources and are characterized with it is large volume of data and it is high velocity\high growth rate is what we call "Big Data".

There are four main characteristics (4 V’s) that define Big Data; Volume, Variety, Velocity, and Veracity.

• Volume - Scale of Data - The size of the massive amount of structured and unstructured data.
• Variety - Diversity of Data - Variety on two sides; Data sources & Data types (Structured, Semi-Structured and Unstructured)
• Velocity - Speed of generating data & processing it - Data could lose it is value, if it wasn't processed instantly.
• Veracity - Certainty of Data - The more accurate & consistent the data, the more accurate the analysis results & the value that we can get out of it!

Finally, all the above characteristics are surrounded by Value. Without extracting significant insights from "Big Data", "Big Data" is just a noise. By using advanced analytics and cutting edge technologies, a greater value from "Big Data" can be achieved and it will definitely change the way organizations do business through uncovering new sources of value in all industries.