THE BIG DATA PHENOMENON

When it comes to the fairly new buzz word ("Big Data") many of us are very confused if not in the dark completely. Yes, we've heard the word before and yeah, we've thrown it into a few discussions about the rapid technological evolution that we're undergoing...but what is Big Data really?

Is it..

• A product?
• A service?
• Some tool the government is utilising to gain sensitive information?
• A font?

What is Big Data?

According to Wikipedia:

"Big Data is a broad term for data sets so large or complex that they are difficult to process using traditional data processing applications. Challenges include analysis, capture, curation, search, sharing, storage, transfer, visualization, and information privacy."

If you're still not too clear on the true definition of the term, you're probably not alone. In the simplest terms, it's data, raw data that's collected from an array of sources, in diverse formats that can provide some sort of insight when processed. Processing is said to reveal the data's true meaning or make the data more meaningful.

7 V's of Big Data

The trailblazers of the current technological era have defined 7 terms in an attempt to quantify Big data and distinguish between what's big enough to fall in the Big data category and what isn't

1. Volume

How much data do we have? It is said that every day, approximately 2.3 trillion gigabytes of data is generated/created. In true fact, when the amount of data exceeds a certain number of gigabytes, it's not actually gigabytes anymore. Companies generate and collect petabytes, exabytes, yottabytes and even zettabytes of data!

2. Visualisation

Okay, so we have hordes of data, they reveal almost nothing so we process it to gain some sort of insight...but how best would this be used by whoever requires critical information? This is where visualisation comes in...the results of processing data needs to be shown to the user in a way that facilitates decision making. These include charts, ratios and other metrics. These visualisation tools are optimised to provide advanced comparison techniques and some even depict the future trends produced by sophisticated in order to gain a competitive advantage.

3. Value

This is the only reason organisations tend to invest time and money, value. If a specific technology or even employee doesn't add value, it doesn't need to be considered.

4. Variability

First a distinction needs to be drawn between the terms Variability and Variety. Imagine a chocolate factory sells 100 different types of chocolate...this is Variety...now imagine you going there everyday to buy a caramel truffle...but everyday but each day it tastes different...this is Variability.

In context, Variability means that the meanings of data are changing very, very rapidly. Example, a Facebook status could have the same words (or similar) yet a totally different meaning. In order to properly analyse data, algorithms need to understand context and be able to decipher meanings in context...This is still not as easy as it may sound

5. Veracity

This entails ensuring the accuracy of your data. Having hordes of data flow in and out your system, through complex decision making and analytical algorithms is pointless if data is inaccurate. Inaccurate data leads to incorrect results from analytical tools which entirely can and in most cases will affect business activities. Example, people entering a system with false names

6. Variety

This is one of the biggest challenges of Big data...it can be almost anything in any format. This can range from video to audio, SMS to XML and structured to unstructured data. Organising the data becomes even more difficult when data is in collected from multiple sources in diverse formats and is rapidly changing

7. Velocity

This is the speed at which data is created, stored, analysed and visualised. Previously slow data access was acceptable and people were accustomed to waiting for what we deem simple processing tasks. In the present day, there is no time, time is money and decisions need to be made immediately. Currently, data is created and processed in a near real time fashion. Statistics have shown every minute, we upload 100 hours of video, make 2.5 million internet searches, send 200 million emails and view 20 million photos.

Examples of Big Data usage:

• Weather conditions are taken every second from every weather station on the globe
• Tipp24 AG, a platform for placing bets on European lotteries, and prediction. The company uses a specific system to analyze billions of transactions and customer attributes, in order to develop predictive models that target customers and personalize marketing messages.This led to a 90% decrease in the time it took to build such models. 
• Tesco PLC : A supermarket chain that collected 70 million refrigerator-related data points coming off its units and fed them into a dedicated data warehouse. Those points were then analyzed to keep better tabs on performance, gauge when the machines might need to be serviced and do more proactive maintenance to cut down on energy costs.

From the above examples we can gauge why Big data and Big data analytics are so important...it helps deliver the competitive edge that business need to gain in order to survive in the information age. Such analytics lead to faster decision making, easier and more effective monitoring and predictions of the future for business based on historical data and sophisticated mathematical algorithms...not just human intuition. This means that we can make immediate decisions based on existing data, which reflects the company's current situation, we can monitor the outcome of such decisions and then use sophisticated technology to predict future trends which we can plan to respond to way before they even occur.

Big data is not the future, it's the present...But it can be used to predict and ensure the future of your business...

"Without Big Data you are blind and deaf in the middle of a freeway"

Geoffrey Moore, Management consultant and theorist