The Impact of Big Data

The scale of data & information has changed the landscape of our lives as we know it.  Very few people understand the intensity & scale of information (data) is playing in our lives.  Only when you work with it do you fully grasp it’s magnitude. During 1992 the world grew at a rate of one hundred gigabyte of data per day, this at the time was an enormous amount of data.  In 2018 it is estimated that we will generate fifty thousand gigabyte of data per second !!!! That is truly mind warping in trying to understand the scale of it... So during the last twenty six years the science of understanding data has grown exponentially, every organization that matters has two or three or a team of data scientists (data analysts) defining & re-purposing large volumes of data to give better understanding / insight about either business or process values happening in real life... The scope for big data analytics has skyrocketed over night and the world is searching for data analysts at an alarming rate. In 1963 there was approximately three billion people on earth and today we have approximately three billion people on the internet.  The current research is saying that only in the last two years did the exponential growth took place with regards to generating mass volumes of data, therefore the urgency to understand this vast amount of data is more pressing than ever before as it can present new opportunities and insight into everything we do as humanity. Here are some staggering numbers to contemplate, the world predicted that in 2016 the total internet consumption would amount to 1 Zettabyte of data. To put that into perspective this is what it looks like for those who understand data sizing. 1 ZB = 10007bytes = 1021bytes = 1000000000000000000000 bytes = 1000 exabytes = 1 million petabytes = 1 billion terabytes = 1 trillion gigabytes.  As you can see, the context of this number is massive !! So for the non IT literate reader what this equate too is the following, you would need fifty eight billion 32 Gig IPads to store two Zettabytes of data.  So if you would stack all those IPads together it would be enough Ipads to build the wall of China at twice the height.

So where does this data come from?  It comes mostly from Computing, Miniaturization, the cloud, Robotics, Connectivity, Economics, People & Wireless. These are the everyday drivers which are integrated into our lives as we know it.  It is estimated also that by 2020 there would be around fifty to two hundred billion devices on earth connected to the internet generating data. The rate at which we are connecting devices to the internet will have an estimated market value of about twenty trillion dollars in 2020.  These numbers are just mind staggering but also important as it is busy changing the culture of things around us daily. The point to make here is that IoT devices, the internet, big data and society or culture is being embedded into an interconnected lifestyle that will build a stronger reliance or co-dependence with one another like never before.  The scary question has to be asked, which is: “What are the effects that this will have on our core values in our society going forward?” Is it okay to rather prioritize digital devices and data over a cup of coffee with someone... ? Is it okay to send a text message to my work colleague sitting in the cubicle next to me because it is more efficient?  Does this not pose a threat to the very DNA of our society which is based upon human interaction? How far do we have to go down this road to first make mistakes before we try to stop this? It is my opinion only that this wave of big data is so large that it will eventually pull everybody with it that has build their life’s onto it. The Christian Bible refers to a passage in Matthew 7:24 which says: “24 “Everyone then who hears these words of mine and does them will be like a wise man who built his house on the rock. 25 And the rain fell, and the floods came, and the winds blew and beat on that house, but it did not fall, because it had been founded on the rock. 26 And everyone who hears these words of mine and does not do them will be like a foolish man who built his house on the sand. 27 And the rain fell, and the floods came, and the winds blew and beat against that house, and it fell, and great was the fall of it.”

So what do we do? Do we build / exchange our values of society on data centres and applications? We need to work with data everyday but how do we balance life / human interaction and this wave of data that is only getting larger by the second? Let me address this point for another time, for now i want to focus on the possibilities & challenges that awaits us.

As a big supporter of GE & what they are doing with data I recently heard that on the new jet engines GE are using around five thousand sensors on one engine, sensors are small instrumentation like devices that tracks set point information in mechanical, electrical & structural operational environments.  It is said that on an twelve hour flight one engine will produce about ten terabyte of data!!! That means both engines will have twenty terabyte of data for twelve hours of flying. So do the math yourself, one engine with approximately a hundred thousand hours or more would have a net result of zettabytes of data… That is only ONE engin of thousands of engines from ONE supplier globally.  The bible again uses a a term called “sela” which means to “pause” on that thought, you would agree with me that there are a few places in this article to “pause” upon. So let's look at Utilization of data first, here are some interesting stats on real life utilization & business impact of data:

BIG DATA UTILIZATION STATS

1. 0.5% of all big data is being used at the moment
2. 75% of all big companies say that big data can transform into results
3. 85% of the economy has not realized the potential of big data
4. 90% of big data is unstructured

FINANCIAL IMPACT

1. Aerospace 0.6 Trillion USD
2. Automotive 0.7 Trillion USD
3. Electricity 1.3 Trillion USD
4. Healthcare 0.4 Trillion USD

So what does this mean?  It means only one thing.. OPPORTUNITY !!!!!!!!  This is equivalent to mining for gold, ore or any mineral worth mining for.  We are sitting on massive amounts of potential minerals in massive quantities all over the globe which is not being mined.  The next generation of humanity is going to transform the earth and beyond as we know it through data mining in all sectors of humanity.

This also speaks to another area of humanity which is urgently needed in developed nations.  Productivity is most probably the first area of change that can be implemented with very low “mining” costs.  The world needs productivity and it needs lots of it, big data has undenying potential to increase productivity.  Therefore how do we “help” the world to gain productivity? The world is faced with huge kinds of problems such as Social Economic, adverse weather conditions, business challenges etc…

The problems are effectively only getting bigger and more global as life happens. The “mining” techniques has not changed which are:

1. Data produces insights
2. Insights produces Actions
3. Actions produces Improvements

These three fundamentals stays the same no matter how big the volume of data!  Within this framework of “mining” we are able solve virtually any problem of any magnitude.  

How does anyone start with Big Data mining?

1. Pick an Industry
2. Pick a problem
3. Think of possible solutions
4. Start small and build
5. Embrace Innovation
6. Apply Digital Applications

Challenges & Opportunities associated with BIG DATA?

1. Network Speeds
2. Storage
3. Security
4. Analytics
5. Visualizations

CONCLUSION

Over the last 150 years of innovation the following technological discoveries has helped to advance our society and culture in enormous ways:

1. 1752 Franklin discovered electricity - 1879 First light bulb -  1882 First power plant
2. 1903 First flight - 1939 First Jet - 1969 Moon landing
3. 1947 The Transistor - 1971 The Microprocessor - 1975 The first PC - 1991 The Internet
4. 1831 Telegraph - 1876 Telephone - 1902 Radio - 1973 Mobile Phone

The above innovations took place with less computing power than a mobile phone.  If we have achieved so much with human innovation how much more the potential with the staggering possibilities of big data. We are effectively standing at the edge of possibility of the greatest discoveries yet to come, discoveries that will change the landscape of our lives forever.  This is a rare point in history and significant!

No matter who you are this “wave” is coming and growing daily the question is? Are you ready to embrace Big Data.  Big Data is here to stay you cannot avoid it, you have to start somewhere so just start doing it! The opportunities is everywhere, go and take advantage as it will transform everything in the future.  How do you start you may ask? Start by finding a partner if you do not have one and set the course. New discoveries are daunting and you must challenge what you know. We don’t know what we CAN know.