Why Big Data Matters To Every Business

Almost five years ago, Google’s Eric Schmidt announced that we had reached the point where more data was being created every two days, than in all of human history, up until 2003.

Since then, companies of all shapes and sizes have been getting to grips with new ways of handling the incredible volume of information that is becoming available to us every day.

For example, users of Facebook upload around one billion pieces of content to the social network site every day. In industry, machinery and vehicles are fitted with sensors and trackers that record their every move, and whenever we call a call centre, an audio recording of our conversation is made, and stored in a huge digital database.

In addition, whenever we go online (as most of us increasingly do for a number of reasons – shopping, socialising, making travel arrangements) we leave behind a digital footprint – a record of websites we visit, products viewed, even how long we leave the mouse cursor over certain areas of the screen, in some circumstances.

We collectively refer to both these huge datasets we are building, and the practice of interpreting, analysing and acting upon insights gleaned from this information, as “big data” – and it is changing the world we live in.

In healthcare, vaccine manufacturers are pooling data from clinical trials and using it to come up with better and cheaper medicine. Millions of financial transactions are logged and analysed every second, to crack down on fraud. And at CERN, the Large Hadron Collider is generating 300 gigabytes of data per second as it attempts to find the fundamental building blocks of our universe.

The online behemoths which have come to dominate business in the internet age – Google, Facebook, Amazon etc. – all base their business models on big data. It is by collecting and analysing huge amounts of information from us that they are able to determine precisely what we want – as well as selling advertising services which are capable of precisely targeting their client’s preferred demographics.

Why size is no issue

But Big Data is not just for the big boys, it matters to every company – no matter how small or traditional. To cater for this huge demand many companies have sprung up to offer services to other businesses, enabling them to launch big data initiatives of their own. In other words, to leverage the information they have available to improve effectiveness and efficiencies in their business, and ultimately increase profits.

A lot of the software and analytics tools needed to carry out big data analysis are built on open source principles – meaning they are essentially in the public domain and free for anyone to use for any purpose.

For example, Hadoop is a framework – a collection of software tools and applications – designed to allow organisations of any size to store and analyse huge amounts of information. It is designed to run on cheap, commonly-available hardware rather than expensive, specialist equipment that would previously have been necessary.

Companies including Amazon, Google, IBM, HP, as well as newer names such as Hortonworks, MapR and Cloudera offer big data solutions and support, as well as tailored versions of the free products, designed to work out-of-the-box and with less complex setup requirements. This also enables companies to minimise infrastructure investments or avoided completely by using cloud-based storage and analysis tools that can be rented when needed.

Big data initiatives, within businesses both large and small, are helping us to pinpoint precisely which efforts are causing which results – allowing processes from research and design, to manufacturing, marketing and after-sales support to be fine-tuned for optimal results.

Big data is big business. Worldwide, companies are expected to spend ￡80 billion on products and services to enable them to carry out big data analysis this year. But when it has been estimated that failing to effectively capitalise on their data costs business in the US alone close to ￡400 billion each year, it’s no surprise that respected market researchers Gartner recently found that 73% of businesses have already invested in a big data plan, or are planning to do so in the next few years.

The relevancy of The Internet of Things

Big Data is closely related to what is known as the “Internet of Things” – the concept that, increasingly, everyday devices are becoming connected and “smart”; able to communicate with each other wirelessly, and log information about the way they are used.

In the consumer sphere, watches, bathroom scales, power sockets, vacuum cleaners and even lightbulbs– are all becoming “smart” – connectable to a wireless network and capable of recording and uploading information about their use. And American industrial giant GE is investing heavily in what it calls the “Industrial Internet”, where sensors relay information that allow faults and breakdowns to be detected and sometimes even predicted before they happen, minimising downtime. By 2020, it is estimate that 30 billion devices around the world will all be connected to each other.

To me it is very clear that we are well past the time when businesses should be wondering if they have to worry about big data. The real question is, can they afford not to? Big data is already revolutionising just about every industry and area of life and if you aren’t on board, you’re risking getting left behind.

Thank you for reading my post. This post was first published in my column for Hiscox, you can read my other posts here. 

Here at LinkedIn and at Forbes I regularly write about management, technology and the mega-trend that is Big Data. If you would like to read my regular posts then please click 'Follow' and feel free to also connect viaTwitter, Facebook and The Advanced Performance Institute.

Here are some recent Forbes articles I have written:

• Meet 7 Brand New Big Data Billionaires
• How Big Data Is Changing Healthcare
• 7 Attitudes That Kill Big Data Projects
• Where Big Data Projects Fail
• Big Data: Saving 13,000 Lives A Year By Predicting Earthquakes?
• Big Data: The Winning Formula In Sports

And here are some related LinkedIn posts:

• Big Data Explained in Less Than 2 Minutes - To Absolutely Anyone
• Big Data: 25 Eye-Opening Facts Everyone Should Know
• Big Data: The Predictions For 2015
• Big Data: The Key Vocabulary Everyone Should Understand
• Big Data: The 4 Layers Everyone Must Know
• 10 Awesome Ways Big Data Is Used Today To Change Our World

About : Bernard Marr is a globally recognized expert in big data, analytics and enterprise performance. He helps companies improve decision-making and performance using data. His new book is Big Data: Using Smart Big Data, Analytics and Metrics To Make Better Decisions and Improve Performance'.

Photo: Lester Balajadia / Shutterstock.com