Your Data is Worthless

Does your company consider itself smart? Are you using your data to make clever business decisions? Is your data the centre piece of your go to market strategy? Do you treat your company data as a prized corporate asset?

It seems that everyone answers these questions the same way. They answer positively and confirm the value of their company’s data. However, the truth is that most companies fail terribly when it comes to corporate data monetization.

An excerpt from the Harvard Business Review tells a very different story compared to what we hear from customers and what some of our customers actually believe is the truth. We knew that progress toward these data-oriented goals was painfully slow, but the situation now appears worse. Leading corporations seem to be failing in their efforts to become data-driven. This is a central and alarming finding of NewVantage Partners’ 2019 Big Data and AI Executive Survey, published earlier this month. The survey participants comprised 64 c-level technology and business executives representing very large corporations such as American Express, Ford Motor, General Electric, General Motors, and Johnson & Johnson.

Here are some of the alarming results from the survey:

• 72% of survey participants report that they have yet to forge a data culture
• 69% report that they have not created a data-driven organization
• 53% state that they are not yet treating data as a business asset
• 52% admit that they are not competing on data and analytics.

Further, the percentage of firms identifying themselves as being data-driven has declined in each of the past 3 years – from 37.1% in 2017 to 32.4% in 2018 to 31.0% this year.

So, what do you think now? Is your company further ahead of these market leaders?

Likely not.

McKinsey Global Institute Study shows organizations that harness Big Data and Analytics are: 23 times more likely to acquire customers; Nine times more likely to retain customers and; 19 times more likely to be profitable

But a year or so into a Big Data project, Gartner estimated that 60% of these Big Data projects fail. As bad as that sounds, according to Gartner analyst Nick Heudecker Gartner was “too conservative” with its 60% estimate. The real failure rate? “Closer to 85%.”

So, why are we failing with Big Data?

Our Thinking is all Wrong – The way Big Data gets treated is like it is a known beginning with a known end rather than an agile journey leading through constant exploration. Using Big Data you can derive patterns for tomorrow’s business success. However to get the answer you can take the exploration in mind but should not expect a defined deliverable out of it. Big Data is a constant research to gaining useful insights rather than deriving fixed conclusions soon. The true value of this data is discovered when it is put in business context else it is just a huge amount of data.

Another reason accounting to lack of proper research in Big Data projects is the unavailability of skilled data scientists. Although companies use agile solutions and tools like ETL, Hadoop, SAS, etc., these tools cannot fill the skill gap alone. The level of expertise and experience also plays its role in indulging in proper research in Big Data projects. More flexibility and a longer period of time are required for such experiments and to gain fruitful information out of Big Data.

The Elusive ROI – There needs to be more space for failures and learnings in the beginning. Until now, companies have been investing only a small portion of their money into maintaining and managing a part of Big Data. Most of the non-captured data are derived from surveys based on customer feedback, emails, social media, and distribution partners.

When companies faced the real volume, variety, and velocity of actual Big Data, they failed to perform. To add to it, enterprises could not cope up with the heavy amount to be invested in making their existing data setup in synch with these new challenges.

This resulted in organizations creating their very own versions of data marts leading to misinterpreted information.

Lack of Clarity – The projects dealing with Big Data are not completely tied to one’s unique objectives. These projects are just thought of as scientific with no business goals or metrics. In order to gain the maximum benefit out of it, you need to point your Big Data to a specific need or problem of your business. In order to justify your investments for Big Data projects, you would require showcasing your results continuously. The demand is for business needs having rapid and agile data access. Businesses look for very low costs for data-driven discoveries.

If operated properly, Big Data offers a wide range of possibilities to businesses today and in future. The problem lies in the lack of skilled professionals and failure in proper execution. It is only a matter of time when Big Data becomes an important part of business decision making. If these mistakes are kept at bay it will become a lot easier to execute any Big Data strategy. One more way to increase your chances of success is using the right tools for the right project.

In general, however the last few years of rampant Big Data failure suggest that a far better way is to start small, and build slowly. A well planned, iterative approach is smarter than some huge monolithic capital expenditure without clarity. Indeed, I would go one step further and suggest that companies seed these projects in a more bottom-up fashion, driven by developers. Let them experiment and grow projects organically. Given the current 15% success rate of big data projects, it is time to try something different.

References:

Asay, M. (2017). 85% of big data projects fail, but your developers can help yours succeed. TechRepublic. Retrieved on September 15, 2019 from, https://www.techrepublic.com/article/85-of-big-data-projects-fail-but-your-developers-can-help-yours-succeed/

Axyrd, S. (2019). Why 85% of Big Data projects fail. DNA – Digital News Asia. Retrieved on September 15, 2019 from, https://www.digitalnewsasia.com/insights/why-85-big-data-projects-fail

Bean, R., Davenport, T.H. (2019). Companies Are Failing in Their Efforts to Become Data-Driven. Harvard Business Review. Retrieved on September 15, 2019 from, https://hbr.org/2019/02/companies-are-failing-in-their-efforts-to-become-data-driven

NewGenApps. (2018). 3 Reasons: Why Most Big Data Projects Fail? New Generation Applications Pvt Ltd. Retrieved on September 15, 2019 from, https://www.newgenapps.com/blog/main-reasons-for-failures-of-most-big-data-projects

About the Author:

Michael Martin has more than 35 years of experience in systems design for broadband networks, optical fibre, wireless and digital communications technologies.

He is a business and technology consultant. Over the past 15 years with IBM, he has worked in the GBS Global Center of Competency for Energy and Utilities and the GTS Global Center of Excellence for Energy and Utilities. He is a founding partner and President of MICAN Communications and before that was President of Comlink Systems Limited and Ensat Broadcast Services, Inc., both divisions of Cygnal Technologies Corporation (CYN: TSX).

Martin currently serves on the Board of Directors for TeraGo Inc (TGO: TSX) and previously served on the Board of Directors for Avante Logixx Inc. (XX: TSX.V). 

He has served as a Member, SCC ISO-IEC JTC 1/SC-41 – Internet of Things and related technologies, ISO – International Organization for Standardization, and as a member of the NIST SP 500-325 Fog Computing Conceptual Model, National Institute of Standards and Technology.

He served on the Board of Governors of the University of Ontario Institute of Technology (UOIT) [now Ontario Tech University] and on the Board of Advisers of five different Colleges in Ontario. For 16 years he served on the Board of the Society of Motion Picture and Television Engineers (SMPTE), Toronto Section. 

He holds three master’s degrees, in business (MBA), communication (MA), and education (MEd). As well, he has diplomas and certifications in business, computer programming, internetworking, project management, media, photography, and communication technology.