The rise of data at internet speed

It’s no secret the internet has revolutionized our daily lives. It serves us in almost everything we do, streaming our favourite movies and TV shows, sharing content on social media, shopping for groceries online and communicating with our friends. The internet has remodelled the way we do things, and this will only continue to morph and progress with time.

Let's remember that the world wide web has existed for years. What’s changed has been the increase in processing power, both on the desktop and in the data centre, as well as the expansion of communications around the globe.

The domain of data and analytics was static for a long period of time. We saw incremental and evolutionary growth in how data is stored, how it’s managed and the value it can deliver.

Over the last 5 to 10 years, there has been a fundamental shift both in how much data exists – which is so much more than ever before – and in the almost-universal shift to digital. The evolution of the internet and growth in bandwidth, coupled with “big data” storage alongside relational databases, and finally processing power available on demand via the cloud, has allowed large volumes of data to be stored and processed quickly.

The continued rise of the internet is fundamentally about communications technologies and processing power increasing. In parallel, the rise of data has been enabled by new/different storage capabilities and processing power as a service. When you bring all of these things together, we see a seismic shift. It’s why data is indeed the new oil.

Communications infrastructure and remote working

If you think about the world we’re in today – after two years of working remotely and adapting to the challenges that came with this – it’s a testament to our communications infrastructure. Although not without its challenges, remote working was possible and, for some, it continues to be a preferred method of working.

Business-to-business and business-to-consumer is transacting remotely, meaning our digital footprint is going up and the amount of data attached to individuals and organizations has increased exponentially. We have always been building reporting and analysis solutions to unlock data inside organizational source systems. What we’ve seen more recently is a real step change, as we have become smarter around digitizing operations.

Processing power for the masses

We now have processing power as a service with cloud-based solutions and platforms. The barriers have come down substantially around the accessibility of processing power; it is no longer the bastion of huge organizations – it is available to all. Our processing power allows us to run analytics on much larger data sets, that previously could take days to process, in a matter of hours.

A great example of this would be a large on-line retailer. They might make an offer to a particular customer – in the background, they are processing so much more data and so many combinations of life activity to make the right suggestion at the right time via the right channel. That embedded analytic, and what powers it, has become far more sophisticated.

Data is driving opportunity

More data equals more opportunity. What we have done previously still needs to happen, we just do it differently. But there’s more we can do. The good news about what we can do is, it’s predicated on a much clearer business case. We still need to build the reporting solutions and data warehouses for analysis, and couple this with analytics and AI. We can be far cleverer about how we get there and more efficient, based on what we’ve learnt.

Our ability to develop solutions and leverage these solutions in future opportunities leads to a reduction in the cost of implementation. However, our ability to track those business outcomes is hampered. And there’s a reason for that; organizations are always moving to the next thing. We’re always time poor and resource poor – we aren’t spending our efforts determining the value of the solution or something we’ve done. We are constantly looking at what else we need to do to meet a value or cost imperative inside the organization.

The needs of the past have not gone away. We still need to move data between systems and will continue to do so. We still need to have strong operational cadence with all the digitized data – in fact, the need for that has increased. The proliferation of data, the requirement for data manipulation, storage, presentation, analysis, analytics and AI has gone up, not down.

Predictions for the future

Big, established organizations, will refine their digital presence and footprint. They will progressively move to a consolidated public and private cloud environment, that’s fit-for-purpose based on their needs and uses. Operational and downstream reporting analytics will be more pervasive. The whole structure and toolset, and the separation of operation versus downstream will start to blur even more. The other thing we’ll continue to see is a substantial rise in the application of AI.

The cost of entry into the world of data and analytics will continue to decrease, and we’ll see organizations that would have deemed these capabilities too expensive in the past enter the space. In addition, I think we’ll see some medium-sized organizations really start to exploit business intelligence, analytics and AI to be more competitive in the market. There will continue to be non-traditional businesses competing, that have a fundamental business model around sharing and moving data – with zero bricks and mortar presence. To compete and stay ahead of the game, you need to be well along the data maturity curve.

Privacy will continue to be a concern. We have already seen a massive upswing in the importance of cyber security. People will become more comfortable with the public cloud as controls and measures continue to develop. As the generations move forward, there will be a continued move towards willingness to make data available.

However, as the level of trust goes up, cybercrime will become more sophisticated. The investment to stay ahead of the curve on all fronts will still be required. Not only will organizations monetize their data – the next trend is for individuals to monetize their personal data too. We will have more control over our own data. We may well see a future generation who holds and protects their data – not because of privacy concerns – but for the ability to monetize its value.

There are so many possibilities in the data and analytics space, as the need for data manipulation, storage, presentation and subject matter experts grows – and that’s exciting.

The views reflected in this article are the views of the author and do not necessarily reflect the views of the global EY organization or its member firms.