Why we need to shine a light on dark data to meet the decarbonization challenge

I’m indulging myself by starting this post with a photo of my dog, Deema (any excuse). Between all of us in my family, we must have thousands of pictures of her. Frankly, too many of these are in various costumes that she did not choose to wear. But that’s another story.

Why am I talking about photos of the dog? Because there are so many photos out there on our devices, in the cloud and on social media. You name it, we’ll post a photo (or video) to it. Selfies, food, animals, shots of us enjoying our best lives… it’s estimated that?1.72 trillion photos are taken every year . That’s 54,400 per second, or around 215 images taken for every person on the planet. By the end of the decade, it will be more like 2.3 trillion each year.

Just think of the sheer volume of data involved – it’s eye-watering. How many of these are near-identical images? And how many are never looked at again?

This is a good example of so-called dark data, where information assets are used once and then either forgotten about, or not reused for other purposes. Regular readers will know I have an interest in how we manage data, whether it’s for?personal ?or?business ?use.

What’s the problem here? Well, storing data requires energy, and as we know, a lot of our current energy sources are reliant on fossil fuels in one way or another, which produce carbon emissions. And as we heard from the?COP27 climate conference , which came to a close at the weekend, we urgently need to reduce these.

When it comes to decarbonization, the focus to date has been on major polluters like steel, oil, gas and fertilizers, and in other industries the challenge is to offset or reduce emissions in their operations and supply chains. For example,?T-Mobile Netherlands , where I’m currently Chairman, has committed to moving to 100% renewable energy and reducing materials use and waste. This kind of approach is common across telecoms and many other sectors.

Less common are strategies around dark data and reducing unnecessary data storage. In business, this relates to all the information collected during regular operational processes that is never used – but never discarded. Things like web logs, old email correspondence, raw survey data, maybe even some transactional data. And then of course there are all those notes, policies, tender submissions, presentations and versions of “FINALFINALv2 (amended)” documents that have been sitting in the cloud or on shared drives since time immemorial, in case we need them again someday.

So why do we not take the?Marie Kondo ?approach and clear all of this stuff out? The answer is that a lot of it is unknown, undiscovered, unquantified, or just considered unimportant. It’s estimated that more than 60% of organizational data could be dark, which has led a few companies to visualize the difference between visible and dark data as an iceberg, with the dark data lurking ominously beneath the surface.

However you frame it, it all takes up space on servers and in data centers, consuming huge volumes of power. So it makes sense for all organizations to get a handle on their dark data. With forecasts suggesting that?more than 100 zettabytes ?(that’s 100 billion terabytes, or 100 trillion gigabytes???) will be stored in the cloud by 2025, it’s already been predicted that the data industry will account for?more emissions than the automotive, aviation and energy sectors combined .

Two professors at Loughborough University in the UK have been promoting the concept of digital decarbonization for a while now, and have come up with some tools to help companies start to dig out and deal with their dark data. One is a?carbon cost of data calculator , which estimates your digital carbon footprint based on the size of your workforce. I won’t scare you with the numbers, but suggest you try it out for yourself.

Atos ?also has an approach, centered around assessing the way data is managed, how employees work, decisions around hardware, software and data center use, the way infrastructure and networks are designed, IT usage, suppliers and company culture. Another tool to consider is the product life cycle assessment, which as the name suggests enables you to evaluate the potential environmental impact of a product or service from raw material extraction through to recycling or reuse. Many providers offer these.

It’s important to remember that in itself, digitization is not a bad thing. In fact, digital processes and technologies are making a massive contribution to reducing our impact on the environment in countless other ways, improving companies’ long-term sustainability. But if we’re going to get serious about decarbonization – and we all need to – then the issue of dark data needs to be tackled.

As for me, I’m going to start clearing out some of those repeat photos snapped on my phone. No more double Deema.

Photo credit:?Elias, Pixabay