Drowning in Data?

We all know how important data has become – it now drives everything, and is reckoned to be the most valuable commodity on the planet – and of course can be a force for good as well as not so good!

The problem I have with it is where is it all kept? And at what cost?

Take your phone for example. Mine currently has 4,885 photos on it, most of which are of my dog (a constant source of disappointment for my 2 children!). And almost regardless of how many photos I take it’s not an issue – for 79p a month they all get backed up to my iCloud account and I don’t have to think about it.

Which is all well and good, apart from when you start to look at the impact of data centres. They currently use 2% of the world’s electricity. It doesn’t sound much but in terms of energy consumption that’s more than some countries and puts it on a level pegging with the aviation industry, and we all now accept that flying isn’t such a good thing after all. Current projections suggest that by 2024 data centre energy consumption will exceed aviation by a factor of 5, and it’s estimated that by 2030 data centres will be using over 20% of the world’s electricity.

All of which means that we really ought to be giving more though to what we keep and what we don’t – surely keeping it all simply isn’t an option. Which therefore begs the question of what to keep and what to delete.

In our case, doing video production, we’ll typically generate 1Tb of data from a day filming. Of that probably a 75% is what you’d term “usable” footage – it’s the best bits from an interview, the best bits of B-roll, and of this maybe 15% would make it into the final edits.

You’d clearly want to keep the final edits, but what about the other 60% that is usable but that just didn’t make it into the edit?

We are big fans of making the most of what we’ve shot through re-purposing footage for social media edits, and footage shot for one project can often find uses as footage in another project for the same client, rather like a self-generated stock library, so it’s certainly worth keeping, but for how long?

Is there a sensible lifespan or value cycle? It’s probably at it’s peak value as soon as it’s shot, and will remain so for maybe 5 years, depending on the sector. After that its value drops as it all looks a bit tired and outdated as it ages, until after maybe 20 years it becomes properly vintage and regains it’s value – a bit like finding a box full of old 8mm film in the attic. And what about the footage of the student we interviewed when they were an undergraduate in 2005, who’s just won the Nobel Prize?

Part of the problem is that by nature we tend to be horders, and we only throw things away if we really have to, usually as a consequence of running out of space, but while we have what is effectively an infinite sized cupboard there is no incentive to sort out what we keep and what we don’t. It is of course a tricky one to predict future value or worth of data, but it is something that must be addressed, unless we’re going to drown under a sea of unneeded data.

Because the truth is, of the 4,885 photos in my iCloud, do I really need 2,855 photos of the dog?

Reference: Nature 561, 163-166 (2008)