Why does Jeremy Corbyn's Labour Party need valid data?

Most of the news stories about VW’s emissions scandal have focused on the impact to the environment, the motorists and the resale values of the affected vehicles. In the last couple of days I’ve started to focus the story a little differently. I have had some very different conversations with my colleagues about the issue: one of data validity.

It’s important that we also bring the validity of the data into sharp focus.

Gartner’s “3 Vs of Big Data” has taught us to focus on the volume, the variety and the velocity of “big” data. In the work we do as commercial data scientists we need to focus on the value that the data can bring but, in light of what is happening at VW, it’s important that we also bring the validity of the data into sharp focus.

According to reports, VW’s EA 189 EU5 diesel engine has a defeat device fitted which senses when the car is being tested and changes engine performance to reduce emissions. The result of this is that the test data provided to the authorities is not an accurate reflection of how the engine performs on the road.

The authorities are using this data to check that the vehicles meet emissions regulations and they are making those decisions based on incorrect data. VW owners around the globe are now wondering whether their vehicles do, or don’t, meet legal requirements.

You trust the data you see on the pump screen.

If you own one of the affected cars, it’s likely you’ll be filling up your tank with diesel once every 500 miles. You trust the data you see on the pump screen when it says you’ve put approximately 50 litres of diesel into your tank. You pay for your fuel and go on your way.

You can trust the pump because, by law, there should be no more than a 2p error on the cost of the actual amount of fuel you put in your car and what you’ve paid. Trading Standards units around the UK enforce this to ensure you get what you pay for.

On Tuesday, Bloom’s CEO Alex Craven, spoke at the Wuthering Bytes conference to showcase the work that we’re doing to apply our innovation in data science to map how individual communities are connected together across cities in the UK.

We can understand the areas that need civic support and then analyse the impact of policy decisions in real time.

The aim of our work is to provide city leaders with a measurement toolkit that allows them to see how a city is responding to external influence and activity. By using social data to identify the strengths and weaknesses of a city we can understand the areas that need civic support and then analyse the impact of policy decisions in real time.

One policy area getting a lot of coverage this week is Jeremy Corbyn’s commitment to embark on a massive council house-building programme.

If this commitment becomes reality, it will impact a number of discrete “things” across the UK. It will have a significant impact on the lives of the eventual residents of the housing, the communities in which they live and it will bring opportunity, growth and innovation to the organisations, suppliers and manufacturers involved in building the houses in the first place. As the construction industry prepare to share best practice during UK Construction Week next week, I’m sure this will be a hot topic for debate.

How can you show how a complex system responds to a set of decisions?

Away from votes and conferences, how do you measure the true impact of a policy shift like this? What does success look like and how we can we really measure the impact of one decision on such an interconnected web of complex connections? How can you show how a complex system responds to a set of policy decisions?

How useful would it be for Jeremy Corbyn’s invigorated Labour Party to see just how many fragments of UK society, economy and infrastructure are positively or negatively impacted by a single decision?  Social data, from Twitter, Facebook and Linked In, can provide some of the colour we need to answer these questions.  

All of this requires accurate, or valid, data. We’ve known for a long time that a large proportion of Twitter data is created by automated accounts. Back in 2012, I wrote an article explaining how spammers had “attacked” a technology conference. Today, things are better than they were, but spam still makes up at least 25% of the conversations we analyse.

Data scientists around the world are making checks on your data so you don’t have to.

To ensure that civic leaders are making decisions based on valid data, we invest a lot of time developing new algorithms to remove the inaccuracies in our data so we can expose the truth and not the noise in our data. We have to build new algebraic methods to help us do this quickly on such large volumes of data.

Like the trading standards officers who check the pumps without you noticing, data scientists around the world are making checks on your data so you don’t have to. The impact of this process falling down in the car industry, or the impact on Corbyn's policy direction, is yet to be seen.

Image Credit: Rama (from Wikimedia Commons).