Uber Opens Big Data Treasure Chest And Shares Data Of Its 2 Billions Rides

Uber started 2017 with the surprise announcement that it will begin sharing its trip data for the first time, with the aim of helping municipal, highway and transport authorities to provide better services.

Although it still feels new, the shockwaves caused by its disruptive business model have yet to subside. In fact they are spreading ever further outwards into more and more industries – a process which has even been given its own name of Uberification.

In reality though they have been operating for nearly seven years and in that time have spread from San Francisco to over 60 countries. In the process they have collected a phenomenal amount of data on how the way we get from a to b is changing over time.

I say the move was a surprise because it’s fair to say that Uber have until now been somewhat shy when it comes to sharing this hugely valuable and insight-rich dataset. In New York the company has resisted pressure from regulators to hand over data on journey start and endpoints. Regular taxi operators are required to provide this information which is used to regulate the number of hours’ worked by drivers. Uber has campaigned against a proposed expansion to these rules to cover other vehicle fleets which would include its own, on the grounds of data privacy.

It seems like it may have had a change of heart though – through its newly announced Uber Movement platform, it plans to make available anonymized data collected over two billion trips “to help improve urban planning around the world”.

Of course, rather than a purely altruistic decision, it’s very likely that Uber has a strong business case for making this change. Uber isn’t without competition – rival services such as Lyft are now operating in many cities, often alongside Uber drivers. It’s likely that, given the amount of disruption and controversy which has followed in the wake of the business model’s spread, bosses are fully aware of the value of having a seat at the table when it comes to working with planners and legislators.

The Movement announcement has sparked much debate and speculation in the tech industry regarding Uber’s motives. Harry Glaser, CEO of Periscope Data, tells me “This is a very clear example of what the industry calls a data network effect. As Uber starts to establish a lead they start to have more data than their competition – that enables them to improve the service more rapidly than their competitors can, which enables them to get even more data, and expand their lead even more.”

Direct parallels can clearly be drawn with Google, in the search engine market, and Facebook in social media. Despite the break-neck pace of chance and advancement across the wider tech industry, these giants (and others, such as Amazon in retail) have now maintained strong leads over their competition for many years, thanks mainly to this ‘network effect’.

Innovation is one key to breaking these ‘data monopolies’ – which obliges these giants to spend a significant portion of the revenue they generate financing innovation of their own, so they won’t be overtaken by cleverer challengers and start-ups.

There’s always another key which might be played, too – regulation. And it’s here that it is likely that Uber sees advantages might be gained from taking a pro-active approach to co-operating with authorities. Unlike Silicon Valley giants, governmental departments responsible for planning and delivering highway and transport infrastructure have to make every penny count – so if they are offered freebies as potentially useful as this, they may be more inclined to act favorably towards those who offer them. This might seem like a cynical interpretation of Uber’s motives – but it would hardly be a first for this sort of interaction between regulators and industry.

Another clear advantage that Uber can gain from this is increasing its own visibility in order to attract more talent which can contribute to its core business. There is a well-publicized shortage of professionals with data skills, and understanding of emerging IT fields such as machine learning and predictive analytics. Movement data is initially being released to interested parties such as planners and regulators but the intention, according to Uber’s news release, is sooner rather than later, to make this a public resource. At that point, these data sets are likely to gain a great deal of exposure in colleges and universities where the upcoming generation of data scientists are currently honing their skills.

As Glaser puts it, “People want to work on cools stuff – to be able to say ‘look at all this cool data I’m going to get to work with. So, data scientists are very attracted to the idea of being able to work on these data sets, and software engineers will be very excited about working somewhere that has access to this kind of thing.”

While it’s easy to be cynical about Uber’s motives behind Movement, it’s clear that, putting aside opportunities for self-advancement, there’s a lot to be gained by putting a dataset as comprehensive as this into the public domain. It would cost cities an absolute fortune to collect data with this volume and velocity themselves, so having it available free of charge is likely to drive innovation and change which will benefit all of us in the long-term. 

As always, I would love to hear your thought in the comments below.

Thank you for reading my post. Here at LinkedIn and at Forbes I regularly write about management, technology and Big Data. If you would like to read my future posts then simply join my network here or click 'Follow'. Also feel free to connect on Twitter, Facebook or Slideshare. 

You might also be interested in my new and free ebook on Big Data in Practice, which includes 3 Amazing use cases from NASA, Dominos Pizza and the NFL. You can download the ebook from here: Big Data in Practice eBook.