Use Open Data!(Wisely)

Since 2015, here at Blue Bike, we've been curating a growing library of what we call "Sector Intelligence" - valuable dashboards and tools, built on publicly available open data resources, that provide insights and a reliable evidence-base to help strategic decision makers in the Community Sector.

A Story About Metrics That Matter, in the Real World

After the devastation of Tropical Cyclone Pam, I was very lucky to have a chance to spend a few months on an innovation project with Blue Bike, DFAT and Red Cross Australia - trying to come up with new ideas to help Pacific island nations (starting with Vanuatu) recover faster from tropical cyclones.

The goal was to build a technology solution (an 'app') that allowed all the players in a recovery effort - international aid organisations, volunteers, local hardware stores and government - to use a common set of data to answer questions like this.

Right now, where are all the ropes and tarpaulins in Port Villa, across hardware stores, stockpiles & inbound aid supplies?

Ultimately, it was a data aggregation problem/solution, but we needed to put that information on the mobile phones of everyone involved and the audience didn't all have the same level of data-literacy.

One big learning for me, from my time in Vanuatu, was that you can't just dump data on people - you must shape data and insights to fit the audience's level of data literacy - meet them where they are.

And data is only as reliable as the way we choose to interpret it - it's easy, and very risky, to produce the wrong information and create the wrong action.

Are We Making the Best Use of COVID-19 Open Data?

Good data should enable outcomes like validation of a hypothesis, or reliable/timely answers to business questions, or gaining insights into what could happen next.

Bad data, or questionable interpretation of good data, can create fear, uncertainty and doubt - and ultimately, decisions will be made in spite of the facts.

A quick look at our Newscorp media this week saw multiple front-page stories about "Sydney Locking Down Harder Than Melbourne - The Data Proves It". Stoking the fires of some inane, tribal rivalry between Australians is not what's needed right now. We need unity...this isn't State-of-Origin.

"When you look at the objective data, Sydney is staying home more compared to Melbourne in the similar stages of their lockdown last year,” Customer Services Minister Victor Dominello told?The Daily Telegraph?on Thursday.

Publishing 'data journalism' that is based on an unbalanced view of the data landscape is at best lazy and disingenuous, or at worst, propaganda. Is that what happened this week?

Q: WHAT IS "MOBILITY" AND HOW CAN WE MEASURE "REDUCED MOBILITY"?

A: When billions of smartphones are pinging Google and Apple as they move around with people, it creates "Mobility Data". In lockdowns, it's possible to determine things like "are more people staying home than normal?" and "are fewer people than normal at non-essential shops?" (and more).

I'd contend this 'news story' is potentially a case of selectively using some specific metrics to prove a point and ignoring other counter-points - typically not an optimal approach to building a good evidence-base.

Can we do any better with the same data? Well, how about we average the key metrics, and see what it tells us, to address the risk a single metric isn't telling the whole story? And let's use exactly the same method to compare over time, and from one setting to another.

Blue Bike's Implementation - Subset of the Same Data, Different Perspective?

In our State-by-State "Mobility Reduction" Sector Intelligence dashboard, you should be able to see pretty quickly a different perspective to the one in the headlines this week. We've tried to simplify it to make this complex data (billions of rows) easier to interpret. A big green bar means a reduction of more than 40% in people movement in a State, per week, across public transport, workplaces, parks and retail/recreation. The red line is the average number of new daily COVID cases, in a given week.

Why are the insights different?

We don't have the privilege of knowing the formula that State Governments are using to calculate "Mobility Reduction", but we do know it is based on "Google and Apple data".

It's curious that there are two very different perspectives on the same information....one obvious difference to call out - we are averaging over the whole State, not isolating specific suburbs/locations, because we felt that was problematic (which suburbs do you include/exclude when comparing two cities). Averaging over the whole State helps remove that risk, but does dilute the data a bit.

One breadcrumb in the newspaper articles gives us a clue about how the open data metrics being used - the focus is on "increase in residential numbers - more people staying home". But why is there no mention of "decrease in workplaces/retail/transport/parks"?

We Made an Index!

1. First, we'll keep it simple, just the Google data will do. It's a very, very, very large sample. We can reliably compare Googles-with-Googles.
2. Second, which metrics show a 'positive vs negative' effect, that could distort our averages? If we are measuring reduction in mobility, and we are in a lockdown, there are certain settings where we will see a "more people than normal" versus "fewer people than normal". There will be more people at home, and fewer people in workplaces. These might cancel each other out, and obscure the insights (distort the average that matters). We need to address that, before we go any further.
3. Instead of averaging everything, we'll only average the group of "reduction" metrics, and put aside the "increase in residential" metric - logically, there should be fewer people in workplaces, on public transport, in retail/recreation settings.

We can explain the information in our Blue Bike dashboard. We aren't doing anything special to the Google Data, just averaging it out, by State, and visualising it. It took about 3 hours to build and doing some sanity checks, like Victoria in 2020, it looks right.

We can't explain the media headlines this week. We weren't able to recreate an identical view, from what we believe is the same raw data, even when isolating only those 'cherry picked' metrics.

The risk of not "Going Fast/Going Hard"?

We all know what happened in Victoria in 2020. A partial lockdown of 3-4 weeks was followed by a massive increase in cases and deaths, then a long period of "Hard Lockdown". The curve was flattened by a massive reduction in people-movement. There were no vaccines, nobody was immune. And we stuffed up the first phase and paid the heaviest price. We were so lucky it wasn't the Delta variant.

Tomorrow's Fish-and-Chip Wrappers are probably worth ignoring - but I'm worried this distortion is happening in lots of places these days. As we move into vaccination ramp-up, we need to ensure our data, and the way the public are provided with reliable information, is the complete picture.

Bringing it Home for Old Ron

Lovely old Ron, my 90+ year-old neighbour who lives over the back-fence, told me this morning he wasn't getting vaccinated because he didn't trust politicians. I was a bit shocked. We talked about how the stuff that goes in your arm isn't made by the politicians, but by clever scientists. He seemed to feel a bit differently about it after our chat - I'll keep gently encouraging him over the next few days. And I'll keep an eye on anything he's heard in the media that maybe isn't helpful for his survival in a pandemic. Bad data is out there, we need to be wise and help the most vulnerable avoid fear, uncertainty and doubt.

DATA SOURCES, LINKS AND TOOLS

• Google Mobility Tracking
• Johns Hopkins COVID Cases
• Blue Bike - SectorFirst and Sector Intelligence Solutions
• Business Intelligence Platform - Domo

Our Data & Insights team have internally peer reviewed the dashboard above and the data source/transformation process - we can't see any obvious problems. We'd be happy to discuss here in the comments, or offline, any suggestions from data experts in the field, where we could improve our approach.