The ever-changing face of the data analysis skills market

I was delighted to be interviewed by KPMG as part of Learning at Work Week. The ODI is a partner of KPMG, part of the consortium of learning providers that work with their Learning Services business. Here is the interview in full, which was shared by KPMG on 19th May.

---

We started off by asking Stuart about what’s proving particularly popular in terms of data analysis skills training right now:

Data analysis skills are hugely in demand right now, with almost every organisation apparently short on them. These skills can take many different forms; everything from the complex analysis of massive datasets to the more rudimentary data skills needed to work with spreadsheets and other everyday tools. There’s just not enough capability in these areas.

In recent years, much of our training offer has revolved around traditional data skills, providing instruction on how best to work with data. What’s changed in the last 18 months or so is that there has been huge interest in our training offer around data ethics.

The rapid developments within generative AI have fuelled this. Data is the feedstock of any AI system – and it consumes a lot of it! Therefore, the rise of AI is bringing added scrutiny to how ethically that data is collected, prepared, analysed and used.

Over the years, we’ve developed a number of tools to help people work with data. One of those is what we call our ethics canvas. It’s a free framework, with accompanying guidance, that explores how to identify and manage ethical issues throughout the lifetime of a data-driven project. The page it can be downloaded from is currently the second most visited page on our entire website; which rather reinforces quite how hot a topic this is right now.

How do you see that interest in data ethics developing??

As part of the journey we’re now on, there’s going to be a different distribution of tasks between humans and machines. Mistakes will inevitably be made along the way and some bad things done with AI.?

But what might emerge from this is a whole new industry of people working professionally and responsibly with data. Just as the financial services sector created its generally accepted accounting principles (GAAP), this industry will hopefully do something similar for data and how it’s used within AI. We hope to play a part in that, helping organisations to put in place the foundational skills needed to manage data responsibly and shape new governance systems.

That GAAP analogy was made in a recent article from respected industry thinker, Tim O’Reilly. In calling for greater disclosure of what’s going on behind the AI curtain, he made the point that you can’t regulate what you don’t understand. It’s a sentiment that applies to leadership too. It’s hard for leaders to keep pace with some of the concepts involved in data analysis currently. If this represents something of a black hole in their knowledge, then that creates huge amounts of risk.

We do a lot of leadership training – and it’s noticeable how these courses are now attracting more and more senior people. I think there’s a growing realisation among senior leaders that they need to understand more than just the basic concepts of data handling and analysis. They’re responsible for managing teams that are working with data day in, day out. To do this successfully, they appear to be acknowledging that they need to have a deeper understanding of how best to use data.

Seeing as you’ve mentioned leaders, what characterises the conversations you typically have with them about skills?

Any conversation with senior leaders about the data analysis skills their organisation needs always begins with a discussion about their business strategy and what they hope to achieve. From there, we work backwards to determine the skills they have most urgent need of and the training that needs to be prioritised.?

But we also try to get them to think differently about data. We encourage them to think of it as part of their business infrastructure, rather than a resource that needs to be accumulated and then bled dry in the search for decision-making insights. The latter approach is still prevalent among many organisations. It’s the legacy of the early years of big data when people treated it like the new oil. It’s an analogy we really dislike.

If you see something as infrastructure, rather than a resource, you think about it differently. You invest in it; you set standards around it; you treat it differently than if it was just a resource that’s waiting to be consumed. That’s not to say that organisations have to spend vast amounts of money on this. But they do need to continually – and carefully – invest in this area if they want their data to perform properly and to provide maximum insights and value to the business.

Last year, we created 97 zettabytes of data globally, up from 6.5 a decade earlier. If they can even get their head around what a zettabyte is, most leaders will admit they don’t need that much data. And they’re right of course – but what they do need is a foundation of sound data infrastructure that’s relevant to their business and that makes data available to their employees and customers in a consistent and reliable manner.

And how do they respond to that?

It always creates a good debate. There’s rarely any pushback. What can be a more challenging conversation however is when we talk about data sharing. A lot of the conversations we have in this space are around government or businesses opening up more of their data to be more transparent, to build trust and boost innovation.

The pandemic had something of a silver lining with regard to data sharing, especially across government where there’s always been a challenge in sharing critical administrative datasets between departments. Covid obliged departments to look at how they could link some of their data to help drive policy decisions during lockdown.

The benefits of continuing to share data in that way could be enormous. Think of the national census, which takes place every ten years and costs a lot of money. There’s an argument here that 60-70% of the census could be done at the press of a button if government datasets were more joined up.

What’s your final pitch to leaders who maybe aren’t as enthusiastic about data skills as you are?

The volume of data and the adoption of AI systems is only going to keep on escalating. We’re therefore going to need to develop new skills to build, manage and lead the teams that can work with that data and AI. Used correctly, and supported by responsibly governed data infrastructure, this can help people to live even better lives and tackle society's most pressing challenges.??

Leaders need to get onboard with this now. They need to build up knowledge and expertise. They need to work with other leaders. They have to be willing to admit to what they don’t understand. They’ll need to be bold and brave but also mindful of what makes us human and how we can use data and technology to elevate that, not diminish it.