Back to basics ... AI foundations

These days, barely a day goes by without another announcement about artificial intelligence – where the tech is touted as the answer to everything! Don’t get me wrong, I firmly believe AI will be an increasing feature of our lives going forward. Just as today we can barely imagine life ‘pre-smartphone,’ in the future, we will come to be just as reliant on AI to help us go about our daily tasks.

However, there is one thing we will have to get right if we are to realise the full potential: the data. Behind every AI is a data-hungry algorithm. In order to train the algo, we need to find the right data in the first place. Data, by its very nature, is often full of impurities and problems – hospital/patient-related data is a great example of this. If the data you’ve sourced on specific types of cancer is from a hospital that happens to be in a demographic which is, for example, predominantly white and middle class, then just applying that algorithm to another location with a different demographic will mean it doesn’t work as well. Sometimes that’s easy to spot – a human check after the AI diagnosis may spot when a diagnosis is missed, but the worst case is that the bias is hidden deeply and not spotted at all.

If we are to trust that AI can help us make important decisions, (and I believe a human/machine combination is definitely going to be a better option than a human-one alone), then we will need to get data governance right.

Unfortunately, I'm not sure that people really understand the heavy lifting involved in collection and curation and preparation of data before you get started on the "fun stuff" like AI. Without carefully going through those steps, the risk of unforeseen consequences increases - and carelessness can have very serious consequences indeed, especially in the world of health, but frankly across public sector.

The problem is that these data preparation steps are not the fun part, they don't get the headlines and it is painstaking work, that does not really get rewarded properly. It's also incredibly time-consuming - it can take months and months to get right .. all whilst you are being asked about when we can start seeing the results of the AI.

The new government has talked about some promising initiatives in the data space, some of which will support AI development, but I worry that public sector organisations will be inundated by well intentioned suppliers trying to help them "do more AI" and the only people who can really get to grips with all the pre work around data prep and curation are those who really understand the data. Inevitably this is more often than not the internal teams which means a lot more burden on these overworked teams. Bringing in new or temporary staff doesn't really work either ... finding the right way to get this done will be key to creating trust in AI, and releasing the potential it can bring.

I'm not being negative, I do believe Artificial intelligence can transform our world and I am excited for the future – we just need to make sure we invest in building the right foundations.

Author's note: I am thinking of making this is a new blog series under the back to basics headlines ...