Prioritizing AI Efforts - Part 1

Recent research suggests that AI is a general-purpose technology i.e. a technology that will evolve rapidly and impact virtually every industry (think electricity, computers, etc). However, for executives, this poses a unique challenge in determining which AI innovations to prioritize and how to focus on them, given the constant influx of new AI innovations to which they could potentially jump.

This question came up during a recent conversation with executives from the music industry. In this post, I’ll share my framework for prioritizing AI initiatives. In Part 1, I’ll focus on “discriminative” ML models, the kinds of models that are great prediction machines and help with tasks like identifying spam or fraudulent transactions. In recent months, Generative models have got a lot of attention. These models don’t just predict but use it create new data/content. I’ll present a framework for that in part 2 of this post.

Coming back to the challenge, we must first identify a super set of initiatives or focus areas, which can come from an innovation tournament with employees or expert recommendations, or senior leadership discussions. Once we have a list of potential initiatives, the key is to prioritize them based on the following three criteria.

The first criterion is to lead with strategy, not AI. Many firms make the mistake of chasing the next shiny AI use case without considering their strategic priorities. Instead, AI initiatives should be tied to key strategic initiatives or ways in which the company adds value in the industry. For example, a Hollywood media company's AI team was working on an initiative to automatically identify similar movies given a screenplay (identify “comps”). In contrast, the C-suite was focused on poor retention rates of their subscription service, so the AI team should have focused on acquiring the right kind of users, personalized recommendations to engage subscribers, better churn prediction, and generating retention offers and incentives.

The second criterion is the value of faster/more accurate predictions. Machine learning boils down to getting faster and cheaper predictions than if you had humans do the same task. Therefore, we should directly ask what fast or more accurate or cheaper predictions will do in the context of the workflow. Sometimes the answer is “not much,” and the AI initiative may not be worth pursuing.

The third criterion is having the right data available at high volume.

We must look at the intersection of these three criteria to identify the projects that should be in our portfolio - the long-term bets that will produce financial results.

For example, credit card fraud detection is a strategic initiative for credit card companies as they lose billions of dollars annually to fraud. Early detection is valuable, and they have historical transactional data to mine for predicting fraud. On the other hand, predicting how successful a movie will be is one of Hollywood's biggest prediction problem, but there is a huge missing data problem. Because out of millions of scripts, very few actually get made into movies. number of movies per year doesn't add up to the data scale where machine learning usually thrives. Another challenge is that cultural tastes change often so the data from five years ago is not as useful as it might be in other settings where consumer behavior may be relatively more stationary. So, I'd rather using A/B testing and modern market research methods than apply ML on a limited dataset.

By following these principles, we can prioritize our AI initiatives and focus on the ones that are most likely to produce the desired results. In part two of this blog post, we will discuss generative AI and how to approach it.

I’d love to get your comments. Have you been trying to prioritize AI efforts amidst a never-ending influx of new AI innovations(GPT3, chatgpt, MS integration, AutoGPTs, open-source solutions, new plugins, etc). Does this framework help sort the initiatives? Is it missing something?