AI Due Diligence: how to pick the best AI cases

The field of data analytics & AI has matured greatly during the last decade. Organisations have made significant investments in new technical infrastructure, architecture and employee competence to position themselves to drive value through data collection and analytics. Many have experimented with new technologies and run machine learning and AI projects. The question is: how do you go from experimenting with AI to scaling it to become an important business value driver?

Through working with customers in different industries and of different digital maturity levels, we have identified several criteria that often cause AI initiatives to fail to deliver the value desired. Using that insight we developed a framework that helps organizations set business driven AI strategies, build initiative roadmaps and deliver AI solutions that create value anticipated. We call this framework AI Due Diligence. This article provides a high-level overview of this framework.

AI Due Diligence is a framework organizations can use to identify, evaluate and prioritize AI opportunities in a structured and efficient manner. The process starts with mapping tasks performed by employees or customers within a chosen business area. The goal is to identify tasks that demand decision making and generate hypothesis on how AI could be used to support that decision making process. Once the long list of hypotheses of potential AI use cases is created, we evaluate them through a three-fold due diligence process:

Business Diligence

First we check whether the identified AI use case supports organizations’ business goals and can be succesfully adopted. By doing so we avoid investing in initiatives that are purely technologically driven (AI solution looking for a problem) and instead prioritise initiatives that are business problem driven (problem looking for a solution, which might be AI). We do so through checking our potential AI use cases against a set of questions. Here are some of the questions we ask:

• Will the use case support companies’ strategy?

Connecting an AI use case to a strategic company goal will help you align your work with organizations’ focus areas and help you get the management suppport

• Will it create business value? For who?

Define clear business contribution the AI use case will have. Will it help reduce costs? Increase revenue? By how much? Who in the organization will benefit from it? Understanding this will help you idenitfy the important stakeholders you need to work with to succeed with the initiative

• Will it create qualitative effects?

Will successful implementation of this AI use case impact employee satisfaction? Improve quality? Have a positive effect on environment? Articulating qualitative effects can further increase the needed support for your initiative

• Will it be supported by the users?

It is necessary to include the future end user of the AI solution early in the planning and design process. I have worked with many companies who implement AI solutions which end users refuse to use, either because they do not trust the model or because it does not fit into their work process. User adoption is a critical success factor for any kind of AI initiative. You cane secure it by involving the user early on.

• Do we need to explain the outcome?

Agreeing on what the model output must look like is an important step in the planning process. Is displaying a model score/prediction enough or does the model need to provide the context to the output? Model explainability is an important topic to discuss at this stage. An answer to this question may put restrictions on the algorithms and methods we can use to solve the problem, which might affect our choice to go ahead with implementation.

• Can we measure success?

It is essential to agree on what success will look like. What is the needed model accuracy level that will instill the needed trust with the end users? How much business value does the AI use case need to create in order for the decision makers to support its development over longer term? These are important metrics to agree on in order to align stakeholder expectations and set a clear investment plan for the AI use case

Technical Diligence

Once the business case is clear, we check that the problem we want to solve is suitable for AI and if so, whether solving the problem with AI is technologically possible. We evaluate the state of the data needed to realize the case, as well as the infrastructure and competence needed for developing the solution.

Legal & Ethical Diligence

Way too many initiatives are shut down due to late Compliance involvement. Making sure your case is both legally compliant and ethical will ensure that solutions are designed in a way that creates the digital trust with your customers — a critical factor for sustainable competitive advantage in the future.

Way forward

Once you run the identified AI use case hypotheses through AI Due Diligence framework, you will be left with a short list of evaluated and priotitized AI opportunities. You can use those to create a roadmap. Through this process, you will also establish an in depth understanding of the potential risks related to these opportunities and be more capable to mitigate those risks going forward. The qualified roadmap of the AI initiatives can be further used in development and operationalization of your AI strategy.

Do you have comments or questions about AI Due Diligence framework? Let’s discuss!