How to set expectations for your AI project

Do you have clear expectations for your AI project and the means to measure and evaluate what happens?

So much attention is paid to “AI” that we forget most use cases depend on structured data already helping measure how our business is performing. Sales, margin, productivity, and efficiency metrics all depend on data that shows executives what’s happening in their business.

Most AI use cases are not initially framed around sales, margin, and other business metrics, but rather goals like personalization, recommendation, recognition, and automation. It’s all areas within our business we look to improve where the subject of AI comes up, not so much the end result we are looking for.

Companies can improve in a multitude of ways, and prioritizing what is most important to your business means looking at your key financial metrics, how they performed historically, and your goals leading into the future. This way, you have a baseline to evaluate the impact of AI, determine what’s working and not, and establish improvement plans as needed.

This also happens to be how you should prioritize your AI use case opportunities before you even get started.

That’s why before you head down any AI use case path you should:

1. Ensure you have the means to establish a baseline from which you expect to improve.
2. Have the data to support the baseline analysis; data that is updated at the right frequency and is available in the right format. Data that connects actions performed by AI to business metrics. If you don’t have this, then it’s unlikely you can support the use case.
3. Be capable of conveying use case results in business metric terms to the executives who fund and depend on the AI. There is no best practice example of a company that achieved breakthrough results without executives who understand the difference AI can make to their business.
4. Plan for the eventuality when one of two things happen. Either results fail to live up to expectations so you must make changes or retire the use case and leverage the learnings into the next use case in your roadmap. Or, results are positive and your executives seek to improve them further, and so you must build upon what you have.

For example, if you plan to improve sales through greater personalization, you must be prepared to isolate and measure the value of customer interactions impacted by the AI model’s output. If you succeed, you will find yourself asked to look for ways to leverage this into improving margin or retention.

Many companies continue to struggle with data quality issues and operational reporting. See how well received cloud data warehousing company Snowflake was by the public market for validation that there is still huge potential in helping executives get a handle on what’s happening in their business.

On the flip side you have companies like Salesforce with Tableau providing the means on top of data to extract insight and share it in the most optimal form so executives can see what is happening and if necessary employ human intelligence to take action.

If you are new to this, understand that most of the first AI use cases a company pursues are extensions of existing analyses. For example, using something like an RFM score to segment customers and understand their offer response propensity to improve conversion. This method does not require a machine learning model and is widely used. Taking it a step further, machine learning could surface unobvious attributes of these segments to improve conversion even more through better informed content, or identify entirely new segments that need consideration and unique treatments that lead to incremental sales.

The advantage of looking at existing use cases is that you have the baseline data and reporting in place to track progress and hopefully, real business improvement. Experience and success begets more value, which in turn can confidently fund ever bigger and more impactful use cases.

Speaking of business improvement, I like the following chart from McKinsey showing common analytic use cases and the sales uplift, margin improvement and cost savings associated with each based on a survey of 100 plus companies. You can use information like this as a first step to prioritize the AI use cases you think apply most to your business and later compare results.

What happens to businesses that do these things well, is they develop a mindset of continuous improvement. The best companies do this seemingly by nature, but they started somewhere. It began with a strong foundation to monitor and manage the processes and actions that impact their key business metrics.

For more about these ideas, I suggest you check out Getting your Analytics House in order for times of crisis to see how some organizations were able to adapt to changes wrought by COVID-19.