Setting AI Business and Product Goals

After reading about Conversational AI and learning about LLMs, a couple of questions came to my mind. With businesses getting on the AI wave, how are they setting their goals, especially the product goals? What kind of impact they are focussing on?

Polly Allen and Rupa Chaturvedi mentioned that goals should be SMART - Specific, Measurable, Achievable, Relevant, and Time-bound, and consider technological capabilities, user needs, market trends, and competitive factors. This is no different for AI products!

So, let’s dive in!

Business Goals

1. Dollars Earned: This can be done either by creating new revenue streams or enhancing existing ones. We can consider metrics such as increases in sales, expansion into new market segments, or improved monetization of existing services.
2. Dollars Saved: Think about ways to save dollars with AI - reduced workload, streamlined processes, automating mundane tasks. Metrics include time saved, reduced call time for call center operations, or reduction in human errors.
3. User Growth: Here the focus is on understanding whether the AI solution can help us enter new markets or segments, or increase market share in existing ones. This will involve analyzing competitor offerings and identifying market gaps. Metrics include market share percentage or user acquisition rates.
4. Engagement and Retention: Can AI help increase engagement and retention? We can consider metrics such as the number of interactions per user (engagement), session length, frequency of return visits (retention), and user lapse (users who stop using the product for a set time, say 30 days).

Product Goals

This includes enhancing the user experience and focusing on user needs and expectations from the product. Take a minute and think about what user goals you can come up with. I was thinking that if as a Product Manager, my team and I are building a Conversational AI product, I’ll have these goals - user adoption, time spent, and conversion rate. However, Polly and Rupa suggested - “Your product goals should reflect your hypothesis about which user benefits will lead to the desired business outcomes.”

In simpler terms, we should first identify what benefits our product offers to users, and then define goals that focus on how those benefits will translate into success for the business. The metrics that can be mapped here are -

1. User’s Time Saved: This will include - less time waiting for customer support, fewer turns or less time to complete a goal, and delegating research to an assistant. These goals are usually expressed in a percentage reduction in time to complete a task or number of turns or steps.
2. User’s Money Saved: This will include - helping users find cost-effective solutions, such as identifying the best deals or comparing prices. These goals can be articulated as a percentage of cost-saving per transaction.
3. Emotional Benefits: These can be quantified through metrics like Customer Satisfaction (CSAT) survey scores, the number of social shares, or engagement rates with community features.

My absolute favorite part of learning about product goals was this ??

While the business goals articulate what business outcome would be considered successful, product goals articulate why a user would act in ways that help us accomplish that goal, and how they’ll achieve that benefit. For example: If a chatbot’s business goal is to increase user adoption - thereby providing user growth - why would a user adopt this experience? Is it the fastest, most efficient experience for a particular purpose? Is it the cheapest? Is it the most friendly, engaging experience? Which of these benefits is most likely to resonate with users?

These questions will help us think backward from the gaps in the market and our go-to-market messaging. They also help us decide if product goals should be set around UX flow efficiency (saving users time), development and delivery costs (saving users money), or user satisfaction and sentiment (emotional benefits).

Technical Goals

Technical Goals were the most obvious goals that came to my mind when I was thinking about this topic. These will be done in collaboration with engineering counterparts and will include -

1. NLP Accuracy and Relevance: Remember when ChatGPT was launched and it gave some weird answers when people asked confusing questions, well we don’t want that to happen to our product, right? Well, to measure this, we can keep a tab on user feedback and error rate analysis in user interactions.
2. Response Time: When reading about AI, you must have heard the term ‘Latency’. In AI, latency is the time it takes for an AI system to process and respond to data. Engineering teams can consider decreasing overall system latency by a set percentage here.
3. Data Security and Privacy Compliance: We all know that emphasis on data security and privacy is super important. Security audits and compliance checks would be the metrics we want to look into here.
4. Error Detection and Management: Understanding where the errors are happening and how effectively we can tackle those errors is crucial. Error logs and user incident reports can be the proof of truth for this.

Did I miss out on anything here? Would love to know what you all think :)