AI in Product Management: the AVOCADO Approach.

Whether you're a senior executive, product manager, or product leader, products and services are part of your company's core capabilities. Increasingly, AI is at the heart of these products and services. You need to understand how AI can be leveraged, the challenges when using AI, and how to operate AI-based products. This post discusses developing an AI-focused mindset when creating products using the AVOCADO and GOAT approaches.

The rise of AI

Before diving into AI and product management, we need a bit of context about what AI is and why it has become so relevant these days.

Figure 1 shows an overview of the different types of AI as well as several of the intersecting domains. This is not meant to provide an exhaustive overview but to highlight the main terms that you frequently hear/read about in papers or articles. In Figure 1, Generative AI such as ChatGPT can be represented as a Generative, Unsupervised, Reinforced learning and Pre-Trained Transformer Large Language Model. Other Generative AI tools are DallE, Stable Diffusion, and Midjourney, which follow the adversarial model instead of the Large Language Model.

Figure 1 does not depict the richness of the many models that are available across the different types of AI/ML. You can see a collection of some of the best-known large language models and their (parameter) sizes in Figure 2 to get some idea of the variety of models for one type of AI/ML.

There are several reasons for the rise of AI in recent years:?

Advancements in Technology: There have been substantial advancements in computing power, data storage, and algorithm development. These technological improvements have created more sophisticated AI models and applications.

Data Availability: The proliferation of digital devices and the internet has led to an explosion of data. AI thrives on large datasets, and the availability of massive amounts of data has enabled AI systems to learn and perform better.

Cloud Computing: Cloud computing platforms have made it easier for developers to access powerful computational resources without significant upfront investments. This accessibility has accelerated the development and deployment of AI applications. Pretrained AI models, such as GPT-3, have made it easier for developers to create sophisticated applications without starting from scratch. These models can be fine-tuned for specific tasks, saving time and resources.

Technology, data, and the Cloud led to an explosion of feasible use cases and applications. The end of the article contains a (non-exhaustive) list of use cases that are enabled or enhanced by AI and showcases the broad applicability of AI.

Today's most prominent commercial use cases are advertising (personalized ads powered by AI) and e-commerce: shopping from search to payments is powered mainly by AI. This is because e-commerce and search companies have a tremendous amount of consumer data, and (good quality) data is vital in developing good AI products.

Product Management and AI

Different (r)evolutions, such as personal computers, the Internet/Web-based, and mobile-first, have influenced the product strategy and the role of product management. Before 1990, your digital product would have shipped via floppy disk, while in the early 2000s, the company likely would have opted for a web page as a portal to the service or product. Today, mobile platforms like iOS and Android must be considered when launching a digital service (mobile first).

As a revolutionary technology, AI is no different and can have a profound impact, especially on data-driven products and services. Traditionally, these data-driven products focused on packaging and presentation of data with some level of projection and extrapolation. With AI, especially Generative AI, this data can be ‘turbocharged’ with the capability of finding new relations and insights, reducing manual tasks and unprecedented levels of personalization. ?

Within the product design and development phases, you need to understand what role AI can play and how it can be used to differentiate a product. Like the mobile-first mindset, product teams will benefit from an AI-first mindset. The following sections describe the AVOCADO (Aim, Vision, Objective, Capabilities, Algorithms, Data, and Observation) approach that can help in cultivating an AI-first mindset:?

Aim, Vision, & Objective: What are you aiming for with the (AI) product, and how does it relate to the larger vision? One must strike a balance between oversimplified objectives and narrowly defined objectives. Small nuances can lead to wrong outcomes and even become intrusive.

Take, for example, the case of a retailer that used predictive analytics to send coupons to its customers. They sent diaper discount coupons to a young woman still living with her parents. Seeing these coupons, the parents were upset with the retailer, only to discover that their daughter had not told them about the pregnancy. The objective was to 'hook' expecting parents early, but it felt uneasy to the customers, and for some, it was creepy.

The objective typically results in a combination of outcomes. For example, if an e-commerce company wants to increase revenue, it can do that in different ways. Should they increase the conversion rate, optimize for a lower risk of return, or send out coupons?

Try also to understand what can go wrong when pursuing an objective. For example, if a social network wants to increase engagement, it can look at how many people like posts and how many times they are shared. But this might lead to algorithms selecting for hate speech, scientifically inaccurate stories, or things that are simply not true, as people might just blindly forward posts with attractive headlines without any considerations. One can (partially) mitigate the risk by analyzing if posts are read completely, lead to quality comments, and are shared more deliberately.

Whatever objective is chosen, throughout the product development, use data to validate hypotheses about the objective and be careful not to confuse correlation with causation.

Capabilities:?What capabilities are available in the company or from 3rd parties? Is it possible to use existing (AI) capabilities, build on top of existing models, or use existing (cleaned) data sets? How do (public) models impact the competitiveness of the AI-powered product? You can start from so-called pre-trained models. GPT (Generative Pretrained Transformer) generalist models are already trained on broad public knowledge. Using pre-trained generative models, as is, can help with a fast roll-out, but one has to be aware of the accuracy and cost. In the long run, building your own foundational model or fine-tuning these publicly available models to fit your data better and creating specialist versions that provide a competitive advantage might be more cost-effective or efficient.

Algorithms: What algorithm should be used, and what data serves as input for that algorithm to build a model?

As depicted in Figure 1, there are different categories based on the type of learning and approach taken.

• Supervised learning. You can label/bucket your dataset (spam vs. not spam)
• Unsupervised learning. There is no pre-defined classification, and you need to classify based on observations (online recommendations use this a lot)
• Reinforcement learning: Trial and error approach. Adjust based on actions taken (navigation, robotics, self-driving cars)
• Generative: Can generate new data points. These models are usually used in unsupervised machine-learning problems.
• Deterministic: These are used for supervised machine learning. Often, they do ‘literally’ what their specific names imply, such as separating the data points into different classes or learning the boundaries using probability estimates.

Besides the algorithm, you need to decide what variables (features) will be used for your algorithm. For example, an algorithm for an e-commerce site might use the price, brand, and shipping cost as features to calculate the chance of conversion.

Cost and speed to market can play a role in deciding what approach to take. Generative AI is capable of doing classifications. It can give you considerable flexibility and shorten the time to market, but in many cases, it is more expensive than dedicated classification approaches.

Finally, you might want to limit the flexibility of the algorithm. Self-driving cars should obey the speed limit and not drive on the wrong side of the road, while in a social network, you might want to restrict hate speech. Some restrictions can be managed by providing the proper training data, while others must be specified as rules.

Data: Where is data located, and what is the quality? An AI algorithm is only as good as the data it gets. AI learns from examples and patterns, and giving it the wrong examples and patterns will lead to wrong outcomes: garbage in means garbage out. Sometimes, data is available in organizations, but it might not be in the proper form or be incomplete. That means an intermediary ‘cleaning’ step is needed to ensure the data input to the algorithm is good enough.

While data is the lifeblood of AI, it needs to be refreshed from time to time. ?Even if you have ‘perfect’ sales data from the last 20 years, that data might be less relevant than the sales data of last year or even last month simply because people’s preferences change. Large pre-trained models might not be sufficiently up-to-date to be useful for the intended tasks. AI data needs to be refreshed, and there needs to be a continuous cycle of data collection, data cleaning/preparation, training, and learning. Data is at the heart of the AI flywheel effect: good data can lead to better-performing algorithms and products, which in turn leads to more users that generate more data to improve the product incrementally.

Observation: How is the algorithm/product performing? Can it be improved? It is important to understand that often, the AI model might not be good enough from the get-go. AI, by its very nature, is probabilistic. It can require time, patience, and experimentation to improve the product to a desirable level. Getting it 100% right is often difficult, and a desirable level could mean that in 60 - 80% of the cases, the AI model provides the best possible answer.

There can be different reasons why the performance of an AI model is lagging: The data quality may not be sufficient, test and training data can be misaligned, or the algorithm's hyperparameters need to be better tuned.

Besides data or algorithm tuning issues, other aspects need to be monitored: Are there biases in the algorithm/product, or is there the risk of copyright or privacy infringement when using data to generate something new?

Deep learning algorithms typically outperform rule-based systems, but sometimes, there is a strategic need to override the deep learning algorithm. For example, you might not have enough data to train AI and need to bootstrap it with rules at the beginning, or there are special cases like breaking news on social networks or Black Friday deals on e-commerce sites.

Generative AI is creating new challenges as it can very convincingly provide you with false answers or strike the wrong tone not aligned with the values or style of your company. When using Generative AI, it is essential to test the output of the service on things like honesty, truth, tone, style, authenticity, etc…

Observation happens across multiple stages during the design, testing, and operation of the model. For each of these stages, you want to have the right processes in place to gather feedback and incorporate this feedback in new versions of the model better aligned with the aim, vision, and objective. There is typically an iterative feedback loop to improve the initial version of the algorithm and associated data, as depicted in Figure 3.

Beyond a single AI product

Given AI's broad applicability and potential (see the Use Cases section), it is important for a company to start thinking about a framework to manage AI on a company level and manage the risk of fragmented AI activities sprouting up within different teams. The GOAT approach (Governance, Oversight, Awareness, and Technology) can help with that.

Governance

• Set up processes to reduce the risk of liability (inappropriate responses, false output, etc…), and privacy, and establish quality guidelines.
• Consider the legal context: different regions/countries have different legislation on topics such as privacy and security. What are the risks of releasing an AI product w.r.t. to regulations such as GDPR or other restrictions?

Oversight?

• Establish an Oversight Committee to align, coordinate, and connect different AI projects across the company, breaking down silos between different groups & teams. Different initiatives can sprout up in different teams. Without oversight, it becomes harder to exploit synergies and share learnings.

Awareness

• Train teams. Are teams aware of the potential and risk of AI, and do they understand how to use it within product development and their daily activities? This awareness can go beyond product management but can also include the use of (Generative) AI and prompt engineering for work-related tasks and associated restrictions.

Technology Platform

• Provide an infrastructure for model and feature lifecycle management
• Lower the barrier to accelerate the use of AI via low/no code capabilities.
• Support workflow management from data to outcomes

You can take a look at platforms from Cloud providers like Azure ML Studio to see what such a platform can look like.

AI Use Cases?

This is a non-exhaustive (short) list of AI use cases that showcase the wide applicability of AI across many industries. These use cases demonstrate why AI is often considered a disruptive technology.

Natural Language Processing

• Customer Service: chatbots provide instant customer support for common queries and tasks, improving customer experience.
• Sentiment analysis
• Language Translation
• Text summarization?

Computer Vision:

• Image and video recognition
• Object detection and tracking
• Facial recognition

Industry Cross cross-cutting generative AI (prompt engineering*) to accelerate productivity.

• Text/Content generation. e.g., 'write a 5 min. A speech about climate change and its dangers.' 'What are the views of Aristotle and Plato regarding the individual?'
• Data interpretation. e.g., summarizing large data sets and creating reports, detecting trends, patterns, and gaps, and suggesting features or new combinations of features to help your customers sift through many options and support rapid product development.
• Coding. e.g., write 'Python code to parse addresses from a text file.'
• Image generation. e.g., 'Show me a monkey in a spacesuit.'

Healthcare

• Disease Diagnosis: Help doctors diagnose diseases by analyzing patient data and symptoms.
• Drug Discovery: accelerates the drug discovery process by predicting potential drug candidates.
• Medical Imaging: interpreting medical images such as X-rays, MRIs, and CT scans to detect diseases.

Retail

• Personalized Recommendations: Provide personalized product recommendations, improving cross-selling and upselling.
• Visual Search: allows customers to find products using images, enhancing the shopping experience.

Finance

• Algorithmic Trading: Analyze market trends and execute trades to optimize investment strategies.
• Fraud Detection: Identify unusual patterns in financial transactions to detect fraudulent activities.
• Credit scoring
• Risk assessment

Manufacturing:

• Predictive Maintenance: predicts equipment failures and maintenance needs.
• Quality Control: Inspect products for defects and variations during manufacturing.
• Supply Chain Optimization: Optimizes inventory management, demand forecasting, and logistics.
• Robotics and automation

Transportation:

• Autonomous Vehicles: self-driving cars making real-time decisions for navigation.
• Traffic Management: Optimize traffic flow by analyzing data from sensors, cameras, etc…

Agriculture:

• Crop Monitoring: assess plant health, detect weeds, and predict yields.
• Precision Agriculture: guide decisions about planting, irrigation, and pest control.
• Livestock Management: monitors animal behavior and health to improve livestock management.

Energy:

• Energy Management: Optimize energy consumption in buildings (heat, light, cooling, etc…).
• Renewable Energy Forecasting: predict solar/wind energy production to optimize grid management.

Legal:

• Document review and analysis
• Legal research assistance
• Contract analysis

IT Infrastructure

• Anomaly detection
• Security
• System Optimization

* Prompt engineering: Learn how to ask AI tools like ChatGPT the right questions to generate the desired output. This typically takes multiple iterations.