What is 'AI Product Management'?

Co-authored with Yuying Chen-Wynn , Product and Technology Executive

Throughout the last five to ten years, Product Management has been seeing an increased degree of specialization. This can be seen as a sign of the maturation of the profession, as increasingly complex systems have required specialized skills - for example, driving user acquisition for growth PMs, customer segmentation and messaging for go-to-market PMs, or hands-on technical expertise for PMs of developer-facing products. An emerging field that is garnering increasing attention - and confusion - is the role of an ‘AI PM’. What does this term even mean? And does it have different implications for individual PMs, teams and product leaders?

Defining a New Specialization

In 2023, ‘AI Product Management’ refers to a specialization in developing and managing products that leverage AI technology, where the modern focus in AI has largely been on supervised machine learning systems. The main distinction between these ‘AI’ systems and traditional software development is that in AI systems, the behavior of the software is based on patterns found in data, rather than on computer instructions and rules. Where accurate, robust software code is the primary focus in software product development, data quality and quantity are the currency for AI projects. Of course, robust engineering systems are part of the picture, supporting the development, deployment, measurement and maintenance of AI models.?

Individual AI Product Managers

There are many skills areas required for product managers where the requirements vary across industries and types of products.? Common skills areas include technical expertise, UX design, data analysis, experimentation, user research, product development methodologies, business modeling and more.? To simplify that, we can group the core PM capabilities into:??

1. Identify the right problem (ie. be the voice of the customer,understand their pain points and what value proposition resonates with them)
2. Design the right solution (ie.? understand how systems are built enough to build the best feature/solution to 10x the end user experience and received value vs. what they have to struggle with today)

User research, data analysis, business modeling and industry experience are areas that focus on the ability to identify the right problem.? Technical expertise, UX design, experimentation and product development methodologies are the areas that focus on the ability to design the right solution.? The best product managers have a specialization in the types of customer problems they solve and the types of solutions they design - they are not one-size-fits-all generalists.??

An “AI PM” needs a superset of skills from both traditional software engineering environments and more data-centric AI teams. They also need to adapt to working with new types of? stakeholders (data science and data engineering professionals) in their cross-functional teams. Communication and translating complex technology concepts for non-technical audiences is a key skill, so that PMs can work with sales and marketing teams on how to position and promote product benefits. Finally, risk management - including compliance with AI-related rules and ethical guidelines - plays a much larger and more visible role in AI systems where often, results returned to users are not as uniformly predictable as in traditional, rule-based systems.

At its core, however, the AI PM role is still that of a product manager first, and an AI specialist second. The three fundamental areas of product management - user knowledge and empathy, driving execution and creation of new systems, and data-driven strategy and decision-making, are even more important - and often underemphasized - in AI product teams.

Hiring and Developing AI Product Management Talent

Similar to hiring and developing product management talent for any specialization, you are prioritizing between the must-haves vs can-develop of the 3 fundamental areas: user knowledge, execution of new systems, and data-driven decision making. The priority is different depending on the stage of the product maturity.

If the product is in the discovery stage of creating product concepts, user knowledge is the must-have to hire for. The deeper the industry and user segment experience, the better. ETS AI Technology Labs has had success with this model since the majority of their work right now is experimenting with different concepts and prototypes that leverage the multi-modal AI capabilities the technologists have developed. Their best product managers come from decade plus of experience in Education or EdTech.?

Developing these product managers’ technical know-how is done by pairing them with an AI technologist during the planning phase to quickly answer feasibility and effort from the AI perspective. Over time, the product managers become familiar enough with the in-house AI capabilities to no longer need the dedicated technologist support. High-level AI overview courses can also shorten their ramp up time.

If instead, the product is in the optimization phase to improve the performance or adding new use cases, the technical experience and data-driven decision making are must-haves for the AI Product Manager. BNED DSS leverages many AI models to improve their Ask an Expert routing for homework help and tutoring services. The in-production models and algorithms need to be continuously optimized to improve overall response time and decrease false positives. A technical product manager runs AB tests with many configuration options to find the optimal one. Since the volume and subjects of questions are constantly in flux as well as availability of subject matter experts, this is an ongoing technical product management function.?

The best way to develop a technical product manager’s user knowledge is to start them on data analysis of usage patterns of their feature areas. When they encounter data patterns that are surprising, pair them with user researchers to find out why it’s happening by talking to users directly. You can also mentor them with brain storming around problem areas that can benefit from a 10x speed optimization or 10x reduction in manual labor.? This is an easier guide post for finding problem areas that AI decisions, detections, recommendations and predictions can benefit. This can be a very successful model in developing user knowledge of technical product managers.

The key checkpoint of how successful your AI Product Manager is growing is testing the value proposition they created with your target customer.? The stronger the response, the better the fit your AI PM is for the role they’re growing into.??

AI Skills and Knowledge for Product Managers

There is a growing corpus of online educational content aimed at filling knowledge gaps for individual product managers and leaders alike to be able to work with data science and engineering teams on AI projects.

• Making Friends with Machine Learning - For a business audience looking to gain an intuitive understanding of how machine learning works, the main models or algorithms in use and what it takes to take machine learning systems from planning to deployment, this 6-hour free YouTube course is a great foundation. Originally created as internal training for Google employees by Google Chief Decision Scientist Cassie Korzyrkov.
• ?AI for Everyone on Coursera - Deeplearning.ai’s online course is designed to teach non-technical individuals the basics of artificial intelligence and its applications. It takes around 11 hours to complete. The course is created by Andrew Ng, a pioneer in the field of AI and a co-founder of Google Brain and DeepLearning.ai.
• Machine Learning Specialization on Coursera - Another deeplearning.ai course, again pioneered by Andrew Ng, does require some basic programming and more time commitment (approximately 3 months to complete at 9 hours/week).? It is one of the first and most well-known introductory courses for Machine Learning. For individual PMs, the course can give a finer-grained intuition about what is possible and how data will be used by different models. It also provides more common language for communication and collaboration with data science teams.

AI Skills and Knowledge for Executives and Senior Leaders?

Executives and senior leaders, regardless of their functional area, own strategy, roadmap and resource prioritization decisions. To effectively include AI as part of those decisions, you need to understand the AI capabilities and value propositions relevant to your industry, product category, and technology stack. The main value propositions for business are:? automation, personalization, and predictions. The main AI capabilities driving these value propositions simply classify into detecting patterns in, and generating data, text, language, audio, images and videos. Know what types of data you have access to in abundance and decide the best value proposition for your area of business you want to pursue. With this, you can ask your AI teams and product managers to give you proposals of potential projects. To become more knowledgeable about AI, try out the recommendations above, or a few AI for Executives courses to get a high-level overview. Even more relevant is reading about specific AI use cases and attending demos of new AI products and solutions in your industry or adjacent industries.

Business executives with a technical background will have more of an advantage here, and should become the internal champion of the company’s AI strategy and investments. Having these executives involved in the product discovery and concept validation stage can be key to success with new AI innovations in a company.

What Doesn’t Change: Core PM Skills, Now More than Ever

When a technology gains rapid popularity, it is tempting (but dangerous) for product professionals to see every product development problem through the lens of that technology, in line with the proverb “if all you have is a hammer, everything looks like a nail”.? Companies just want to be able to add the keyword of the technology to the description of their products to look innovative.? They are also tempted to turn over business functions to technologists - perhaps hiring PMs with tech or data science backgrounds, ignoring core product competencies. The field of AI has existed in this space for too long, with an over-investment in tech-led startups that have failed to solve meaningful customer problems. True end user value of the technology adoption in commercialization often arrives 10-20 years after the capability is available.? The area of AI of machine-learned decision making, predictions and recommendations from data and natural language has just hit this zeitgeist - along with current media darlings ChatGPT and the cohort of competitive Large Language Models.? Considering the distance between technology capability and productization maturity - there has never been a more important time for organizations to invest in the development of their product management function to fully capitalize on the potential of these exciting technology advances.

ARTICLE TAKEAWAYS

• AI Product Management is a new area of specialization of Product Management, similar to specializations like ‘Growth PM’ or ‘GTM PM’ in its reliance on specialized skills, that relates to developing and managing products that leverage AI.
• AI Product Managers require traditional core product management skills - being able to identify the right problem, and being able to deliver the right solution - with additional depth in software engineering, data-centric skills, communication and risk management.
• Hiring and development of AI Product talent depends on the stage of your project. For discovery-stage initiatives, emphasize user knowledge and domain expertise. For optimizing and scaling, look for technical depth and data-driven decision-making.
• Develop technical knowledge in domain-expert PMs by pairing them with AI technologists during planning. Develop user empathy in more technical PMs with data analysis of usage patterns, and pairing them with user research teams.
• Many AI initiatives have been technology led, leading to a bias to hiring AI technologists as PMs rather than upskilling domain experts or optimization-focused PMs in AI. Organizations need to invest in developing AI knowledge in their product teams to fully take advantage of the latest innovations in AI.