What are the common challenges when tracking optimization metrics and KPIs in product development?

There are a whole lot of KPIs (and SMART KPIs). It's important that we first understand the data we want to track. We should then choose KPIs that: a) align with the specific objective as well as the overall strategy, b)are unique and implementable, and c)makes maximum impact to achieving the objective. It's always better to choose 3 to 4 KPIs per objective to start with, before expanding your list of KPIs.

Choosing the right metrics starts with a conversation with the business constituents who are going to be measured. It should be clear what actions the KPIs drive at various levels of performance. The litmus test is if the KPI doesn't lead to a defined action then it may not be worth tracking and creating/doing the data work necessary to display it on dashboards etc. Action Plans and Definitions of what good looks like should be included in the KPI definition, in addition, it is often worthwhile to consult Gartner, McKinsey, or other providers of business research for industry benchmarks. Also if there are too many metrics for one single business question it could be a sign of confusion about what to do and also too many nice to haves.

I will invite other people to expand on what I am about to say, but there are many newer tools, architectures, and strategies to move data and consolidate data. With the Data Mesh, there is this concept of data containers/products that may make it easier to distribute and connect data. Also, the Data fabric allows for data virtualization that lets the data analytics team connect data without moving it so that additional variables can be brought to bear to solve any particular business question, regardless of where that data sits.

Data Visualization via Analytical Self-Service is an important topic as firms shift from Descriptive Analytics to Prescriptive and on to AI. Ensuring highly curated dashboards that are beyond a cacophony of pictures but also have the key insights already called out so that they produce enough signal and not noise is key. Leveraging a client-centric data visualization process that pulls from agile and design thinking principles is key. The dashboard should have an insights summary and key actions to take based on some of the findings. A self-service analytics unit can help produce visualizations that create an insights-driven business. Favorite authors: include A. Kaushik, Knaflic, and S. Few. Tableau/PowerBI expertise is key.

One thing for product development is to conduct a brainstorming session to catalog not only the types of analytics that would be helpful but also to understand what motivates product managers to use the analytics to make decisions. Where are there roadblocks to fact-based decision making and what are the collective reasons for wanting a particular type of analysis? Knowing the why and the true motivations of the product team is a key aspect of being successful with analytics. It is not only about the technical depth of the team but also getting underneath the reason for the business questions and how insights will be actioned and deployed.