How can you involve the product analysis team in post-launch evaluation?

In our Growth team, only the product analysts and the Product Manager have the power to stop a product launch or experiment. Analysts forecast the possible reach and impact of the hypothesis and if they think a new idea won't make a big impact, we might not go ahead with it. And of course this forecast is used as our target metrics to understand the success of the launch - if the real results don't match up with what we expected, we'll think about whether to keep going with the new feature or product.

Defined metrics are crucial for tracking performance and identifying areas for improvement. Systematically categorizing user feedback enables product teams to discern customer sentiment effectively. In B2C environments, where the product is highly customer-facing, the impact of any shortcomings can be significant. Therefore, every metric that affects product performance—including security and usability—requires detailed tracking and analysis. Once the product is launched and tested by customers under real conditions, growth metrics become a priority. Until then, product uptime and usability are important

As a Product Manager, we are always looking to create positive outcomes. For this, we use data. We use data to find the problems and frame them in an understandable format. We use data to drive development of the best solution. We use data to prioritise solutions. It is only natural to use those same data points as success metrics or KPIs for a new product release. For me, there are two main reasons to measure success metrics after release: 1. To measure the positive outcomes that we have created 2. To justify the work that has been done Focusing on outcomes and using data to back up any decisions helps build trust in the product department. Therefore, product analysis needs to be integrated into the full product development lifecycle.

Ideally, the product analysis team should be involved from the time the success criteria for the feature or product are finalised before implementation is done. Product analysis teams can give inputs on the tracking that needs to be in place to evaluate the feature's success whether its user activity events in tools like GA or parameters to be captured in the backend. They can also give inputs on the best approach to test the feature whether it's A/B testing or pre-post evaluation. This will help them have an evaluation plan ready by launch and the tracking can start immediately after launch.

Product Success starts during ideation. What customer problem are you solving? Defining objectives and key results before development, and include your product analysts from the start. Allowing them to understand the problem and what success looks like will allow them to ensure the proper tagging is in place to report on the product once in market.

In my experience with Digital Products, the product analysis capability is vital for product planning. Capturing product usage data is relatively easy for digital products (when compared to non-digital products), and techniques such as page tagging are quite popular. The product usage insights that include KPIs such as active users, session length, bounce rate etc. enable the Product Managers to understand how the product is perceived and used. I usually depend on two inputs 1. Product Analysis, 2. Voice of Customer and by finding the relation between the two I am able to identify the point points and opportunity areas correctly, and this helps me in my roadmap decisions relating to key feature functionality and user experience.

In my experience, closing the loop is one of the most difficult items. To secure this and reinforce the mindset of learning from past experiences or products, I always add a check in the 1st milestones of a new development a review of the lesson learnt between the development team and the product lanch team.

Emphasize the role of cross-functional workshops where product managers and analysts come together to deep-dive into data-driven findings, user feedback, and market trends. This promotes a holistic understanding and decision-making process.

No one can describe the performance conditions for the performance team as well as the analyst team. In some cases, environmental conditions may create challenges for the performance team, which is key to meeting these challenges by the analyst team. Sometimes after installation , some problems occur in the operation process that the presence of the analyst team is very important to identify these obstacles.

Going even further in that area would be letting the product analytics team own the decision-making process/framework for the product as a whole. Continuous data-led and data-informed decision making is key to evolving the product after launch. A naturally seen positive bias in the product management team after launch has to be countered with an objective and plausible approach to get to decision making that is bringing the product forward. Even if it means uncomfortable truths have to be faced.

Post-launch, involve the product analysis team for: 1. Data Collection:Gather user feedback and performance metrics. 2.User Surveys:Collect opinions and identify areas for improvement. 3. User Analytics:Analyze behavior and pinpoint features to enhance. 4. Product Issues:Encourage issue reporting for collaboration on resolution. 5. Competitor Analysis:Stay informed for differentiation opportunities. 6. Performance Metrics: Share indicators to evaluate alignment with goals. 7. Iterative Development:Use insights for continuous real-world improvements.

Once your product is live, ensure you're measuring progress toward your goals - this could include product analytics, voice of customer or other product insights like call center volume. If you are not meeting your defined goals, identify what problems you need to iterate on and continue to measure progress regularly.

Product analytics should lead lots of aspects of a launch from goal setting, data collection, analysis framework to actual measurement and reporting. Any product launch without involving analyst effort from the beginning has a high chance of failure.

Post launch product analysis is one vital channel to capture metrics and data points that generate out of customer interaction with the product. Be it user experience, missing features or resequencing MVPs/Beta, post launch data is highly sort after for in depth analysis.