What are the best practices for collecting and analyzing product reliability data?

When selecting reliability metrics, it is essential to align them with your product goals, user needs, and industry standards. By doing so, you ensure that the metrics are meaningful and relevant to your specific context. Additionally, it is important to ensure that the metrics are SMART: specific, measurable, achievable, relevant, and time-bound. This ensures that the metrics provide clear and actionable insights, can be accurately measured, and are aligned with realistic goals and timelines.

Defining reliability metrics depends on the expectation of reliability for the customer, not what the internal team believes "reliable" means. A product that needs to work in a specific time and always does is considered reliable, even if it often goes down. A product that is always up except when it's necessary is unreliable. Discover through your user behaviour what "reliable" means for them, and enforce this as your metric.

Best practices for collecting and analyzing product reliability data include setting clear reliability metrics and thresholds, regularly collecting data from multiple sources, and using statistical methods to identify trends and failure patterns. Additionally, conducting root cause analysis for failures and continuously iterating on product improvements is crucial for enhancing reliability.

It's important to measure how much time you spend analysing the data, given the impact and size of the problem you're solving and decision you're making. If it's a small problem, with a low risk of a bad call, or a quick decision, easily reversible, don't spend too much time on data analysis. If it's a big decision, high risk to the company or product, or it's a huge problem, then invest in the analysis. On this last approach: merge methods, from data points, to qualitative data, to internal insights. All should be balanced for a great decision.

Data can be complex, you need a good story can make it compelling. Use this framework to create a great story with data Using Context, Dispute and Solution framework, we can turn data into a story that is both informative and engaging. Let's say we have data on the increase of plastic waste in the ocean. 1. The Context is the alarming amount of plastic in the oceans into 2040 2. The Dispute is the lack of recycling and waste reduction. 3. the Solution is implementing more sustainable practice, a scenario to reduce plastic leakage into oceans. Data should not to be dry and boring. Storytelling techniques make it tool for change. Create a story that not only informs but also inspires action.

Most people look at a chart and fail to grasp its meaning. Few will look at a chart and understand what it means, but fail to know what to do with it. The last ones that can look at a chart and understand the next step are the force multipliers. Your role as someone conveying data into action is to turn everyone into this last group. The best way is to combine data, design and storytelling. Inspire with the vision, show the data to back why that is a valuable path, and show how the action can turn the numbers around, going up into the right.