The Dual Metrics Model — an Approach to Product Design Metrics

Metrics aren’t just abstract numbers — they reflect real user experiences. When users abandon tasks midway, it’s more than just a dip in completion rates — it’s a signal that something within the design is causing friction. Maybe the onboarding process is too complex, or perhaps critical features are hidden too deeply in the UI. Metrics help tell these stories, providing designers with concrete evidence to diagnose and address issues.

Despite their usefulness, metrics always come with risks. Focusing on the wrong metrics or over-optimizing based on chosen metrics can lead to unintended consequences.

Using the wrong metrics means that designers may optimize for outcomes that don’t actually matter to users or the business, especially if the chosen metric doesn’t align with the core goal of the project. For example, improving a minor feature’s engagement might come at the cost of neglecting more critical aspects of the user experience.

When design teams focus too heavily on a single metric, such as reducing bounce rates, they may miss the bigger picture, sacrificing user experience or brand identity. This is an example of over-optimizing for a metric. This tunnel vision can result in a product that meets specific goals but fails to deliver a holistic and satisfying user experience.

To mitigate these risks, designers can use a Dual Metrics Model to ensure their design decisions are balanced and aligned with user needs and business goals.

A Dual Metrics Model provides a balanced framework for setting, tracking, and adjusting metrics in a way that supports both primary and secondary goals.

The Dual Metrics Model encourages a holistic view of design success by not tracking different metrics separately and later evaluating them one against another but by pairing distinct yet interconnected goals and tracking them simultaneously. Balancing between two goals prevents over-optimization and ensures that the most important aspects of the design are accounted for.

Core Concept

At the heart of the Dual Metrics Model are just two key elements (in theory, there can be more, but in practice, this leads to chaos and neglects the model).

• Impact Goal and Impact Metrics

This is the primary objective of the project or product, tied to a specific, measurable outcome. The Impact Goal represents the core focus of the design or development effort, such as increasing user engagement or reducing churn.

• Balance Goal and Balance Metrics

The Balance Goal is a complementary objective that ensures the quality, user experience, or long-term sustainability of the primary goal. This acts as a safeguard, preventing over-optimization of the primary metric. Suppose the Impact Goal is to increase user engagement. In that case, the Balance Goal might be to maintain high user satisfaction, ensuring that the push for engagement doesn’t compromise overall user experience.

By defining both Impact and Balance metrics upfront, the Dual Metrics Model ensures that the team remains aware of the broader impacts of their work, leading to more balanced and user-centered design outcomes.

Steps to Implement the Dual Metrics Model

There are just five simple steps design teams need to follow to introduce the approach in their design process.

1. Define the Impact Goal: Start by identifying the core objective of the design project. This is typically aligned with business objectives or user needs, such as increasing user engagement, reducing churn, or improving task completion times.

2. Set the Impact Metric: Choose a quantifiable metric that directly tracks progress towards the Impact goal. This metric will be the leading indicator of success.

3. Define the Balance Goal: Choose a secondary goal that balances the Impact goal. This goal ensures that while focusing on the primary objective, other critical aspects, such as user satisfaction, quality, or costs, are not neglected.

4. Set the Impact Metric: Choose a metric that measures the secondary goal. This metric acts as a check against over-optimization and helps ensure that the broader system remains balanced and effective.

5. Track and Iterate: Continuously monitor both metrics throughout the design and development process. If one metric begins to dominate at the expense of the other, adjust the approach.

Benefits of the Dual Metrics Model

• Balanced Design Approach: Teams focus on both immediate performance (Impact goal) and sustained quality or user experience (Balance goal).
• Informed Decision-Making: Decisions are based not on one dominant or multiple possible different metrics (not every single metric should be counted and influence the design or development decision) or, in the worst case, gut feelings, but on a strategic combination of interconnected metrics leading to more objective and effective outcomes.
• Enhanced Collaboration: The model unifies cross-functional teams by providing clear goals and metrics that align with design, development, and business objectives.
• Improved User Satisfaction: Balancing performance with quality leads to a more user-centered product, ultimately enhancing the overall user experience.

Applying the Dual Metrics Model to a SaaS — Document Management System

Let’s consider a scenario where a design team is working on improving a SaaS Document Management System. This system allows organizations to securely store, manage, and share documents in the cloud and offers features such as intelligent indexing, workflow automation, etc.

For this example, we’ll explore how the Dual Metrics Model can be applied to a SaaS DMS to ensure that both immediate performance goals and long-term quality objectives are balanced effectively.

Step 1: Define the Impact Goal

Impact Goal: Enhance document retrieval speed to reduce the time users spend searching for documents.

In a DMS, the ability to quickly locate and retrieve documents is vital. Organizations depend on this feature to maintain productivity and streamline their operations. Thus, improving the speed of document retrieval becomes the primary objective.

Step 2: Set the Impact Metric

Impact Metric: Average time taken to retrieve a document.

This metric directly tracks the progress towards the Impact Goal. It provides a clear, quantifiable measure of how efficiently users can retrieve documents from the system. A reduction in this time is indicative of improved performance and user satisfaction in the context of task efficiency.

Step 3: Define the Balance Goal

Balance Goal: Maintain high accuracy and relevancy of search results without significantly increasing operational costs.

While the primary focus is on improving retrieval speed, it’s crucial that this does not come at the expense of accuracy or cause an increase in costs that could make the system less sustainable or user-friendly. The Balance Goal ensures that the push for faster retrieval doesn’t compromise the overall quality of the system.

Step 4: Set the Balance Metric

Balance Metrics:

1. Search result accuracy: This can be measured by user feedback on the relevance of the search results, ensuring that users consistently find the documents they are looking for.
2. Cost per document retrieval: Tracks the operational cost associated with each document retrieval, ensuring that improvements in speed don’t lead to unsustainable increases in costs.

These metrics serve as checks to ensure that while the system becomes faster, it remains accurate and cost-effective, preventing the over-optimization of the primary metric at the expense of overall system quality.

Step 5: Track and Iterate

Continuously monitor both the Impact and Balance Metrics throughout the design and development process. If data shows that the focus on speed is decreasing search accuracy or increasing costs, adjustments must be made. This might involve refining the search algorithm, enhancing indexing methods, or optimizing resource allocation.

For example, after implementing changes to reduce retrieval times, the team might notice that accuracy has slightly declined, with more irrelevant documents appearing in search results. In response, they could adjust the algorithm to better prioritize relevance without sacrificing too much of the speed gains. Simultaneously, cost metrics ensure that these improvements remain financially sustainable.

Benefits of Using the Dual Metrics Model

• Balanced Design Approach: By focusing on both the speed of document retrieval (Impact Goal) and the accuracy and cost-effectiveness (Balance Goal), the team ensures that the DMS meets user needs holistically, not just in isolated aspects.
• Informed Decision-Making: The use of interconnected metrics allows the design team to make data-driven decisions, ensuring that improvements in one area do not negatively impact another. This strategic combination of metrics avoids the pitfalls of focusing too narrowly on a single measure of success.
• Enhanced Collaboration: The Dual Metrics Model fosters collaboration between different departments — such as design, development, and operations — by providing clear, aligned goals that consider both immediate performance and long-term sustainability.
• Improved User Satisfaction: By balancing performance with quality, the model helps create a product that is not only fast but also reliable and cost-effective, leading to a better overall user experience.

Afterwords

Relying on a single metric can be dangerous — everyone knows that — and relying on many different metrics can be even more dangerous. The Dual Metrics Model offers a balanced, holistic approach, pushing to pick mutually influencing goals and metrics upfront. By considering both Impact and Balance Goals, designers can use data to meet immediate objectives and provide long-term value to users and businesses alike.