Product Goals and Metrics: Part?3

In Part 1 and 2, I covered the following topics:

• defining a clear vision for your product
• selecting key customer outcomes and corresponding “external” metrics for the product

In Part 3, we will discuss what we can call “internal” or “secondary” metrics for the product. Examples include “site load time” or “error pages returned”. These metrics are important to help define internal goals for different parts of your product and team to enable the overall objective and vision for the product.

In order to define “internal” metrics, it is important to connect them to the vision and the primary customer outcomes we have talked about in Part 2 of the series. The primary metrics we had outlined include:

1. Percent of sessions which use the search engine
2. Percent of search queries that lead to a click on the first search result
3. Percent of search sessions that lead to a search result added to cart

Each of these metrics depends on several others that define the overall customer experience. For example, let’s talk about “percent of sessions which use the search engine”. We could assume a large percent of customers landing on a retailer site are interested in finding or purchasing a product but not all. Some might be looking for the status of their order they just placed, others might be looking for at their store’s opening hours, while some might simply want to see if they have any deals running. If we believe the best way for customers to find any of this information is search, we could say close to 100% site sessions must use the search engine.

In addition, most sites have users directly opening the site or app to use it and those that come in via emails, external advertising campaigns, and from search engines like Google and Bing. A lot of this traffic might be transient. They might not find what they need or might have no familiarity with the brand and might leave immediately. Hence, we need to understand the nuances of each metric and what impacts the numbers, before we set goals. Having said that, if the search engine is amazing i.e. it is fast, simple and easy to use, and gets customers what they need, it will have a positive impact on customers and they will use it more.

Hence, the usage metric depends on multiple internal factors including:

1. speed of the product or “time it takes for customers to completely see the top or all search results” or “percent times search does not load or throws an error”; this, in turn, depends on how the search engine is built, the technology stack used, what device and bandwidth the user is on, and a variety of other factors.
2. the quality of the search results or “relevance of top results”; this depends on how good we are at understanding user intent and matching it with their preferences and the best products. it also depends on how well we rank these matches putting the best of the best at the top.
3. the ease of use of the product; this could be measured in several ways including “user satisfaction scores” in surveys or qualitative studies, or “number of steps it takes for search users to click on a result” where steps could be many activities including querying, sorting, filtering and more. the number of steps would vary by the shopping category, the device the user is on and more. hence, it is important to look at these metrics from those angles too.

The above metrics are a few examples of “secondary” or “internal” metrics that product owners need to think about. In order to define goals, both “external” and “internal” metrics are important. “External” metrics help define clear customer outcomes while “internal” ones define how to achieve them. Even “internal” metrics will have several smaller metrics helping move it in the right direction. Depending on what part of the product one owns, it will key to understand these and find which ones fit in one’s situation. Going back to the photo sharing app, internal metrics tied to "percent users who use the app to share photos daily" could include "coverage of share button on photos", "number of messaging and social media apps connected to the photo sharing app", "ease of sharing like number of clicks, time to load" and more. You get the gist.

In Part 4, next, we will start setting goals and walk through a real example. Thank you for reading.