Bias in Product - II

Removing bias from product strategy

Cognitive biases and distortions are what we use to make faster assessments and decisions when presented with new information. They are meant to save us time and effort but can stop us from registering all the information and fully understanding its context.

In a product strategy and development setting, this can rob us of insights that directly impact the product and the people using it, ultimately leading to decisions that cost the company a lot of time and resources.

Related read:?Bias in Product — Most Common Biases in MENA Product Development

How to keep bias from creeping into your product

A simple method to keep track of your biases is to write down what you assume is true. This will help you identify what needs to be validated, regardless of the phase of development. Assumptions will start being more obvious to yourself and your team. Your goal should be to prove that you are wrong just as much as you want to prove that you are right. Here is how this applies to the phases of a product lifecycle:

1. When you start working on a new product or an iteration

Make a list of your assumptions before you start any work.

Do you assume that people will find value in your idea? Perhaps you think that by adding a feature, your conversion rate will increase. Write down these assumptions for yourself and your team.

Now think how this would work and - very importantly - how it would fail. How you would measure if it fails? Detail the metrics around it and split them into success and counter metrics. Do not look at totals but instead focus on precise information, at least 2-3 dimensions for the metric. For example: “repeat users within 30 days of account creation”; “sessions/user/week in a 3 months timeframe” and so on — based on what will point out(painfully) whether or not this can fail.

Now find a simple way to test this that ideally does not involve any code being written. It can be a landing page, social media campaign, a post on a well-known forum/platform for your target users, an email reaching out to some of your users, in-app notifications, early bird payments or a sign-up option on a form. Whatever it is, keep it simple, and remember to keep your bias in check when setting up the experiment.

Experiments can be set so that they give false positives, and you do not want to keep putting money and time into something with the idea that “people just don’t get the value yet”(confirmation bias). You could instead use the failed experiment to do qual research and see if you can pivot.

2. During research

Run your research in the medium where your assumed customers will be using your product. There is a huge difference between testing your product in a quiet room when in fact, your users would be using it while having to hold their kid’s hand, carry a bag of groceries on their shoulder and run towards the bus stop. You want to get as close to the real-life situation as possible to be able to understand what they need to pay attention to. Beautiful design won’t be able to cover this if research doesn’t help understand it. Don’t underestimate the importance of clear user needs, pain points, and priorities.

Write your assumptions down before running research and ask the other team members doing research to do the same. List them before you conduct any interviews, testing sessions etc. so that you are aware of what you expect.

Showing it upfront makes it easier to see where you might be suggesting them as problems instead of focusing on the users’ issues since it is already documented.

Do not direct(lead) testers during interviews or testing sessions to confirm any of your preferred solutions or suggest the path they should take. If a user starts talking about a solution, ask them what they believe that this would resolve to uncover the source of their confusion or problem. Say “tell me more” a lot. Do not ask about loved, liked, or hated products or features, it narrows down what people will think about and will give you partial information, while never reaching potential gold mines.

People try to be helpful and not hurt others’ feelings; users are not your design team or PM. Dig deeper into the problems that users find and the areas where they do not behave as expected.

3. Documenting research findings

When analyzing your user research, do not write solutions. Write the clear, cold findings. What did the testers struggle with? What did the people in the interview do when they were on a certain page? Do not assume or interpret anything.

We tend to have a preference towards something because we thought of it, and this is confirmation bias. Just because the solution that you thought of can solve what the interviewee is struggling with does not make it the only or the best solution at this given point in time. Solutions do not point towards common issues. Do not rob your company and your colleagues of the chance of finding a less costly and time-saving solution by imposing your opinion over the findings.

4. Reviewing your findings

Identify the problems and set the themes. What are the top common problems that people had? Have them next to your company goals, support ticket themes, and start discussing the priorities of the issues you have uncovered. Which ones point towards your core product values and goals not working as expected? Which make your product be the furthest away from what you want it to be in an ideal world scenario? Which are a pure nuisance in the day-to-day for your users?

This will help create alignment with the team so that features and personal preferences do not come into play. It helps you push aside your own biases and focus on your users and their problems as human beings, not abstract notions.

4. Organizing and communicating

Use short-term goals and initiatives to set up your roadmap. These get benchmarked to the overall company goal. Now it is not a competition between features and personal preferences, but a user-centered conversation that revolves around what will solve peoples’ problems with the least amount of effort.

Define your sprint goal or your cycle and let the team dig into what the solutions look like. Put a time limit on it so that your team can shape the solution without having a never-ending project. You want to focus on quick iterations that deliver value to most of your users.

Prioritize solutions by writing down how much effort you need to put into creating each and how to test them by defining your success and counter metrics. It doesn’t have to be complex to work. It has to hit the metrics to help you learn and move to the next step.

5. Reviewing results of experiments

Once you have your research, themes, goals, and potential solutions you can start running small experiments to see if the solutions will actually solve the problems that were identified, especially if they include big changes to existing modules or you’re working on a new idea. Sometimes, these experiments can be small product changes as well.

Biases can trickle into this part of the process, so to avoid them, make sure to list your success and counter metrics before the experiment is launched, and after you run it make a list of your expected results and ask your colleagues to do the same, then review the results together. Share your expectations, then see how they compare to the actual metrics.

Doing the “expectations review” exercise before the testing and research phase and using it when reviewing the results can also identify where different departments have slightly different views of what your product does and its ideal direction and can indicate if your company’s goal and strategy is not clear enough across the board. Using findings to clarify what the entire company needs to work towards helps reduce biases which could steer departments in different directions rather than working as a whole, and costing a lot in the process.

To dig deeper into research and possible biases found there, here is a book recommendation: “Just enough research”, Erika Hall . A short read, with comprehensive lists and roles in research that any team can adopt regardless of its size, types of biases encountered during research, and types of research that will fit your organization’s needs. As the title says, it’s structured so that you do?just enough to get you to your next step.

URLs:

https://abookapart.com/products/just-enough-research