How do you avoid bias and noise in user feedback data?

If it involves understanding user behavior, go for analytics. Need a qualitative feedback on your product, usability test. Want to understand your user pain points and needs, go with interviews. Need a quantitative feedback, go with surveys. You can always mix and merge them to get answers to your questions. Choosing the right method is important. Identify what you actually need to find out, is it success rate, user behavior, task analysis, etc. One way to do it is by clearly communicating with your stakeholders and come to an agreement where user needs and business values overlaps (imagine Venn diagram).

Super important to accurately define your target audience first. The target audience need to include everyone that would use your product and interview representatives. When recruiting interviewees be aware of your own biases. When inviting them to the interview make sure no one somehow slipped in and isn't that persona. If they aren't your persona , do not continue interviewing them so irrelevant information doesn't get in the way.

Start with why. Why are we asking? Why do we care? Why would users help us? Why do we need this research now? Asking why, will begin to sort out your problem statement. Spend the majority of your time on problem discovery. This will help clarify your goals and what methods you need to employ. After figuring out your problem and after defining your methods; you need to show how your methods will relate to company OKRs. This is what will bring true value to your company and because you are empathetic, you will create value for your users as well!

Hallway studies are considered to be the easiest but it has a huge pitfall of not carefully selecting your participant. Hallway study will not give you enough time to decide if the candidate fits your criteria, they may or may not be in your target audience. Never choose hallway studies if you are designing for a niche audience. Better invest your time in moderated user testing.

Recruiting a sample can be really easy or really hard. Using large platforms that have a pool of users is great, but often the answers will be generic and rambling. To combat this, you need to know your users. Personas are a great way to find a sample. But they need to constantly be updated. The best way to get a sample is to you in-app requests to amass a pool of your actual users! When you begin to collect a sample of your users you will get specific, focused insights. Too many times we take the easy path and just say, "close enough." Your users deserve more than that. Find ways to connect with them and extract their pain points and insights. You will have a much better product when you do that.

It's impossible to ask an unbiased question as we are filled to the brim with bias. But we can mitigate that bias through continually refining our question set to be as clear as possible. One of the ways I do this is through iteration. I will write a set of questions and do mock interviews with people on my team (if there is no team, friends and family can help). As you do this, you will begin to see where things are unclear and biased. This is the time to iterate and adjust accordingly. I try to do 2-3 rounds of iteration before going to users with my questions.

Don't ever "validate your designs". This means you created a design in a bubble and are now looking for users to confirm that you are right. That is not how you conduct research. Start with a problem statement, connect with your users, and analyze the data. This should be the most exciting part of research! Now that you have connected with users, you have lots of great data points; how do you analyze objectively. Ideally, you spent time setting up a proper problem statement. Why is this important? Because it will help eliminate the subjectivity or validation we so desperately default to as humans. When you have a poor problem statement, you end up attempting to validate a users answers and cherry-pick the results you want.

Quantitative data is good. Qualitative data is good. On their own, they will never give the full story. The way to find the truth is to blend all of this data to come to the proper conclusions. I have watched hours of screen recordings and it tells one story. Then I have analyzed dips in CTRs and user journeys, they tell a different story. Even further, having talked to users they had another story as well. Yet when I combined all of those methods together, I was able to come up with patterns that led to real solutions. This was not an easy, short-term task. It was focused and went through several conversations. This led to better products & better user experiences. It was a win-win for users and the business. That is always the goal!