User feedback is essential for product engineering, as it helps you validate your assumptions, identify problems, and improve your solutions. But how do you test user feedback effectively and efficiently? In this article, we will share some tips and tools for testing user feedback in different stages of product development.
Before you collect and test user feedback, you need to define your goals and hypotheses. What are you trying to learn from your users? What are the key metrics that measure your success? How do you align your feedback with your product vision and strategy? By defining your goals, you can focus on the most relevant and actionable feedback, and avoid being overwhelmed or distracted by irrelevant or conflicting opinions.
User feedback can be very subjective and can be misleading if the goals (expectations) are not clearly defined. The end customer may not be able to articulate effectively. A positive to one can be a negative to another. Hence interpretation of user feedback is crucial. Once interpretated, this must be mapped to goals to quantify the feedback
In my experience, we must define the goals by focusing on the jobs to be done from our customer. This will make sense for them to accomplish the tasks as well as give honest feedback since we are solving their problem
Testing and gathering user feedback is an essential part of product development and improvement. Here's a general process for testing and collecting user feedback: Set Clear Objectives: Define the goals and objectives of your testing and feedback process. What are you trying to learn or achieve? Be specific about your expectations. Select the Right Testing Methods: Choose the appropriate testing methods based on your objectives. Common methods include usability testing, surveys, interviews, A/B testing, and beta testing.
Depending on your goals, you can choose different methods for testing user feedback, such as surveys, interviews, usability tests, A/B tests, analytics, or beta testing. Each method has its own advantages and limitations, so you need to consider factors such as the size and diversity of your user base, the type and complexity of your product, the stage and scope of your feedback, and the resources and time available. You can also combine different methods to get a more comprehensive and reliable picture of your user feedback.
Survey is a common method, however it can be tricky. Survey / customer interview if not done properly can be misleading, as these could be interpreted differently. The questionnaire for the survey or user interview must be vetted thoroughly to ask straight and pointed questions. Any room for ambiguity must be curtailed here! If physical or software product, it would be better to get a live user demonstration to understand the different scenarios in which the customer is trying to use the product. This can generically be applied to SaaS or EdTech or electrical products.
Prototyping tools are useful for testing user feedback, as they allow you to create and iterate on your product ideas quickly and easily, without investing too much time and money in development. Prototyping tools can range from simple sketches and wireframes to interactive mockups and simulations, depending on the level of fidelity and functionality you need. Prototyping tools can help you test user feedback on various aspects of your product, such as the design, the features, the flow, the usability, or the value proposition.
Before prototyping, feedback must be collated from different types of user segments - different geos, genders, age groups, etc. This will ensure that the feedback so collected is neutralized and doesn't have a polarized effect. A few simulation tools can be used after prioritizing the demands, for feasibility check!
Once you have collected and tested user feedback, you need to analyze and prioritize it. You can use various techniques and frameworks to organize and categorize your feedback, such as affinity diagrams, feedback matrices, feedback loops, or the Kano model. You can also use tools such as spreadsheets, dashboards, or feedback management platforms to store and visualize your feedback. The main goal of analyzing and prioritizing feedback is to identify the most important and urgent issues and opportunities for your product, and to align them with your goals and resources.
Prioritization must be based on business goals & "customer value proposition". These 2 segments must have the highest weightage during analysis phase. Based on the goals already defined and mapping of goals vs feedback received, it could be better to do a weighted measurement to each of these to identify the criticality and then roll it up with the customer values & business goals. Otherwise these feedbacks will be futile!
After you have analyzed and prioritized feedback, you need to implement and validate it. You can use agile methods such as Scrum or Kanban to plan and execute your feedback-driven changes, and to track and measure their impact. You can also use tools such as version control, testing frameworks, or feature flags to manage and deploy your changes safely and efficiently. The main goal of implementing and validating feedback is to deliver value to your users and to verify that your changes have achieved the desired outcomes.
Post implementation, a subset of users (if possible) who were a part of the initial survey can be requested for a quick validation of feedback, to ensure that everyone's on the same page! This UAT, if possible, will ensure positive vibe before launch.
The final step of testing user feedback is to communicate and iterate feedback. You need to inform your users about the changes you have made based on their feedback, and to thank them for their input. You can use channels such as email, social media, blogs, or in-app notifications to communicate with your users. You also need to collect and test new feedback from your users, and to repeat the process of testing user feedback. The main goal of communicating and iterating feedback is to build trust and loyalty with your users, and to continuously improve your product.
Constant communication is good, but over communication is bad! Crisp iterated communication of what's rolled out, why, "We hear you" messages will build trust and loyalty!
The 'DONTs' of testing user feedback are equally critical. E.g., avoid: fishfooding (using a proto with your team); dogfooding (using a proto within your company); outsourcing (say, to 'digital marketing companies' specialising in A/B Testing etc.). Crowd-sourcing or 'real' users are better (but slightly expensive) test beds for testing user feedback for UX/UI leaning products. For physical products, skipping: scaled/clay models; real (e.g. 3D printed) parts in a (near) real system/mock up; and tests of divergence with Industry Standards (if they exist); and confirmatory simulations often lead to skewed user feedback reconciliation later in the development cycle (especially for new products with new subsystems/components/technology etc.)
It depends on what type of feedback the study team is looking for. For more technical product, I believe we should use different use case / scenario with agreed sample size to determine a more reliable test results. If it’s just service/ human interaction, it varies with many factors; hence a large sample size with tracking of certain keywords might give better insights on the feedback collected. At the end of the day, determine the acceptable framework before start collecting.