Principle 5: Reviewing Data - Reducing Bias in Interpretation

Note: This article is part of a series. I recommend reading the intro, principle 1 (part 1 and part 2 ), principle 2 , principle 3 (part 1 , part 2 and part 3 ), and principle 4 before continuing below.

Throughout this series of principles on social issue research, we created a framework of key considerations when managing or even executing social issue research. Each of the principles provides contextual guidance for any marketer or leader to follow as they look to understand how the public is engaging in social issues.?

In this last article, we will discuss reducing one’s bias in performing market research on social issue research analysis and interpretation. While there are protocols for research to reduce bias, which I will not cover here, there are particular areas of social issue bias that are important to address to ensure that studies do not give the impression of larger interests and/or false indications of participatory behavior.

Interpretation Bias

Interpretation bias occurs during the analysis and interpretation of data when the researcher views data points through their personal, social and cultural lens that provides skewed perspectives, claims and outcomes statements.?

An example of this is as follows:

A data point states that 45% of people would vote in favor of a policy on a particular?social issue. The research claim is presented as almost half of the public is in favor of?the issue. However, the researcher does not note that more than half are not in favor - and therefore, it would be potentially challenging for the organization, if there was a vote today, to get the policy passed.

This type of bias is sometimes conscious and sometimes unconscious as researchers bring their lived experiences into the analytical process. Internally, research teams should find a balance of objectivity by diversifying research members of each team. This diversity reflects culture, race, ethnicity, experience and areas of research focus. Research teams should be constructed with a diverse and inclusive mindset to ensure that viewpoints and experiences can be balanced.

If you work with outside research vendors, ensure that teams have diverse backgrounds and lived experiences as well. Seek information on not just the research experiences but how they address interpretation bias in their analytical work, what are the internal quality controls/reviews around analytical reports and finding briefs, and lastly who is responsible for research bias oversight.

Research Guidance Panels

Guidance panels provide a great opportunity to balance interpretation of data and supplement the work of internal research teams or outside vendors. Social issue guidance panels are constructed of individuals with live experience and subject matter expertise. Guidance panel members sit in on interpretation review meetings, review briefs and reports on data interpretation and provide recommendations and feedback on all claims and research statements of findings. Members of guidance panels can be very helpful in reviewing also the setup/contextual interpretation of data beyond just the data point but rather how the facts/data are presented.?

Putting It All Together

As I noted at the outset of this series, the quality of data in reflects the quality of data coming out. Many leaders are too quick to perform studies without doing the necessary due diligence: building accurate samples, asking the right questions and performing the right analysis to get at the best strategy for public engagement.?

When doing social issue market research on the general public, it’s critical to ensure that we truly get an accurate sense of their knowledge, attitudes and behaviors on a particular issue or topic by following five guiding principles. These principles help ensure we deliver not just the insights we need to make better strategies, but that we also ensure we have the real opportunity, challenges and barriers reflected in our work - especially when it comes to marketing and activating the public.?

Follow the principles we’ve outlined in this article series, and you’ll be on your way to creating an accurate sample, understanding issues in context, creating better questions, segmenting audiences appropriately, and reducing bias when interpreting - to get better results overall.?