6. What should you look for in a poll?

If it’s true that “you are what you eat,” then it’s also true that what you know about public opinion depends on your diet of polls.

When nutritionists tell you to “eat right,” they usually mean that you should eat high-quality foods and have a balanced diet. Same with polls: Try to consume high-quality polling and limit your intake of junk polls. And if you want to better understand a topic, look at a variety of survey questions about it.?

How do I know what’s high-quality polling?

The easiest way is to rely on polling from organizations with a track record in survey research, like Pew Research Center or Gallup (you knew I’d say that, didn’t you?). Major media organizations like Fox News, CNN, NBC, CBS, ABC and The Washington Post also have dedicated teams that conduct regular, high-quality polling on the issues of the day, as well as about elections.

Many private survey companies and university research centers do polling of their own, sometimes in partnership with news organizations (for example, SSRS and CNN, NORC and The Associated Press or Siena College and The New York Times). Other universities that conduct polls include Monmouth, Quinnipiac and Marist. Of course, just because you haven’t heard of a particular polling organization doesn’t necessarily mean its results are wrong.?

Critics of polling sometimes say that polls are designed to promote a particular point of view. While it’s always possible to find a poll or individual question that might be biased, there are at least two reasons to be skeptical that intentional bias is widespread in public polling.?

First, all the organizations mentioned above – and others – have been around and doing polling for many years. They have reputations to protect, both among their audiences and among their peers in the polling community. (And believe me, we carefully watch what our peers do and vice versa.) If a particular poll publishes a questionable result, another polling organization will sometimes conduct a similar poll to see if it can confirm the result.??

Second, you can usually make a reasonable assessment of the trustworthiness of a poll yourself. A reputable polling organization will make available some basic information about how the poll was conducted, how the questions were asked and what questions were asked. If the organization won’t provide that information, you shouldn’t trust its polls. (The major association of polling professionals, the American Association for Public Opinion Research, or AAPOR, has a?list of polling organizations?that have?committed to transparency.)?

OK, I’ve found some info about how a poll was done. But what exactly am I looking for??

There should be a methodology statement and a list of the questions asked, which, along with the description of the findings, should enable you to answer questions like these:?

• Who conducted the poll, and who sponsored it??Sponsorship can provide some clues about possible bias or agenda.?

• Who was interviewed??Was it all adults in the population, just the people in one state,?only registered voters, or some special subgroup in the population such as teachers or people who home-school their children??

• How was the poll conducted??Was it by telephone, mail, online, in person? No method is perfect, but this information should always be disclosed.?

• How were people selected for the poll??Were they selected randomly, or were they allowed to “opt in” to be a part of it? This can sometimes be difficult to determine, but?random samples?(also called “probability samples”)?usually?are?better samples.?

• When was the poll done??Public opinion on most issues is remarkably stable, so you don’t necessarily need a recent poll about an issue to get a sense of what people think about it. But outside events can and do change public opinion, especially when people are first learning about a new topic. It’s also true that polls taken immediately after a dramatic event may overstate the event’s impact on public opinion.?

• How many people were interviewed??A poll with more interviews is usually better, but there’s no hard-and-fast rule. A national poll with only 500 respondents, for example, can measure views nationally, but it will generally not be able to break out the views of smaller groups in the population, like Black or Hispanic adults. And if the poll uses inexpensive opt-in participants to pad its sample size, there may be no additional value added.?

• Was the poll weighted??If so, how? This is one of the more technical topics in polling, but a good poll almost certainly has been weighted – that is, statistically adjusted to correct for the inevitable difficulties in reaching a representative sample, as mentioned in Lesson 2. Polls should state whether or not they are weighted, and good polls should provide details about the weighting. The absence of this information is a red flag.?

• Do the poll’s questions look like they are neutral and fair??You may not be an expert on polling or the subject of the poll, but you can read a question and make a judgment as to whether the pollster is trying to get a particular answer (again, review Lesson 2). Be sure to look at all the questions in the poll, too: Sometimes a question asked earlier in a poll can affect how later questions are answered. This?short video?provides more detail about best practices in writing survey questions.?

If you are looking at election polls specifically, all of the above matters, and there are?a few additional things?to be aware of.

Why should I turn to polls to understand public opinion?

Polls aren’t the only way to learn about public opinion, but they are the most comprehensive and least biased way.?

Calls and letters to lawmakers provide a view of public opinion, but they’re limited to those who feel strongly about an issue and have the time, energy and skills to craft and send a message. Talking to your friends may give you a sense of how they feel about a subject, but your friends are what we call a “self-selected sample” and can’t represent the range of experiences and opinions found among the larger population of your country, state or even local community.?

By contrast, polls work the way democracy is supposed to work: They give everyone an equal chance to be heard. Good opinion polls represent the full range of views in a population.?

If you’re trying to learn what the public thinks about a subject, it’s important to remember that no single survey question can provide a perfect view of public opinion on an issue. Most issues are complicated and multifaceted. Moreover, the specific words used in a question may evoke different responses than alternative words that seem similar to the researcher.

For this reason, we highly recommend that you look at a variety of measures of opinion on a topic, including:

• How important is the issue to people, especially in relation to other issues?
• How much do people know about the issue??
• How has opinion on the issue changed over time??
• How are different groups experiencing and thinking about an issue?
• Can data from sources other than surveys help us understand what surveys tell us??

Pew Research Center tries to provide that kind of variety in many of our reports. But you can also look for questions about a topic from many different polling organizations. We’d particularly recommend polls by Gallup, The Associated Press-NORC Center for Public Affairs Research and KFF (for health-related public opinion). You can search a database of our survey questions?here.?