Put an End to Reporting on Election Polls
Image by m kasahara / Flickr

Put an End to Reporting on Election Polls

We now know that the US election polls were wrong. Just like they were in Brexit. Over the last few months, I’ve told numerous reporters and people in the media industry that they should be wary of the polling data they’re seeing, but I was generally ignored and dismissed. I wasn’t alone?—?two computer scientists whom I deeply respect?—?Jenn Wortman Vaughan and Hanna Wallach?—?were trying to get an op-ed on prediction and uncertainty into major newspapers, but were repeatedly told that the outcome was obvious. It was not. And election polls will be increasingly problematic if we continue to approach them the way we currently do.

It’s now time for the media to put a moratorium on reporting on election polls and fancy visualizations of statistical data. And for data scientists and pollsters to stop feeding the media hype cycle with statistics that they know have flaws or will be misinterpreted as fact.

Why Political Polling Will Never Be Right

Polling and survey research has a beautiful history, one that most people who obsess over the numbers don’t know. In The Averaged American, Sarah Igo documents three survey projects that unfolded in the mid-20th century that set the stage for contemporary polling: the Middletown studies, Gallup, and Kinsey. As a researcher, it’s mindblowing to see just how naive folks were about statistics and data collection in the early development of this field, how much the field has learned and developed. But there’s another striking message in this book: Americans were willing to contribute to these kinds of studies at unparalleled levels compared to their peers worldwide because they saw themselves as contributing to the making of public life. They were willing to reveal their thoughts, beliefs, and ideas because they saw doing so as productive for them individually and collectively.

As folks unpack the inaccuracies of contemporary polling data, they’re going to focus on technical limitations. Some of these are real. Cell phones have changed polling?—?many people don’t pick up unknown numbers. The FCC’s ruling that limited robocalls to protect consumers in late 2015 meant that this year’s sampling process got skewed, that polling became more expensive, and that pollsters took shortcuts. We’ve heard about how efforts to extrapolate representativeness from small samples messes with the data?—?such as the NYTimes report on a single person distorting national polling averages.

But there’s a more insidious problem with the polling data that is often unacknowledged. Everyone and their mother wants to collect data from the public. And the public is tired of being asked, which they perceive as being nagged. In swing states, registered voters were overwhelmed with calls from real pollsters, fake pollsters, political campaigns, fundraising groups, special interest groups, and their neighbors. We know that people often lie to pollsters (confirmation bias), but when people don’t trust information collection processes, normal respondent bias becomes downright deceptive. You cannot collect reasonable data when the public doesn’t believe in the data collection project. And political pollsters have pretty much killed off their ability to do reasonable polling because they’ve undermined trust. It’s like what happens when you plant the same crop over and over again until the land can no longer sustain that crop.

Election polling is dead, and we need to accept that.

Why Reporting on Election Polling Is Dangerous

To most people, even those who know better, statistics look like facts. And polling results look like truth serum, even when pollsters responsibly report margin of error information. It’s just so reassuring or motivating to see stark numbers because you feel like you can do something about those numbers, and then, when the numbers change, you feel good. This plays into basic human psychology. And this is why we use numbers as an incentive in both education and the workplace.

Political campaigns use numbers to drive actions on their teams. They push people to go to particular geographies, they use numbers to galvanize supporters. And this is important, which is why campaigns invest in pollsters and polling processes.

Unfortunately, this psychology and logic gets messed up when you’re talking about reporting on election polls in the public. When the numbers look like your team is winning, you relax and stop fretting, often into complacency.When the numbers look like your team is losing, you feel more motivated to take steps and do something. This is part of why the media likes the horse race?—?they push people to action by reporting on numbers, which in effect pushes different groups to take action. They like the attention that they get as the mood swings across the country in a hotly contested race.

But there is number burnout and exhaustion. As people feel pushed and swayed, as the horse race goes on and on, they get more and more disenchanted. Rather than galvanizing people to act, reporting on political polling over a long period of time with flashy visuals and constantly shifting needles prompts people to disengage from the process. In short, when it comes to the election, this prompts people to not show up to vote. Or to be so disgusted that voting practices become emotionally negative actions rather than productively informed ones.

This is a terrible outcome. The media’s responsibility is to inform the public and contribute to a productive democratic process. By covering political polls as though they are facts in an obsessive way, they are not only being statistically irresponsible, but they are also being psychologically irresponsible.

The news media are trying to create an addictive product through their news coverage, and, in doing so, they are pushing people into a state of overdose.

Yesterday, I wrote about how the media is being gamed and not taking moral responsibility for its participation in the spectacle of this year’s election. One of its major flaws is how it’s covering data and engaging in polling coverage. This is, in many ways, the easiest part of the process to fix. So I call on the news media to put a moratorium on political polling coverage, to radically reduce the frequency with which they reference polls during an election season, and to be super critical of the data that they receive. If they want to be a check to power, they need to have the structures in place to be a check to math.

(This was first posted on Points. Image from Flickr by m kasahara.)

Chris Zamborsky

Senior Project Manager / OCM Consultant

8 年

Self reporting and biased questioning are just 2 extreme flaws in any polling. The mainstream media wants to report their agenda. They fish for data and polls that agree with them. It isn't even close to science in most cases.

回复
Miodrag Joseph Erl

Agile Organisation Design and Governance Architect

8 年

Danah, is there any substantial connection between BREXIT and Mr Trump's victory?

回复
Amelia Ketzle

IT Project Coordinator - SIU Foundation

8 年

Polls could be a lot more accurate, if we did them scientifically.

回复

Si les sondages deviennent dangereux quand leurs réglementations ?

回复
Louis-Pierre GUILLAUME

Knowledge management | Direction de projets digitaux | Change management | Coaching d’équipe | équipes internationales

8 年

Poll results depend on the willingness of the people to answer and on trust. we are all fed up with calls from call centres on the evening or WE. we lie to be left alone or to trick the system. we do not reveal our opinions to strangers if these opinions are not mainstream. I would not be surprised that some pressure groups make fake polls to skew even more the results of the "serious" pollsters.

回复

要查看或添加评论,请登录

danah boyd的更多文章

  • The Ministry of Empowerment

    The Ministry of Empowerment

    Fuck you Facebook. That was the first thought I had when I woke up this morning.

    18 条评论
  • Struggling with a Moral Panic Once Again

    Struggling with a Moral Panic Once Again

    I have to admit that it's breaking my heart to watch a new generation of anxious parents think that they can address…

    3 条评论
  • MSR AI + Orgs Research

    MSR AI + Orgs Research

    As part of Microsoft Research’s AI & Society fellowship program, I am working with Nancy Baym and Neha Shah to support…

    5 条评论
  • Degradation, Legitimacy Threats, and Isolation

    Degradation, Legitimacy Threats, and Isolation

    tl;dr: 1. New paper with Janet Vertesi: "The Resource Bind: System Failure and Legitimacy Threats in Sociotechnical…

    1 条评论
  • KOSA isn’t designed to help kids.

    KOSA isn’t designed to help kids.

    Congress is using kids to hold Big Tech accountable. Kids will get hurt in the process.

    15 条评论
  • The Screens are the Symptom.

    The Screens are the Symptom.

    I decided to re-read Fahrenheit 451 with my eldest this last week. I don't think that I have read this classic Bradbury…

    6 条评论
  • Dear Alt-Twitter Designers: It's about the network!

    Dear Alt-Twitter Designers: It's about the network!

    Last week, tech commentators were flush with stories about the speed of new users on Threads. Unprecedented downloads!…

    4 条评论
  • Seeking examples of mundane low-stakes data modeling?fails

    Seeking examples of mundane low-stakes data modeling?fails

    tl;dr: Can you give me examples of where data modeling in your everyday life is deeply flawed? I’m looking for…

    7 条评论
  • Too Big to Challenge?

    Too Big to Challenge?

    I find it deeply disturbing that the tech industry represents 9% of the U.S.

    4 条评论
  • Deskilling on the Job

    Deskilling on the Job

    When it comes to AI's potential future impact on jobs, Camp Automation tends to jump to the conclusion that most jobs…

    10 条评论

社区洞察

其他会员也浏览了