Re-thinking Election Forecasting Based on Horse Race Polls

By David Iannelli, Amber N. Ott and Andrew Sullivan

In the aftermath of Trump’s win on Tuesday, it’s time for a serious re-think of election forecasting that relies too heavily on horse race polling.

We write this as readers of FiveThirtyEight, TheUpshot and other polling aggregators. It’s clear there are many reasons the political and media establishments whiffed on predicting Trump’s win. The size and makeup of the electorate were misread; no one had a bead on turnout; assumptions based on the 2012 election turned out to be wrong; the impact of Clinton’s ground game was overestimated; averages of publicly available polls offered a false sense of certainty; and few polls were conducted in key states in the final week before the election.

And yet. There are fundamental, troubling problems with using polls to forecast elections. First, there is more to a poll than the horse race, but that is typically all that gets reported in the media. The horse race is just the tip of the iceberg. As Trump pollster Adam Geller put it, “Simply reading a top line number and reporting it as a results is not analysis. It is not even close.”

Second, people expect too much from polls, which are not designed for making predictions. Polls are designed to give a snapshot of opinions at a moment in time and are not absolutely precise. They are only predictive if nothing changes?—?and voters don’t change their minds?—?between the day the poll is taken and election day.

In this election, voters did change their minds. Nationally and in key states such as Wisconsin and Pennsylvania, Trump prevailed among voters who said they decided which candidate to support in the last week before voting.

So here’s how we see it. The media’s election forecasting industry has been built upon a misunderstanding of the highest purpose and value of public opinion research. A misunderstanding which they surround with shiny objects in the form of engaging forecasting tools and models. These tools are so intuitive we have become addicted, covering a blip in the polls but not considering the totality of the poll and the story that it tells or exploring the values which underpin the American electorate’s preference in candidates.

We need to do better. And doing better means focusing on the issues, understanding the public and how people expressed their values on Tuesday.

There is no more valuable instrument for understanding the public than opinion research. And this value is tied in a fundamental way to asking questions. By asking questions, polls help us understand public preferences and expectations, how the public thinks and views the world and how the public reacts to different ideas.

This is an important democratic function which we should be concentrating on now. What are the frustrations and hopes of the roughly 50 percent of the electorate who supported Donald Trump? Likewise, what are the frustrations and hopes of the roughly 50 percent of the electorate who supported Hillary Clinton? How about the views of the many millions of Americans who did not vote? Understanding the areas of overlap among these groups will give us a roadmap for American policymaking in the weeks and months ahead. (And will help reduce blood pressure in cities across the country.)

We’ll close with a tip of the hat to the Gallup Organization and its founder George Gallup, a pioneer of survey sampling techniques. In the face of criticism last year, the Gallup Organization chose not to conduct presidential horse-race polling, turning its focus instead to measuring voters’ attitudes and opinions on a range of issues. The group’s decision is a reflection of its founder’s philosophy: George Gallup conducted political polls only to prove accuracy of his sampling techniques. “The polls themselves, he thought, were pointless.”

It’s encouraging to hear that efforts are already underway in the polling industry to analyze this election and make improvements in techniques and polling accuracy. But we hope that in 2018 and beyond, the political media and election forecasters will focus their energies?—?and audiences?—?on the issues and values motivating voters, and not just Las Vegas style oddsmaking.

This piece also appears on Hudson Pacific's news site.