Polls did not see what’s coming, again

Two weeks after the presidential election in the US, the world has already adjusted to the choice of American voters, who decided that Donald Trump would become the 47th President of the United States. Currently, the hottest discussions revolve around Mr. Trump’s nominees for the upcoming government. Focused on debates about the candidates and how qualified or unqualified they are, we have already forgotten one of the greatest disappointments of the election night. For the third consecutive time, pre-election polls underestimated Trump’s performance and failed to properly estimate his voter base. What was expected to be a tough and close race turned out to be an impressive, one-sided victory. The embarrassing inaccuracy of American polling firms is hard to understand, explain and forgive. Moreover, it raises questions about whether we can trust market or consumer research for decision-making. In this post I’m? exploring this issue in the context of my professional experience in digital banking.

Consistent inaccuracy 2016 - 2024

This November we were promised the closest presidential race in years. Just before the election the average of the polls was suggesting the minute advantage (one percentage point or so) for the Democratic candidate Kamala Harris. Those who were thriving to see a narrow, exciting battle on the election night would’ve been disappointed. Donald Trump got the land-slide victory, exceeding Mrs. Harris by two percentage points in popular vote. That inconsistency between polls and actual outcome is surprising for it happened for the third time in a row that polling firms underestimated Trump’s performance. One could’ve assumed that eight years should’ve been enough to learn how to optimize data analytics to account for the systematic bias in the polls. Apparently, the pollsters didn’t learn quick enough.? The problem have been known since Donald Trump unexpectedly defeated Hillary Clinton in 2016. It have been discussed? in the pre-election time in 2020 and the common conclusion was that it would not happen again, because polling companies could not afford the same mistake again. Yet, Joe Biden’s victory was not as spectacular as poll-based estimations. The numbers (according to concise analysis from Reuters) clearly shows the lack of progress in polls accuracy. In three consecutive elections: 2016, 2020 and 2024 polls underestimated Trump’s result by 2, 4, and 3 percentage points, respectively. Pew Research Center gives even more convincing figures which shows, that the inaccuracy increases over time: 1.3 percentage point miss in 2016 versus 3.9 in 2020. It does not seem as any substantial progress in pollsters’ accuracy to me.

‘Harris narrowly favoured to win” - one of the polls in favor of Harris’ win – Focal data final forecast on November 4th, 2024,? (source: www.focaldata.com), the prediction was 276:262 for Harris (actually it was 312:226 for Trump)

If leaders can’t do it, what about regulars?

The conclusion which could be drawn from the American polling business’ pathetic inability to improve voting prediction raises important questions about the whole category of customer research products. Election polls are the most important example of customer research in US and, arguably, in the world. If this multi-million industry can’t handle the well-known, repeatable mistakes, fix them and eventually provide better estimations, how reliable are results of other, less prominent surveys? How data-driven business can perform if there is so much bias in customer research? But, maybe, polls regarding less critical issues, for example preference of banking customers, are more predictable? Actually, there are some factors which could make them more accurate, including more tools validating polling results. Customer preferences are best measured by their behavior, which can be track continuously, not only every four years. Americans shop every day, but vote for the president quadrennially. Elections polls are harder to fix due to less frequent feedback, the rare moments of reality check. In well-monitored businesses systematic bias of customer surveys is quickly identified and, consequently, eliminated.

Facts, not opinions

Customer surveys are as good as their factual confirmation. This is not new or controversial that in business facts should prevail over opinions. Customer centric business caters for actual customer needs, not for those expressed in surveys and interviews. There are many behavior-based methods to understand customer base, more precise, reliable, and effective than polls and interviews. Still the latter are used extensively, due to their simplicity, availability, and ease of interpretation. They play an important role during design phase of product development and strategic preparations. However, there is a limited trust in them among experienced product managers and strategy leader. They use many different tools to validate results from customer polling, they value facts more than opinions, even those of customers. Once the product is launched or strategy deployed, the direct readings from the business performance become the most valuable source of data-driven decision making.

Digital banking – typical cases

To illustrate the process I will go through two common situations in? professional life of digital banking expert. Let’s start with mobile app’s UX development. Customer research is usually a major driver of change. UX is altered to meet users’ needs which means that users’ complaints are a major source of inspiration. However, the direct user feedback is? extensively amended by various experimental methods which reveal the actual customer behavior rather than opinion. The first priority is to improve the elements of design where customer stuck and get confused. They are identified by traffic monitoring (in real time, retrospectively and in the lab experiments). Clearly, UX improvements are based more on facts than polls’ results. The other example – the functional enhancement of digital channels. The badly designed mobile banking often comes from wrongly applied “we listen to our customers” attitude. ?What could be wrong in such strategy? Well, a lot, and I can relate to many situations experienced during my career in digital. The common mistakes include listening to wrong, not representative group of customers, following every suggestion without validation of its importance, and altering or removing functions actually used by customers, but not included in their feedback. The fact that humans prefer to complain about wrong stuff rather that praise good things does not help. Again, monitoring of actual user preference and behavior helps a lot. In any scenario those who work in digital banking got much more comfort that political analysts for we can always confront polls and reality, opinions and facts, without obligatory waiting for another election which will be, well … in four years from now.