Why Otto von Bismarck Knew More About Brexit Than the Data Analysts
The fallout from Brexit continues to elicit cries of astonishment. Having brought Britain to the brink of the unknown, David Cameron stepped down. Boris Johnson fled. Andrea Leasham imploded, leaving Theresa May to slide unopposed into the role of Prime Minister.
The UK’s proverbial jaw keeps dropping.
Check any of your news feeds; any source you read, watch or listen to and it seems everyone was stunned by the Brexit outcome that spawned this political spiral.
“Brexit: Europe stunned by UK Leave vote” – BBC
In fact, even the gamblers got it all wrong, according to The New York Times:
"Britain’s decision to leave the European Union on Thursday was a big surprise. As late as 6 p.m. Eastern in the United States, less than five hours before the results became clear, betting markets gave “Remain” an 88 percent chance to win the election, but it wound up losing by four percentage points."
And by the way, as did the financial analysts who send the prices of publicly traded markets up and down based on their deep knowledge and insightful understanding. The article continues…
The forecasting error wasn’t confined to bettors, either. Financial markets didn’t anticipate “Brexit,” resulting in a decisive correction in equity and currency markets that usually would have already been priced into trading.
To be clear, I am not an economist and although, like all, I am ready to discuss politics over a beer or bottle of wine, why then am I writing about Brexit…beyond the obvious populist reasons?
When the Brexit vote happened, I was at the annual Cannes Lions International Festival of Creativity, the most global of events in my industry, and the buzz was, needless to say, all Brexit.
During the week, I heard many political and news pundits confidently predit the outcome of Brexit but I also listened to many company leaders and digital experts pontificate about the ability to use data to do just about everything from getting you to buy what you never wanted to buy all the way to using AI (artificial intelligence) to predict that you really wanted those purple pants with the embroidered red whales months before you came to that conclusion yourself…
DIGIBABBLE…raw and unadulterated.
Way back in November of 2012 I wrote a piece on Post-Election Learning…how was it that in the US election contest between still President Obama and then candidate Mitt Romney, everyone got it wrong? I wrote then:
Data and demographics made a lot of headlines lately as pundits handicapped their way to Election Day.
Worse, even the analysis of the Digibabblists was proven wrong, as in their own self-serving way they tried to prove that Obama’s victory was all due to digital data linked to digital social expertise and primacy…turns out the Obama folks used good old-fashioned data to knock on doors and drive folks to the polls….
But I digress…returning to Brexit and The Economist in an article called, “Polls Versus Prediction Markets:”
The list of losers from Britain’s vote to leave the European Union is long indeed, but very far down on it are evangelisers for the accuracy of prediction markets. It is an article of faith among economists that betting markets on politics provide by far the most reliable forecast of future events, easily outclassing both polls and panels of experts. Yet for the two most important political developments of 2016, and arguably of the past few years – Brexit and Donald Trump winning the Republican nomination for America’s presidency – simple polling averages have put punters to shame.
The Economist continues:
How did the wisdom of crowds fail so spectacularly?
…the explanation probably lies in the familiar litany of cognitive biases that lead people astray despite their best efforts to be rational. Historically prediction markets, like horse races, have tended to demonstrate favourite-longshot bias – overestimating the chances of improbable events and underestimating those of likely ones.
Frankly, I think the explanation can be stated much more simply:
People never lie so much as before an election, during a war, or after a hunt. – Otto von Bismarck
And there you have it…
We are in danger of losing the unique human ability of deriving insight from situations that make our instincts different from the instincts of other life forms.
The more we rely on software loops and algorithms to make our decisions and draw our conclusions, the more in danger we are of creating an ever narrower world of options and, worse, an ever scarier world of surprising and startling outcomes.
Truth is that in the UK the Brexit polls were so close and the differences seemed to be statistically invalid, leading one to wonder why so many gamblers (a sure bet) seemed so sure that Britain would remain – or as The New York Times put it:
…a dose of wishful thinking from a pool of relatively affluent globalist gamblers who simply didn’t believe that Brexit could really happen.
And therein lies the answer – “affluent globalist gamblers” the likes of analysts, Digibabblists and others who would have us believe that our opinions, sentiments, beliefs and feelings are no longer relevant in a world of Big Data.
Folks…Brexit was simple…binary…yes or no…the upcoming election in the United States will be the same…how wrong will we get it?
At Cannes I was privileged to host a dinner where IN-Q, a National Poetry Slam Champion and multi-platinum winning songwriter, shared some of his deep thinking with us. He talked about the difference between ideas and ideals and said that the danger is getting locked into the rigors of an ideal that could forego our ability to create new ideas…
Or as articulated by Marshall McLuhan:
A point of view can be a dangerous luxury when substituted for insight and understanding.
Or as specifically related to Brexit, Hillary and Donald…an incredibly relevant thought from a US President of the last century…Listen:
The method of political science is the interpretation of life; its instrument is insight… Woodrow Wilson
Finally and perhaps the most stunning outcome of Brexit from Google Trends:
“What is the EU?” is the second top UK question on the EU since the #EURefResults were officially announced
A little insight and less “affluent gambler” POV might have actually changed the outcome…
And there you have it and why I wrote this from Cannes and the Festival of Creativity.
Insight, Understanding, Ideas….
Time to stop wondering how to get me to buy those whale pants…and time to ask me why I won’t…
Let’s be clear – Data – Big or Small – is obviously important and has always been so – but the winners are those who know how to derive insight from it and then use it with even more power.
And if you don’t think I have a point, I suggest you follow the growing stories of “buyer’s remorse”/second referendum coming out of the UK…many on the topic of “we just didn’t understand” or “we were fooled”…
What do you think?
Psychologist and Founder
8 年Simplify. You could have said it more simply, in one paragraph.
Make better use of your information - Data management + development projects under your control - Database Consultant
8 年I'll refrain from commenting on the actual politics, or the ways in which the campaigns were fought. The UK has seen three important and close political polls in the last three years – the Scottish independence referendum, the 2015 general election, and the recent EU membership referendum. In all three cases, the pollsters’ competence has been knocked. But I believe that the problems with these polls were quite specific to this type of national political vote, rather than being failures in collecting and processing survey data that would affect other (non-political) surveys. In all three cases, the predictive polls were close, and the results of the predictive polls were revealed to the electorate before the official election or referendum, repeatedly. These two factors meant that the predictive polls could, and (I believe) did, affect the official votes. In a close political election or referendum, turnout on the day is everything, and this is harder to predict than the question of which way voters will vote if they turn out. Regardless of how accurately a sample poll captures the intentions of voters ahead of the official vote, it’s too easy for voters to change their minds at the last minute. In a close vote, I can imagine that voters on the side that is predicted to lose (the Leave side in the EU referendum) would make an extra effort to make their votes count on the day, whereas some voters on the side that is predicted to win might feel that the result is in the bag and therefore they personally don’t need to turn up to vote. In the specific case of the Scottish independence referendum, one shock predictive poll led to a last-minute improved offering from the shocked side, which could have been a factor in the result. A last-minute improved offering wasn't possible in the EU membership referendum; if it had been possible, then the result of the EU referendum may very well have been different. Also, a subsequent analysis of the polling methods used for the 2015 general election revealed that voters with intentions to vote Conservative were relatively under-represented - and vice versa - in the predictive polls, due to a slight lack of rigour in the data collection process. That systematic error was introduced because the pressure to produce predictive polls quickly led to data gatherers on the ground having to give up on waiting to cross paths with sampled voters, to degrees that were different between the different voting groups, hence detracting from representative sampling.
Extensive Design Build, Pre-Bid through Final Construction Professional, Senior Project Management and Development
8 年Interesting read
PMO and Planning assignments at Various Clients UK & Europe
8 年David as ever a good read - Last year you posted about a piece of work trying to replicate / derive the "Human Choice / Human Heart" factor to integrate into decision making/predictive Techno. . . .As a Brit who likes IT I can say - Last year all the pundits got the UK election wrong . . . Over the last 12 months all the pundits called the Trump primary election wrong (by pundits i mean data predictors / insight experts etc ) - Circling back to your Human Factor piece from last year - If we did a survey 500 years ago based on popular responses - Big Data would be declaring the world is flat definitively & absolutely and even though there was a tiny percentage of dissenters they would be entirely covered by the margin of error at most and probably a tiny element of that margin - so their dissent should really be discarded as a microcosm/false positive - Think we need your human factor piece more than ever David - Is it still underway or nearly finished - I think a lot of big data people will be knocking on your door mate