Economic Forecasters need to learn from Pollster mistakes

The recently concluded U.S. Presidential elections had not a single Pollster predicting the right outcome. The actual outcome (a Trump win) was predicted as an outlier event by almost every poll taken before the event.

The Pollsters have got it wrong that the aggregation of fat tail and long tail distributions is fraught with risks. Preponderance of undecided voters will lead to a fat tail distribution, while a strongly biased voter segment living within this could end up strongly in the poll samples.

Here is a case where Statistics is not at fault, but the design of the experiment, when a prior event is used as a proxy for a posterior event and the sampling error, together with the sample size, biases of all kind with a host of other factors actually played havoc.

Economic forecasting may be somewhat different, but it raises a great deal of doubt as to the process adopted for the same. If the process is flawed so will be the result. In recent times we have a plethora of cases where economic forecasters went wrong about growth, prices and factor advantages.

First of all, of the very few people who were right about predicting a Trump win, Michael Moore was one and he almost correctly created the fundamental arguments in favor of a Trump win, with very little data to back up his claims. Here we have a classic case where arguments without data seem to be winning, while a deluge of data missed to collate the basic premise why the writing was on the wall.

As Pollsters create samples from a population, these samples sometimes become biased estimates of a population. For example selecting a mix of suburbs where a known bias exists should be adjusted for the bias.

Sometimes the problem lies in the framing of the question itself, which could influence the response. Framing issues are well known and adequate precaution is needed to articulate a questionnaire that would not evoke emotions that could bring in a response that is not necessarily the one the responder was actually bent upon giving.

But the bigger issues are well known that making an estimate of how many voters of a particular leaning would eventually turn up on the voting day could be a long guess.

I see in all this a pattern of a herd phenomenon, where a particular poll is acting to corroborate claims of another poll, which is otherwise known as a confirmation bias. It seemed that most pollsters where confirming a general hypothesis that they believed in and there was very little stress testing done or what-if scenarios built to challenge some of the basic foundations of the design of the poll.

The economic forecasting is actually mired in the same pitfalls, the forecasters have a general tendency to follow each other and very rarely do we see outcomes that are polls apart. However in some areas like prices of commodities, exchange rates or growth rates, we do have a range of predictions that could be wide apart at certain limited time periods.

Economic forecasting is based on linear models and multiple regressions that have many variables that correlate with each other. Sometimes we have the issue of collinearity as well, a problem when two variables are so high in their correlation that running a regression line would always be less accurate predictor of their relationship. In economics many variables interact with each other and their preponderance sometimes makes it impossible to create a cogent relationship that would be static and linear at the same time.

But if economists do not forecast they would be rendered less useful and therefore the attempts have been always to create a range in every forecast with a rider. The trick is to remind every user that the relation between variables is only limited to certain boundary conditions.

In the last one year, almost 100% of all forecasts have been wrong. The Federal Reserve never actually raised the interest rates for the last eleven months, but every forecaster had a say before the Fed meeting about the possibility of a rate rise either in the current quarter or the next quarter.

The Oil prices have remained sub-$50 per barrel for a long time, although the year started with a prediction of its steady rise. The OPEC agreeing to a production freeze was also predicted but it never materialized.

The Chinese economy was predicted to move more sideways and down, but it never happened. The Rupee was predicted to depreciate long back, but it had till late October maintained its solid position.

The economic forecasters must learn from the mistakes of the recent Pollsters. The Brexit was one other example where the Polls completely missed to see the writing on the wall.

The Bayesian Conditional Probability reminds us that no matter what may be the prior event, whether be it the Poll or the economic model used for estimating a posterior event, the conditional probability cannot be equated with the probability of the independent event.

The Pollsters have also got it wrong that the aggregation of fat tail and long tail distributions is fraught with risks. Preponderance of undecided voters will lead to a fat tail distribution, while a strongly biased voter segment living within this could end up strongly in the poll samples.

This and many other abnormalities would have to be corrected. We also should expect that Pollsters should differ in their predictive outcomes as a reflection of these aberrations.

That will be a proof that the polls are unbiased and are not behaving like a herd. Economic forecasters also should take note.