Why Predictions Fail Us

"It is difficult to make predictions, especially about the future." Yogi Berra

I remember first hearing of Y2K sometime in 1999. For those unacquainted, the idea went that when the clock struck midnight on December 31st and 1999 became 2000 computers worldwide would fail leading to mass chaos. It had something to do with our original computers never being programmed to transition from years beginning with 19- to years beginning with 20-. Our infrastructure would be compromised. The grid would crash. Life as we knew it would cease to exist for a time.

Spending New Year’s Eve with friends in Florida, I vividly recall the countdown to the ball drop. When the time arrived, I winced. Braced. And then…

Nothing.?

The anticlimactic outcome was among the first of many similar realizations throughout my life. Which is to say most predictions are wrong.?

Why are we enamored with predictions? If most are wrong (and they are by a wide margin), wouldn’t we grow wiser to their pursuit? But for as long as man has lived, predictions have engendered great interest. Biblical prophecies have been closely followed for millennia. Nostradamus’s forecasts continue to elicit response many centuries after his death. And have you ever listened to college football fans’ overly assured prognostications? Let’s just say they’re better off graded on a curve

But it’s one thing for armchair quarterbacks to issue predictions, and quite another for an expert to do so, right? Wrong. Experts are just as bad at issuing predictions as anyone else. In his book, The Wisdom of Crowds, James Surowiecki demonstrates how under the right circumstances large groups are more accurate than an elite few with specialized knowledge. Adam Grant has argued that expert predictions are historically no better than chance.?

How can this be?

One explanation can be found in overconfidence bias. Despite indicators to the contrary, presidential historian, Allan Lichtman, who had correctly predicted the vast majority of presidential elections since 1984, incorrectly predicted the 2024 presidential election. His confidence in the system he’d created led him to believe his interpretation was foolproof.?

The more knowledgeable we become about something, the more susceptible we are to the curse of knowledge. Because of this, we wrongly assume others have the same level of understanding as us which only increases the likelihood that our ideas will go unchallenged. And when our ideas go unchallenged, we tend to feel even more strongly about our “expert hunches.”

The easiest explanation may be that the future is a very uncertain place. We simply cannot account for the myriad of factors impacting how things turn out. No manner of expertise can overcome our inability to see into the future with prescient accuracy.?

We are better off adopting postures of humility. When we hold things loosely, we wear less egg on our faces than when we double down on something we cannot possibly know. A simple rule of thumb when it comes to predictions: The more consequential the matter, tread very lightly. The less consequential the matter, have some lighthearted fun.?

But in either case, don’t hold your breath.