You Can’t Make This Stuff Up… Or, Can You?

[this article is cross-posted from multithreaded and co-authored with Joerg Fritz]

On November 7, 2016 we witnessed one of the greatest moments in the history of basketball. Steph Curry of the Golden State Warriors broke the record for most 3-pointers in a game. But, he did more than that. It was a comeback of sorts - Curry had been struggling and just one game earlier he and his Warriors suffered a blowout loss to the Lakers. The lowly Lakers - the team that finished dead last in the Western conference the previous year. Curry himself went 0-of-10 from 3-point range, ending his NBA record streak of 157 straight games with a made 3-pointer. Speculation ensued. Some were beginning to question whether Curry had lost his touch. Others suggested that team chemistry had been thrown out of whack with the introduction of Kevin Durant (KD) to the roster. Still others posited that it was defense; opponents had figured out how to guard against Curry’s once-lethal shot.

But on that Tuesday night, each rumor would be rendered false. Curry shut them down one by one. The world watched in awe as each shot rained down. It was as if each one was a closing argument by a skilled trial attorney. His masterful shots - some out there from 26 feet - refuted any claim of “loss-of-touch”. He and KD fluidly choreographed their movements to get Curry the open shots. Chemistry! At times Curry had two or even three defenders on him - it just seemed to make him better, debunking theories that they had figured out how to guard him. Curry had something to prove and he brought a sledge hammer to drive it home. When he made his final 3-pointer for the night - the one that would break the record - Curry backed away, bobbed his head in triumph, and then pointed to the crowd as if to rest his case. He’s back and better than ever. The crowd, more than convinced, broke into chant “M-V-P, M-V-P”. It was a thing of beauty - a sequence of events and circumstances coming together to create one of the most epic moments in sports history. You can’t make this stuff up…

Or, can you?

Is it possible that there is another explanation for what we witnessed - one that is far less sensational? What if what we really saw that night had nothing to do with comebacks or refuting critics, and instead, was nothing more than an expected, albeit rare, sequence of events? What if that dramatic story was completely fabricated? Before you boo and hiss and cry Debbie Downer, let’s see what what the data can tell us.

If we look at what we might to expect to happen - not necessarily on this particular night, but rather what should happen on “some” night - then what we saw was not so surprising. Curry’s historical 3-point shooting percentage at the time of this game was p=44.4%. This means that the probability hitting k=13 or more of n=17 consecutive 3-point attempts is:

That is a small probability. But, given enough trials we should expect it to occur. Curry had taken 3,661 3-point attempts in his career to that date. In that amount of trials we would expect him to have had about 7 sequences as good or better just by random chance alone (he’s had 6, the other sequences just span 2 or 3 games). Narratives like ‘proving doubters wrong’ or getting ‘the hot hand’ are not necessary to explain these events.

What about the fact that Curry had an especially bad sequence of 0 of 10 in the game just before? Wasn’t this evidence of a slump and drove his determination for ‘comeback’? Unlikely. Would we have told a different story if it had been 1 of 11 instead? Let’s assume that we would have accepted any string of 17 prior shots that was as much below his normal percentage as the next exceptional 17 shots were above. How likely was it that he was to go 3 of 17 (or worse) immediately followed by 13 of 17 (or better)? It turns out that this outcome had almost a 35% chance of occurring by that point in his career.

If we look at the data graphically, we see something that looks more like a random process. Below we plot the 3-point percentage, attempts, and makes by Curry for all regular season games since the beginning of the 2014/15 season. This graph makes it far more difficult to construct a clear narrative of explanation. Would you have spotted these two “exceptional” games? They are games 514 and 515 on the x-axis. They don’t exactly leap out at you as an ‘epic moment in sports history’.

Curry’s performance was certainly a spectacle. To say it was “expected to happen” is only because he has an exceptional 3-point shooting percentage - that, and the fact that he makes many attempts. But the sensationalism - the narrative of the comeback, the refuting of rumors, etc - was created to match to the data. It was not causal. Curry didn’t break the record because of the rumors and speculation - that was our interpretation of what we saw that night. We couldn’t see it as being within normal statistical variation. We couldn’t accept that. Instead, we unconsciously fabricated a story that gave meaning to the events.

This inclination for fabrication is not limited to sports entertainment. We also fall victim to it in how we operate our businesses. It happens often - even in the most empirical of companies. When metrics vary from historical trends people are quick to offer an explanation despite the stark absence of any data to support their claims. “It’s the weather”, someone invariably will suggest in response to an unexpected uptick in business volume. Perhaps, “It’s the new ad campaign; it’s quite compelling”. Or even, “It’s macroeconomic; people are optimistic due to the jobs report that was just released”. These are all plausible explanations but they were not borne out of the data. In fact, we are often inspired to create such narratives precisely because we lack of anything else that explains the variation. We can’t seem to accept the unexplained so we convince ourselves to see patterns in the data where there are none.

This tendency to create meaning from events - even random events - is innate. Science writer Michael Shermer calls it patternicity. It’s a trait resulting from millions of years of selection pressure as our ancestors evolved the brains that we have today. Patternicity was a favored trait of our ancient hunter-gatherer predecessors. Shermer explains that those who responded to a rustling in the grass as if it were a dangerous predator were more likely to survive than their more cavalier peers - even if the rustle was merely the wind. This is because the cost of being wrong about the predator (ie. it was only the wind) isn’t much. However, the cost of not responding - believing it was the wind when it really was a predator - is devastating. It’s the cost of life and the ability to pass on one’s genes! Therefore the trait of patternicity will become prevalent in a population when the cost of making Type I errors is less than the cost of making Type II errors [1]. This would have certainly been the case for survival in our hunter gatherer times, hence we are wired with the natural inclination to assign meaning to events.

This is why we get so uncomfortable in the absence of an explanation for events we witness. It’s the same phenomenon Nassim Nicholas Taleb wrote about in his book, The Black Swan. Taleb coined the term the “Narrative Fallacy”, referring to our limited ability to look at sequences of facts without weaving an explanation into them. “We seek explanations even to the point that we will manufacture them”, he writes. Whether it’s a sudden uptick in a business metric or someone hitting 13 out of 17 three-pointers, we demand an explanation! We can’t stand to leave it unexplained or attribute it to random variation. Even for known random processes - say tossing coins - we conjure up stories to explain the outcome (e.g. “After three tails in a row, heads was due to come up”. See The Gambler’s Fallacy).

Patternicity is inherent to us because it served us well in our hunter-gatherer days. Today, however, it can be outright dangerous. While fabricating a narrative to explain our sports-entertainment may be innocuous (save for the impassioned gambler), it can be extremely consequential in business decision-making. It can lead to irrationality, poor decision-making, and bias. What’s worse is that it gives us the illusion of understanding. And, the narratives are contagious—they spread extremely quickly throughout an organization, increasing the danger of making a decision based on a false belief. We can even do so with vast confidence and in alignment with our peers.

We have to protect ourselves. Patternicity is a one of many traits where our brains don’t know what’s good for us. We have similar predilections for gluttony, avarice, lust, etc - each is a vestige that once served us well but now we must develop tactics to mitigate them. Fortunately, there are things that we can do to defend against our natural urges to make up potentially false narratives.

• First, whenever possible, and as much as possibly, rely on well-designed randomized controlled trials (RCT, aka A|B testing) for measurement, inference of causality, and decision-making. RCT insulates us against our innate temptation to make Type I errors (believing something is true when it is really false) by forcing the evaluating of data on a valid estimate of sampling variance rather than from our biased intuitions [2] 
• Next, take the time to model and the understand your business. The factors that affect your business will vary over time. The composition of customers and inventory can vary, the presence or absence of marketing campaigns can change, even the operational constraints can change and impact your business metrics. It’s a good practice to develop models to capture the variation owing to each factor (we’ve written on this before. See here). While a model won’t always give you a true causal relationship, it’s far more rigorous than using our intuition alone. Further, it provides a principled framework for analyzing variation which can be helpful in many ways: it can provide visibility into opposing factors that may be offsetting each other giving the appearance of stability. It can also provide insights in how much variation should be expected naturally. When there is some unexplained phenomenon taking place, it can provide an upper bound on how much of the variation is due to it. 
• Finally, we have to develop a discipline for when our intuition goads us into seeing patterns where there are none. If we find ourselves creating narratives absent any evidence to support it - and especially when we create the narrative because no data supports it - then we have to be very skeptical of our motives. Know when you are most vulnerable to this. The more uncertainty and the more desperation, the more likely we are to latch on to specious narratives. 

Back to sports. Was there anything more to Nov 7th than Steph Curry carrying out a routine demonstration of random variation? It’s unlikely. Curry himself, when interviewed, mentioned nothing of a comeback or having something to prove. He just said he “needed to keep shooting”. Still, we like the narratives better. It heightens the experience. It’s the same reason we go to Vegas, eat unhealthy fast food, play with ouija boards, heed the advice of fortune cookies, kiss blarney stones, and indulge in the occasional excess drinking. They are pleasurable experiences (at least in the moment) that satisfy a primordial urge. We know it’s not good for us. But, we are entitled, once in awhile, to lean into our vices. It makes life far more enjoyable. Of course, in a business setting, we need to contain our urges.

Postscript

This blog post has a recursive property to it in that patternicity is cited as the explanation for why we have the innate urge to explain the events we experience. It is entirely possible that we were involuntarily compelled by the explanation that patternicity affords. Alas, we don’t know who to trust anymore - certainly not ourselves!

References and Further Reading

1. Shermer, Michael. “Patternicity: Finding Meaningful Patterns in Meaningless Noise.” Scientific American. N.p., 1 Dec. 2008.
2. Tversky, A. and Kahneman, D. Belief in the law of small numbers,. Psychological Bulletin, 1971, Vol. 76, No. 2. 105-110.
3. The Gambler’s Fallacy on Wikipedia
4. Hot Hand Fallacy on Wikipedia
5. Gilovich, T. Vallone, R., Tversky, A. The Hot Hand in Basketball: On the Misperception of Random Sequences, Cognitive Psychology 17, 295-314 (1985).
6. Taleb, Nassim. The Black Swan: The Impact of the Highly Improbable. London: Allen Lane, 2011. Print.
7. Lewis, Michael. The Undoing Project: A Friendship That Changed Our Minds. New York: W.W. Norton, 2017. Print.
8. Kahneman, D. (2015). Thinking, fast and slow. New York: Farrar, Straus and Giroux.
9. Harari, Y. N. (2016). Sapiens: a brief history of humankind. Toronto, Ontario: Signal, McClelland & Stewart.
10. Miller, Joshua Benjamin and Sanjurjo, Adam, Surprised by the Gambler’s and Hot Hand Fallacies? A Truth in the Law of Small Numbers (November 15, 2016). IGIER Working Paper No. 552. Available at SSRN or DOI