Unskilled and unaware of it

What is it?

Sometimes it seems that the most incompetent people have the greatest self-confidence. Just take the example of the basketball fan with the big belly and flat feet who explains to his kids why the coach of the team is an idiot and what mistakes the Center made in the last 15 minutes; and who should be on the field instead. Or the coach potato, watching “Who Wants to Be a Millionaire?”, being rock-solidly sure that she would have won the million although she would not have passed the ￡500 question in the first round. Or that guy who is so convinced about his superior skills to govern a country and who conveys the message so credibly that very many people actually vote for him…

As it turns out, there seems to be a system behind it. To be more precise, a psychological phenomenon called Dunning-Kruger effect. It was Charles Darwin who apparently noticed its existence much earlier when he said: “Ignorance breeds self-confidence much more often than knowledge”. But the effect has been named after David Dunning and Justin Kruger who studied the relationship between self-perception and competence and found that incompetent people regularly overestimate their own ability. At the same time, these people are unable to recognize the extent of their own incompetence. Due to their ignorance, they are unable to increase their competence and remain blind. Which is why they regularly underestimate the superior abilities of others.

This sounds a bit like folk psychology or simply common sense – what is perhaps the reason why the effect became so popular. And why Dunning and Kruger received the satirical Ig Nobel Prize in psychology for their study and not the “serious, real” Nobel Price.

Why does it happen?

There are quite a few explanations for the Dunning-Kruger effect. The initial and most common one is based on meta-cognitive abilities (what a word!). Put simply with a quote from David Dunning this means: “If you're incompetent, you can't know you're incompetent ... The skills you need to produce a right answer are exactly the skills you need to recognize what a right answer is.”

While this sounds convincing, it might as well be that the effect is just a statistical artifact. There are studies by Ed Nuhfer and colleagues which argue that the Dunning-Kruger effect could be replicated by using random data. What happens here is known as “regression toward the mean” (see also my newsletter #29 “The curse of success”). To illustrate how this effect could explain the “observation” of Dunning and Kruger let’s take a simple thought experiment. Imagine you are asked three questions and you have no idea which is the correct answer. So you guess. In a second step, you are asked, “How many of your answers do you think were correct?” Some of you will say “no answer”, some will guess that it was one or two, and a few will bet that they got all three right. Now, if we pick out only those who were wrong with all three guesses, there must be some percentage of them who bet that they had one, two, or even three answers right, simply for statistical reasons. These individuals obviously “thought they did much better than they actually did” (actually they didn't think so). The same thing happens at the other end of the distribution, where you pick out those who had all three answers correct. Here, “they estimated themselves to have done worse than they actually did” (and again they didn't think so). Now if you draw the full graph with all the data, you get the same pattern as observed by Dunning and Kruger. However, other studies showed that incompetent people could reduce the extent of their overconfidence through training – they became much better able to understand how much they did not know and how wrong their answers could have been.

How can we avoid it?

Well, if the effect truly exists, then you can only avoid it if you are intelligent, skilled (or trained) enough to avoid it. What is another way of saying that you perhaps cannot avoid it. But let’s be fair with David Dunning and Justin Kruger! The effect they originally described in 1999 was the observation that students who are bad at a particular test think they are better than they are, while people who are very good at it tend to underestimate their results. The Dunning-Kruger effect was never about what it became popular for: that “stupid people don't know they are stupid” or that “ignorant people are arrogant and confident in their lack of knowledge.”

What’s your thinking around that?

Does this sound familiar to you? Any own experiences or stories you would like to share? Please start a conversation in the comments section!

#decisionmaking #bias