55% believe they're in the top 10% - Exposing our bias

Let's talk self-bias. But first, a warning:

A recent study at Columbia University found that 41% of social media readers don't read sourced articles beyond the headlines. If you think you know where we're going, you don't. Read on.

Earlier last week, I posted this simple question on LinkedIn: How good are you at what you do? It was followed up with a quick acknowledgment that I could see how people voted, but I wouldn't look. I didn't look.

I didn't pose the question to prove a point. All told, I was interested in offering a question where the safe answer would be to pick the middle, but there was no middle. Offering an even number of answers when asking people about their performance forces them to choose better or worse than normal. (Low-grade evil, I know, but asking partially-loaded questions on social media is the least of my vices.)

137 people responded. Of those 137 responses, 76, or 55%, said they were in the top 10%. Another 55 (45%) responded as being above average. To put this into perspective, 90% of respondents said they performed better in their field than most.

Allow me to introduce the Dunning-Kruger Effect. In a nutshell, it is a phenomenon where people who are less skilled in a particular task will overestimate their competency. You don't know what you don't know, so you think you're doing better than you are. Remember this. It is important.

Statistically, the responses to this question should not have been so heavily weighted toward the top 10%. If you don't know what a normal distribution is, I promise you already intuitively understand. This is what we would have expected the results to look like:

50% below half, 50% above half. We effectively asked people if they were 1 in 10, and almost 6 in 10 said "yes."

Stopping here for a second, how are the results different from what you expected? Why do you think the results differed? What does this tell us about the people who answered? What does this tell you about yourself?

Now consider the Dunning-Kruger Effect again. How good are these respondents at estimating their own competency? How good are you at estimating their bias?

What you are witnessing is an example of the Dunning-Kruger Effect in devastating detail. Why devastating? Because I fell for it, and it was devastating.

You see, the capabilities overestimated are not the respondents' abilities in their profession. The joke isn't on those who responded in the 10%. The joke is on us. The reader. On me. Probably on you.

How competent do you really think you are at identifying the bias and why it exists? What do you really see in those results? Be honest with yourself. Did you feel a sense of satisfaction reading the chart knowing you probably did not respond?

Don't feel bad. I did. It's natural. It's the Dunning-Kruger Effect.

At first take, I overestimated my ability to identify bias. My own little social media game backfired. The easy answer to the skew in the responses is Dunning-Kruger. Look at how smart people think they are! Look at how dumb people are compared to what they think! I can spot stupid people from a mile away! Haha!

"But, wait a minute," you say. Why aren't the results of the poll an example of the Dunning-Kruger effect?

They may be. We don't know. But we do know that our overestimation of our ability to identify bias and assume we know why is the Dunning-Kruger Effect. There is no spoon. Meta-Dunning-Kruger.

Here's why the results aren't what we think they are:

Consider that 55% of the 137 said they were in the top 10% in their field. That shouldn't be possible. But what if I told you it wasn't 55%? What if it was only 2% said they were in the top 10%?

We received responses from 137 people, but over 3,300 people saw the poll and chose to not respond. Here are the proper results:

Why didn't they respond? All sorts of reasons. Maybe they caught on to my evil little soon-to-backfire scheme. Maybe they felt they were in the bottom 50% and didn't feel like announcing it - even anonymously. Maybe they didn't believe me when I said I'd keep the results anonymous. Maybe LinkedIn is a better venue for announcing intentions rather than introspection. Maybe 96.1% of the world's left mouse buttons and touch screens suddenly stopped working.

The point is: we don't know.

Here is what these results do say: of the people somehow connected with me who saw this poll and are of the personality, mood, and comfort to respond publicly, 55% believe they are in the top 10% of their field.

I must point out that it is very possible that this truly is the case. What we want to see in the conclusions aren't necessarily conclusions at all.

Now, reconsider every other "data-driven" analysis offered as proof of a simple story. Is it in their best interest to actually consider all data, or did they stop when they found the story they wanted?

"There are lies, damn lies, and statistics." - Mark Twain, who attributed the saying to British Prime Minister Benjamin Disraeli (which nobody knows whether he actually said it).

But the problem isn't data. The problem is us. We believe what we want to believe and are being fed "data-driven" conclusions that aren't conclusions at all.

As concluded by the Columbia University study, the chance that we're even reading beyond the headline is already low. You did better. You're in the top 41%!

The next step is in questioning our own bias, even if the topic is someone else's.

Stay curious.

If you like insights critically arrived at by analysis and questioning of data, we do have a free newsletter. We also really like it when the subscriber numbers go up. Data-driven dopamine, we call it. We'd love it if you gave us our next dopamine hit: https://weekly.visualapproach.io/