Can you solve this statistical riddle better than Derren Brown?

If you, by chance, don't yet know who Derren Brown is, just do a search on YouTube, watch a few of his videos (TV shows are great, but live performances are even more impressive), then come back.

Why? Because he is a great showman and I think everyone can learn something about himself through enjoying his performances.

His book from 2006, "Tricks of the Mind" is also quite a nice read, sprinkled with his unique self-conscious humor. He touches on many controversial topics, such as alternative medicine, psychic reading, New Age bs, NLP, vaccination scares, etc. and does a fair job at exposing the lack of support for these manipulations. In his book and in his shows he shows how some of the apparently "psychic" or "extra-sensory perception" phenomena actually works.

He also touches on common cases where most people's intuition about statistical measures tricks them into making a bad judgement or prediction. It is in one of those example where I couldn't help but notice a significant error in a riddle he gives his readers, so I thought I'd share it with you, so you don't make the same error if you come across such an issue.

Here is how the riddle is presented in the book:

"Harry was very creative as a child and loved attention. He didn't always feel 'part of the gang' and this led to a desire to try to impress others with his talents. He went through school rather self-obsessed, and tried his hand at any creative field. He really enjoyed any opportunity to give a presentation or to show off in front of an audience.

Take a look at the following statements regarding Harry as an adult, and place them in order of most likely to least likely:

1) Harry is an accountant.

2) Harry is a professional actor.

3) Harry enjoys going to classical concerts.

4) Harry is a professional actor and enjoys going to classical concerts.

Please, go ahead and mark them in the order you think is appropriate, before reding any further."

This is the riddle. Do take your time and think through it. No cheating!

Now that you've made your call, here is how Derren sees the answer to the riddle:

"Did you decide Harry was more likely to be an actor than an accountant? Mistake number one. There are many, many more accountants in the UK than professional actors. Can accountants not be self-assured and good speakers? Because the description fitted your sense of how a 'typical' actor might describe his background, you were blinded to making a sensible estimate.

Did you decide that number four was more likely than number three? Stop and think: how can 'Harry is a professional actor and enjoys going to classical concerts' ever be more likely than just 'Harry enjoys going to classical concerts? In number three, Harry could do anything for a job. The probability of two pieces of information being true is necessarily lower than that of just one of them being true. It sounds obvious now, but we still tend to blindly choose the necessarily less likely option as the more likely."

Now think about his two points above. Don't you think there is something fishy in at least one of them? Maybe both? Take your time.

And now, my take on it: Derren is wrong with the first part of his solution to the riddle, where he says that Harry is more likely to be an accountant than a professional actor and he justifies it by the fact that there are many more accountants than actors in the UK.

The error here is that the statistical information he applies does not have any bearing on the question he poses. Since we know a lot about Harry, we are in fact narrowing the pool of potential people to a small portion of the population. Probably no more than 5% of the UK population would match the description of how Harry was as a child. So, if we are to make predictions about these 5%, knowing something about the 100% of people in the UK might be useful, but it might also be highly, highly misleading.

Why?

Well, if we know not only that there are many more accountants than actors in the UK, but also that they are evenly spread among the population according to various metrics, including childhood experiences, only then it would be valid to make the inference Derren makes.

However, this is a huge assumption on Derren's end that he fails to communicate explicitly. We don't know if people are evenly distributed and, in fact, it is generally not likely that they are, as a vast number of human characteristics follow either a normal (Gaussian) distribution, or a Pareto distribution.

In short, it is entirely possible that while there are, say, 100 times more accountants than actors in the UK, most actors come from the 5% of people like Harry, while almost no accountants come from those 5%. So, Harry's chance to be an actor as an adult may be 50/50, or 90/10, or 10/90... Unless we have some data particular to those 5%, we can't really say!

Since Derren doesn't give any such data and bases his conclusion solely on the prevalence of accountants vs actors in the general population, as opposed to the relatively small percentage of children like Harry, his conclusion is not warranted.

What this shows is that statistics is hard and even well-meaning and well-read people can get it wrong. Don't be fooled by statistical intuition - educate yourself and avoid being made a fool by your own flawed perceptions. After all, without good stats, whatever we do won't have predictive value, so it will be worthless or worse - will mislead us or give us false confidence.

Anyways, that is my take on Derren's solution of the riddle, let me know if you think I made a mistake ;)

P.S. Even if children like Harry were 50% of the population, Derren's argument would be strong, but still not as strong as he makes it to be. It could still be that most accountants come from the other 50%, while most actors come from Harry's 50%.

#statistics #riddles #datascience