Data-driven decisions: Should FIFA look for an alternative to penalties shootout?
Euro 2020 is over, but polemics continue. Here I want to talk about a problem that the former Spanish player Gerard Pique raised just after Spain quit the competition against Italy: "there is an unfair advantage for the team that goes first in a penalty shootout". Is this a real thing? Many journalists backed him up, and there is also a study from the London business school of Economics which argues that if you win the toss and go first, you have a 60% chance of winning.
Framing The Problem
I decided to collect some data and investigate the problem. The first thing to do is to define the question to investigate. In this case, it's pretty simple:
Has the team that start the shoutout significantly more chanches to win?
Evidence favouring this statement may lead FIFA to explore different solutions in the penalty sequence arrangement (ABBA is one suggested alternative to the alternate system ABAB).
It is also essential to bear in mind what we call the null hypothesis, namely the statement that denies what we are trying to prove:
There is no correlation between the team that starts the shootout and the chanches of winning the match
Note that the null hypothesis does not perfectly deny our question because the question is not framed correctly, but this is just a technicality, and I will stick to it for the sake of simplicity.
Analysis and Results
I found in Wikipedia the list of all the penalties shootouts of the leading three international football competitions for national teams:
It's a total of 81 matches ended with a penalties shootout. For each game, I checked who started the shootout and who won. Here's the result:
So, in 81 matches, 41 times the starting team won (50.6%) and 40 times lose (49.4%). The closest possible measure to 50%-50%, having an odd number of measurements.
We could say that starting the shoutout does not influence the outcome because our pie seems to be precisely split into two equal parts, but we have not finished yet. Is this degree of precision enough to say that starting the shootout is not a relevant factor?
Let's imagine tossing an unbiased coin 81 times. Let's repeat this action 1,000,000 times, and let's draw a pie each time like the one before. In 95% of cases, a repeated coin flip will return less balanced results (e.g. 51%-49%, 52%-48%, etc.). In other words, the analysis' results are comparable to a random outcome. A statistician would say that the Null Hypothesis is likely to be true; few things in this world are certain...
It seems that it is not important who starts the shootout. Other factors not discussed here appear to influence the result, or maybe the whole process is perfectly random, and penalties are "a lottery" for true.
In the previous image, an example of a biased coin toss. For those interested, here you can find more about the"coin toss math".
About the London School of Economics study, I would recommend the journalists add some indication about the level of uncertainty of results next time they speak about percentages. Also, this 60-40 rule appears not to be true, at least in major competitions.
Additional considerations
It is worth spending a couple of words on what happens if we analyse the three tournaments (World cup, European Championship and Copa America). Isn't it curious that we obtain a similar division even if we break out the general results in unrelated clusters?
So What?
First, I would recommend FIFA wait a bit (to gather additional data on major competitions) and don't listen to the many voices asking for an alternative to penalties shootout.
Then, I would ask Pique why, if he believes that penalties shootouts are not fair, he did not raise the problem after the match before Italy-Spain, where Spain won at shootouts and kicked the first penalty.
Additional Questions and possible flaws in the analysis
Even if the global result indicates the absence of a relationship, more data must be gathered and analysed. A massive amount of data backs up the more extensive study of the London School of Economics, which indicates the presence of a relationship: maybe the effect is present only in minor competitions? And if so, Why?
PS. To my Spanish Friends: please don't take it out on me; it's just a bit of teasing. After all, we have just won the European Championship after 53 years :)