Is the Proof in the Pudding?

Thoughts on Foresight, Retrospective Coherence and COVID

I was watching the excellent?ABC Insiders Program?this morning while having an early breakfast. The burning question was whether the NSW Premier had been too late in declaring a lockdown for Sydney in the face of the current outbreak of COVID. One of the panel members replied “the proof will be in the pudding” or in other words we will only be able to tell from what happens.

That statement is both reasonable and untrue at the same time. The COVID pandemic is an object lesson in foresight because people are making decisions on incomplete information and thinking about what might happen in the future. This means there is inherent risk and we cannot assess decisions just on outcomes but also on the risks that were apparent at the time. We have a natural tendency to praise good outcomes and criticise bad outcomes without taking that risk element into consideration. Nassim Taleb, who is famous for his book on Black Swans has a great story about this. He talks about a speaker at an annual event calling you up on stage and handing you a six chambered revolver with a bullet in one chamber and telling you if they put the gun to your head and pull the trigger and don’t die then you will receive $10 million. You do this and a year later return to the annual event a year later where everyone cheers because you have built a great house, and started a fabulous charity helping victims of domestic violence. Everyone wants to adopt your strategy. Everyone has forgotten about the risk.

So lets look at the risk inherent in the lockdown decision making because if the NSW lockdown is highly successful that does not mean that the timing of that decision was right; if it drags on for months it does not mean that the timing was wrong.

First of all we have to understand that luck has a large part to play in pandemics when the infection numbers are low. This is because when numbers are low the details of individual cases, the circumstances of transmission and the status of individuals as highly infectious super spreaders or non transmitters is critical. This is somewhat analogous to the butterfly effect story in complexity where are butterfly flapping its wings can cause storm on the other side of the planet but mostly does not. For the same starting conditions you can get totally different results. When you have lots of cases that is not relevant because the numbers tend to average themselves out.

As an example of this I did some very simplistic modelling in a post that I published last month (Fooled by COVID Probabilities in Melbourne). In this very simplistic example I showed that starting with one case and no public health response you could either get to a zero transmission scenario in 35 days where there were only 3 cases in total or a very high transmission scenario where there were 7,308 cases in 35 days. If you are interested in the detail then go read the full post but essentially the difference is where in the early chains of transmission a super spreader person gets infected. They key lesson here is the high variability of possibilities before you even know what is going on.

Secondly we need to understand exponential numbers which as humans we are not great at. If we assume that people with the Delta strain infect on average twice as many people as the original strain and people with the original strain infect two people and we ignore super-spreaders for the sake of simplicity then this is what happens:

This is barely believable. I have calculated exponential growth several times for scenarios and articles in the last 18 months and I can scarcely believe it myself. What this means is that if the infection period is 4 days then the 10X column is 36 days after the first infection, but 8 days after the first case (3X) there is very little difference.

If we now take that information and look at inherent risk then there is one other key factor we need to take into account. That is that humans tend to look at other decisions that have been successful in similar circumstances and re-apply the same logic. This makes decision making in situations of high uncertainty and exponential growth problematic because we are not sure how much luck has played in previous results.

So lets look at the possibilities:

1. Previous decisions in NSW have been the right ones to deal with the Pandemic and continue to be the right decisions even faced with the Delta strain.
2. Previous decisions in NSW have been the right ones to deal with the Pandemic only because of luck in trains of transmission. Essentially they were the wrong decisions but they look like the right decisions because of the outcomes.
3. Previous decisions in NSW have been the right ones to deal with the Pandemic but are not the right ones in the face of a more infectious strain of the virus.

But risk is a combination of likelihood and impact and the impact is higher for a more infectious strain, especially in a country with very low vaccination rates. As an example here is a graph of the UK from 24th June showing a fourth wave:

The graph is from?Worldometer. As of June 24th the 7 day daily average UK case numbers were above 12,000, Almost 6 times what they were on May 18th. Daily case numbers of 16,703 on June 24th were the highest since Feb 6th. Since I captured that graph the numbers on the 25th June were 15,187 and the number for June 26th is 18,185. The cause of this surge is the Delta variant which is the same variant that is now causing problems in New South Wales. Additionally the UK is highly vaccinated.?Our World in Data?is showing that over 47% of the population has had two vaccine doses. This compares to about 5% in Australia. If this variant can cause this sort of a surge in cases in a highly vaccinated population imagine what it can do in a population that has such such low vaccination rates. The impact is very high even if the likelihood is low.

In the end all of the decision making is a judgement call. In general with low likelihood events that cause very high impact we tend to be cautious although with very very low likelihood events with very high impacts we can tend to rely on ignorance.

My judgement call is that locking down earlier was a better decision because we?MIGHT?have?been lucky so far (we just don’t know) and the consequences of being too late are fairly catastrophic (either in terms of rampant infection or long term lockdown across the whole country). The political reality is that our leaders will continue to be judged on outcomes.

For those who want more detail on how we judge in hindsight they should read the excellent The Halo Effect:?How Managers let Themselves be Deceived.

If you want to talk about how these issues effect your organisation and how foresight approaches can assist you then please?Contact Us.

If you want to receive our regular newsletter then please subscribe.