Probability - Lies, Damned Lies, and Statistics

'Probability' is the statistical likelihood of a particular event occurring, usually expressed as a ratio of the specified outcome to the total range of possible outcomes ... eg a 1 in 10 chance, 10% chance etc. Real-life scenarios are most generally a combination of concurrent independent events, which is where the maths get interesting and the understanding of true likelihood grows less intuitive.

Let's look at a fictional example; a newspaper headline screams 'COFFEE DOUBLES HEART ATTACK RISK' before sumarising a 2-year medical study that suggests coffee drinkers were "twice as likely to suffer heart problems as those who preferred tea". So far, so alarming. But, as with most hysteric newspaper articles, some further scrutiny is warranted.

One key issue is the size and selection of the sample cohorts used. A small sample set can only provide a less than representative representative result, while selection effects can introduce bias - were there any differences between the cohorts, for instance, was there a greater proportion of obese or older participants in the coffee sample?

How was the data gathered? - surveys commonly (wittingly or unwittingly) introduce biases through the nature of the questions asked.

Underlying all these concerns is the concept of statistical significance - do the observed results differ meaningfully from a normal or expected outcome?

Basically, is this headline grabbing soundbite merely a quirk of random chance.

Then of course there are the actual numbers - which generally don't make it into the headlines. Would you be really deeply concerned if your coffee drinking impacted risk, at say 0.00000001%, was double that of the tea drinker's 0.000000005% chance of a heart attack? It's the same outcome, merely stated in a less sensational manner. Presentation matters.

Examples of the use (and mis-use) of statistics abound in everyday experience, particularly when earnestly presented by those looking to sell a product or influence opinion. Consider the small print that usually flashes across the your TV screen during advertisements.

"Proven to reduce the signs of aging" they proclaim. Really? ... tell me more.

The briefly displayed small print states "60 of 113 customers surveyed agreed that our product helped them look younger" Well, let's examine that statement in some more detail.

60 of 113 participants - not too far from the 50:50 chance of a simple coin toss, and certainly not statistical significant.

Hmm OK, so who took part in this survey?

Answer - "Customers".

Presumably people already purchasing this product - and if they're willing to shell out ￡10 for a small pot of cream, they're going to be inclined to believe it must be effective. Hardly an independent sample.

How was the test conducted?

Answer - "a survey"

Not generally a scientific approach, particularly when no details are given on the nature of the questioning, how participants were chosen, if there was a control group etc.

And the claim is that they 'agreed', which prompts the further question - what exact statement were they asked to agree with?

Overall, the only thing that can be concluded from this evidence is that there are at least 53 people who appear willing to buy an expensive beauty product despite their failure to observe any benefit!

It's a trite example in some ways, but it does highlight some of the concerns investors and entrepreneurs should raise when evaluating investment opportunities or considering where best to allocate capital or reduce costs.

As we've mentioned in previous posts, the intrinsic value of a business is ultimately driven by the size, timing, and probability of future excess cashflows.

There's plenty more relevant ground to cover on this subject and we'll return in more detail in future articles.