Made to Measure

Disclaimer: The statements, views, or opinions expressed herein represent my own views and not those of any employer or LinkedIn.

Recently, the outgoing Chair of APRA, Wayne Byres, made a speech to the RMA Australia CRO Conference.?The title of his speech was ‘Risk Management: Made to Measure’.?The speech is, in part, a retrospective on his time as Chair of APRA for eight years as well as a prospective on where risk management needs to go.

While there is a lot in the speech (you can find a copy here), I am going to focus just on his comments regarding measurement. Or at least my interpretation of his comments; any misunderstanding is entirely my own.

The measurement challenge

Byres’ quite rightly highlights that many of the risks that organisations face today, do not readily lend themselves to quantification.?Or at least, as he puts it, not with precision.?Byres goes onto say that not only are some of the risks that organisations face now are novel, but that their interrelationship and interconnectivity with other risk types adds to the level of complexity.?I think that the interconnected nature of risks can often be lost in how they are reported i.e., as neat discrete categories.?His main focus on the measurement challenge is with respect to climate change.?In fact, in Byres’ view, “empirical measurement, rather than a subjective judgement, is going to be needed fairly urgently”.?But I think that his statement should be applied to existing risks as well.

But what is measurement?

Typically, when people think of measurement, they are referring to a single point of measurement.?The coffee cost AUD 3.50 or they need 2 litres of milk.?In this context, most people would consider that trying to put a single value on many risks is a tall order, if not impossible.?And they would be both right and wrong!

The contradiction lies in the definition or meaning of measurement.?Yes, a single value is a tall order but that is not how measurement should be approached.?The trick, is not to think of measurement as a single value, but as a range with (a given outcome).?

Concept, Object & Method

Douglas Hubbard has written extensively on the measurement challenge in the context of risk management.?The below is taken from ‘How to Measure Anything’.?It, and his other books, are well worth reading but I’ll do my best to briefly and faithfully summarise his comments.?In short, measurement has three components:

• Concept.?Measurement should be thought about as a quantified reduction in uncertainty.?Take someone’s height.?If you think of it as trying to guess it to the exact centimetre then there is a good chance that you will miss it if even only by a little bit.?But the task suddenly becomes a lot easier if you start to think about the measurement in terms of ranges, say 165-180cm.?A reduction in uncertainty can be as little as going from ‘I don’t know’, to risk ‘x’ could cost us up to $100m!?Yes 0-$100m is a big range and indicates a huge swath of possible outcomes.?But it gives size and scale to the problem in a way that ‘High’ does not.
• Object.?Put simply, what are you trying to measure and why do you care??Hubbard argues that often insufficient thought is given to what and why you are trying to measure something.?If you are not clear about it and what you intend to do with that information, then measuring something like ‘climate risk’ will remain a herculean task.?Really understanding the ‘what’ and ‘why’ helps to really focus on what it is you are trying to do.
• Method. There are more means and methods available than most people initially appreciate.?In addition, often much less data is needed than is typically thought.?If you bear in mind that measurement is ultimately about a reduction in uncertainty, then any ‘reduction’ will bring value.

Can you put a value on it?

Many may find it objectionable to put a value on things such as the climate, reputation or indeed a life.?But if we do not at least attempt to attribute a range of values, it becomes exceptionally difficult to prioritise competing demands.?If I have a credit provision of $500m, a compliance risk of Medium Low and a Reputational Risk of Medium, how do I prioritise between them??If the compliance risk is a low frequency event that could include the loss of license, then surely this must represent a potential loss of value to me (i.e., the value of my business)??

Why does any of this matter?

I firmly believe that if we take the approach of Douglas Hubbard and others like him, then the issues that Byres raises are solvable.?If many of the risks that organisations face could ‘only’ be quantified in terms of outcomes in the tens or hundreds of millions, then that would be a far better basis for decision making then any ‘High’ or ‘Medium’ assessment could give.

I don’t doubt for one second that some of the issues we face will be very hard to measure but whether it is Sustainability Risk, Operational Risk or Strategic Risk, the opportunity presented by reorientating our thinking on measurement are immense.?I think that the value that this would bring to organisations would be, well, immeasurable!

Further Reading

APRA Chair Wayne Byres - Speech to RMA Australia CRO Conference | APRA

How to Measure Anything Book | Douglas Hubbard (hubbardresearch.com)