'What's Going On Here?' and other radical insights for ESGers from "Radical Uncertainty" - John Kay & Mervyn King

Radical Uncertainty - John Kay & Mervyn King

I'm a big fan of dog-earing; you know, folding down the corner of a page on which you've just read something insightful and worth reverting to later. But I don't think I've ever dog-eared any book as much as Radical Uncertainty by John Kay & Mervyn King.

I don't think I'm exaggerating by estimating the 'dog-earing' rate of this book at 25%; that is a key insight or valuable bit of wisdom on every one out four pages. With a little over 400 pages of text, that's about 100 dog ears.

In trying to figure out why this is, I identified 3 key reasons: (1) the focus is on economics and economists, but I find their insights are applicable in almost every facet of human life where decisions need to be made (that is, pretty much every facet of human life). In any case I find almost all of it applies to sustainable finance and ESG. (2) They describe a lot of things I already knew, or suspected, but provide a logical, rational and well-researched foundation for these ideas. And articulate them much better than I ever could. (3) It is not just a 400 page rant about why numbers, models and statistics are usually not the whole story, they provide real guidance on how we can all do better, with some simple, easy to remember rules: Ask yourself 'What's going on here?'. When presented with data, ask 'where does the data come from? What are we measuring?'. Distinguish between puzzles and mysteries.

Ask yourself 'What's going on here?'.

In the end, the key insight from this book for me, is that there is no substitute for informed judgment.

So, here are the dog-ears - not necessarily in any kind of logical order, but grouped as much as possible by idea or concept.

Hope you enjoy reading.

Best, Harald

Three main propositions run through this book.

First the world of economics, business and finance is non-stationary – it is not governed by unchanging scientific laws. Most important challenges in these worlds are unique events, so intelligent responses are inevitably judgments which reflect an interpretation of a particular situation. Conventional statistical inference rarely applies and forecasts are often based on shifting sands.

Second, individuals cannot and do not optimize; nor are they irrational, victims of ‘biases’ which describe the ways they deviate from ‘rational’ behavior. The meaning of rational behavior depends critically on the context of the situation and there are generally many different ways of being rational. Many so-called ‘biases’ are responses to the complex world of radical uncertainty.

Third, humans are social animals and communication plays an important role in decision-making. We frame our thinking in terms of narratives. Able leaders make decisions by talking with others and being open to challenge from them.

Able leaders make decisions by talking with others and being open to challenge from them.

1 Non-stationary

There is a limited class of problems in which stationary processes generate an observable frequency distribution, and in such cases statistical methods are powerful. But these achievements have led to many inappropriate applications of seemingly similar techniques.

Keynes’ main criticism of (econometrics) was that it assumed stationarity of relationships: ‘the most important condition is that the environment in all relevant respects, other than the fluctuations in those factors of which we take particular account, should be uniform and homogeneous over a period of time’. Keynes said of Tinbergen, ‘The worst of him is that he is much more interested in getting on with the job than in spending time in deciding whether the job is worth getting on with.’

The continued attention paid to these forecasters, and the popularity of their writings – which seems to continue even after it is obvious that they were wrong – reflects the common human predilection for apocalyptic narratives … Small-world models… are valuable in framing arguments but useless as forecasting tools.

... And the writers of the Club of Rome report all have backgrounds in natural sciences, and apply an erroneous assumption that demands for resources such as oil, electricity and water are determined by physical relationships among commodities. They ignore the impact on prices of changes in the balance between demand and supply. They also ignore the fact that, although unpredictable, technology is likely to respond to emerging challenges – the essential non-stationarity of the environment.

2 Optimization

Real households, real businesses and real governments do not optimize; they cope. They make decisions incrementally. They do not attain the highest point on the landscape, they seek only a higher place than the one they occupy now.

The innovative success of a market economy does not result from individuals or firms trying to ‘optimize’ but from their attempts by trial and error to navigate a world of radical uncertainty. In practice, successful people work out how to cope with and manage uncertainty, not how to optimize.

Behavioral economics / satisficing

Behavioral economics has contributed to our understanding of decision-making in business, finance and government by introducing observation of how people actually behave. But like the proselytizers for the universal application of probabilistic reasoning, practitioners and admirers of behavioral economics have made claims far more extensive than could be justified by their findings… Some of his suggested interventions to ‘nudge’ people towards more appropriate behavior appear sensible... We have sympathy with such policies – up to a point.

Simon … suggested that one way in which people might approach decisions in a radically uncertain world was to use a rule of thumb to search for a ‘good enough’ outcome. Such behavior was described as ‘satisficing’... Real people do not optimize, calculate subjective probabilities and maximize expected utilities; not because they are lazy or do not have the time, but because they know that they cannot conceivably have the information required to engage in such calculation.

3 Narratives

Robert Lucas, the Chicago-based father of modern macroeconomics: We are storytellers, operating much of the time in the worlds of make believe…. It is the only way we have found to think seriously about reality.

Storytelling is how humans normally try to interpret complex situations. … The power of a narrative ultimately rests on its capacity to help us make sense of a complex and confusing world. … Credibility is the consistency of the narrative with real or imagined human experience. … Credibility is closely related to coherence: a story is coherent if its components are internally consistent. … Credibility and coherence are the hallmarks of a compelling explanation.

Successful narratives survive continual challenge and unsuccessful ones are displaced – sometimes not soon enough.

Successful narratives survive continual challenge and unsuccessful ones are displaced – sometimes not soon enough.

The dominant characteristic of modern finance is the constant interplay of competing narratives. Shiller gives many illustrations of trivial or false narratives which have nevertheless received widespread attention.

The narratives we seek to construct are neither true nor false, but helpful or unhelpful. The exercise of judgment in the selection of narratives is eclectic and pragmatic.

Computers don’t do narratives.

Challenging and Changing Narratives

And prevailing opinion … has altered because there is a marketplace for ideas in which there is a tendency, not always sufficiently rapid, for valid new ideas to drive out older, erroneous ones. Knowledge is itself the subject of an evolutionary process. … Narratives change and evolve over time and need to be continuously challenged.

Narratives change and evolve over time and need to be continuously challenged.

Current conventional wisdom is embodied in a collective narrative which changes in response to debate and challenge. Mostly, the narrative changes incrementally, as the prevalent account of ‘what is going on here’ becomes more complete. … And the mark of the first-rate decision-maker confronted by radical uncertainty is to organize action around a reference narrative while still being open to both the possibility that this narrative is false and that alternative narratives might be relevant.

Beliefs are embodied in a narrative, and the prevailing narrative can change in an abrupt or discontinuous fashion when a sufficiently large number of people see evidence that leads them to change their view. … When discovering the next big thing, Steve Jobs was not selecting from a menu of existing options, but using his imagination to create something completely new.

What’s Going on Here?

A great deal of strategy work is trying to figure out what is going on. Not just deciding what to do, but the more fundamental problem of comprehending the situation.

“What’s going on here” sounds banal, but it is not. In our careers we have seen repeatedly how people immersed in technicalities, engaged in day-to-day preoccupations, have failed to stand back and ask, ‘What is going on here?’

Risk, Uncertainty, Rationality

In this book we describe the considerable confusion and economic damage which has arisen as a result of the failure to recognize that the terms ‘risk’, ‘uncertainty’ and ‘rationality’ have acquired technical meanings in economics which do not correspond to the everyday use of these words. Economists have attempted to elide that historic distinction between risk and uncertainty, and to apply probabilities to every instance of our imperfect knowledge of the future.

Discussion of uncertainty involves several different ideas:

-         Frequency – I believe a coin falls heads 50% of the time

-         Confidence: I am pretty sure the Yucatán event caused the extinction of dinosaurs, because I have reviewed the evidence and the views of respected sources

-         Likelihood – it is not likely that James Joyce and Lenin met because one was an Irish novelist and the other a Russian revolutionary

The meaning of risk is a product of the plans and expectations of that household or institution. Risk is necessarily particular.

There is no such thing as a risky asset, only a risky collection of assets, a point which is still not well understood by many investors and financial advisors today. … And the CAPM makes a distinction between specific risk associated with a particular security … and the market risk associated with general economic conditions relevant to all securities.

Risk means different things to different people.

In Buffet’s words, ‘volatility is almost universally used as a proxy for risk. Though this pedagogic assumption makes for easy teaching, it is dead wrong.’

In chapter 19 we described how pension planning in Britain, under regulatory pressure, had demanded a certainty which is unattainable, represented it with a spreadsheet of imagined numbers, and far from achieving certainty created a structure so fragile that the outcome was the destruction of the schemes whose security it was designed to protect.

Risk means different things to different people.

Models

Portfolio theory, the capital asset pricing model and the efficient market hypothesis are useful, indeed indispensable models, but none of them describe the ‘world as it really is’. When people take these financial models too literally, populate them with invented numbers and base important decisions on them, the models become misleading, even dangerous. … We are glad we know about these small-world models and we think we are better investors for knowing about them. But we do not make the mistake of taking them too seriously, and we certainly do not believe that they describe ‘the world as it really is’.

First, the modelling exercise applies a common template to disparate situations. … The belief that a common approach achieves either objectivity or comparability has been falsified by events. Second, the modelling exercises rely on filling gaps in knowledge by inventing numbers, often in immense quantities. Third, these exercises necessarily assume, almost always without justification, stationarity of the underlying processes. Fourth, in the absence of stationarity, these modelling exercises have no means of accounting for uncertainty and there is no basis for the construction of probability distributions, confidence intervals, or the use of tools of statistical inference. … They begin by considering how you would make a decision if you had complete and perfect knowledge of the world, now and in the future. But very few of the relevant data are known. The solution? Make them all up. … The expressions of probabilities disguised uncertainty rather than resolved it. … The misuse of pseudoscience to rationalize administrative decisions made in the face of radical uncertainty.

Using models appropriately: First, deploy simple models to identify the key factors that influence an assessment. Second, having identified the parameters which are likely to make a significant difference to an assessment, undertake research to obtain evidence on the value of these parameters. Third, simple models provide a flexibility which makes it much easier to explore the effects of modifications and alternatives. … (Scenarios are always useful in conditions of radical uncertainty – how might this policy decision look in five years’ time – or fifty?). Fourth, under radical uncertainty, the options conferred by a policy may be crucial to its evaluation. In the end, a model is useful only if the person using it understands that it does not present ‘the world as it really is’, but is a tool for exploring ways in which a decision might or might not go wrong.

Scenarios are always useful in conditions of radical uncertainty.

When we provide such a critique, we often hear another mantra to which many economics subscribe: ‘It takes a model to beat a model’. On the contrary, we believe that it takes facts and observations to beat a model. … If a model manifestly fails to answer the problem to which it is addressed, it should be put back in the toolbox. And if it fails to answer any economic problem, it should not be in the toolbox in the first place.

If a model fails to answer any economic problem, it should not be in the toolbox in the first place.

Subjective, personal probability

The assessment is not objective but a matter of individual judgment, and that different people may attach different probabilities to the same past, present or future event, both before and after it has occurred. In conditions of radical uncertainty, subjective probabilities are necessarily sensitive to trivial information and details of problem specification, and it therefore makes little sense to formulate or act on them.

Reference Narratives

We believe the best way to understand attitudes to risk is through the concept of a reference narrative, a story which is an expression of our realistic expectations … Since different people start with different reference narratives, the same risk may be assessed by different people in different ways. Risk is a failure of a projected narrative.

We cope with the future by organizing our lives around reference narratives. … We change the reference narrative in response to disconfirming events, but infrequently and discontinuously.

Judgment

Judgment cannot be avoided in a world of radical uncertainty. ... We have no need to fear computers: we should use them. To do that requires judgment. Good judgment cannot be summarized in twelve rules for life, seven habits of effective people… Artificial intelligence offers the prospect of ever faster ways to solve complex puzzles, but it will not resolve mysteries.

Informed judgment will always be required in understanding and interpreting the output of a model and in using it in any large-world situation.

A mystery cannot be solved as a puzzle can… But even to frame a problem requires skill and judgment. That is one of the most important contributions that economists can make. A mystery must first be framed, well or badly, to aid people in reaching the decisions they have to make in conditions of radical uncertainty. Framing begins by identifying critical factors and assembling relevant data. It involves applying experience of how these factors have interacted in the past, and making an assessment of how they might interact in the future. … The role of the economist, like that of other social scientists, is to frame the economic and social issues that political and business leaders face when confronted by radical uncertainty. The role of the practical economist, like that of the firefighter, the doctor, the dentist and the engineer, is to be a problem-solver. These other competent professionals – foxes, not hedgehogs – do not begin from a set of axioms or an overarching theory. … The good doctor begins by listening to the patient, asking pertinent questions, and gradually forming a provisional diagnosis; then reaching for specific tools relevant to a solution in that particular case.

Public Policy

Sensible – adaptive – public policy and business strategy cannot be determined by quantitative assessments of policies and projects, made by an industry of professional modelers using probabilistic reasoning. In this book we explain how it is that so many clever people came to believe otherwise – and why they are wrong.

Economists such as Paul Collier argue that these clusters of capabilities are both essential to reviving post-industrial cities and only feasible in the context of wide-ranging cooperation between governments and businesses.

Well-intentioned regulation has gone off the rails in creating an extraordinarily complicated and detailed rulebook. Such regulation cannot take into account all, or even many, relevant circumstances in a world of radical uncertainty. But the typical response to the demonstration of the inadequacy of such rules has been to write yet further rules. The recent experience of financial regulation illustrates the importance of avoiding the pretense of knowledge. We do not know when the next crisis will come, nor what it will look like. We need simple, robust principles to guide us, not tens of thousands of pages of detailed rules which elevate the duty of compliance over the spirit of proper stewardship of other people’s money. … The more regulators attempt to define precise, detailed rules, which confuse more than clarify, the more likely is a counter-productive outcome. If only someone would stand back and ask ‘what is going on here?’ rather than tweak processes which have acquired their own seemingly irresistible momentum!

The typical response to the demonstration of the inadequacy of (regulations) has been to write yet further rules.

The British government’s admirable emphasis on evidence-based policy too often reduces, as it did in the estimation of migration flows, to policy-based evidence: information is provided to support the conclusions that those who prepare the studies believe policy-makers seek…. Models are rarely used as an input to the decision-making process; their purpose is to help justify an already determined course of action. 

Struggling to cope with a large world which they could only imperfectly understand, the proponents of these calculations invented a small world which gave them the satisfaction of clear-cut answers. And financial regulators claiming to monitor risk in the financial system did the same. It is understandable that people who are given a job which they cannot do find instead a more limited ask which they can do.

Puzzles and Mysteries

A puzzle has well-defined rules and a single solution, and we know when we have reached that solution. Puzzles deliver the satisfaction of a clear-cut task and a correct answer. Even when you can’t find the right answer, you know it exists.

Mysteries offer no such clarity of definition, and no objectively correct solution: they are imbued with vagueness and indeterminacy. We approach mysteries by asking ‘what’s going on here?’ and recognize that even afterwards our understanding is likely to be only partial.

Quantification / measurement / mathiness / forecasting & planning

The Golf Club had fallen into a common and modern trap of bogus quantification. The authors of the rule believed that attaching a number to their judgment gave it an objectivity and scientific precision which a qualitative assessment would have lacked. Chicago’s Frank Knight … : Insistence on a concretely quantitative economics means the use of statistics of physical magnitudes, whose economic meaning and significance is uncertain and dubious… In this field, the Kelvin dictum very large means in practice, if you cannot measure, measure anyhow!

The belief that mathematical reasoning is more rigorous and precise than verbal reasoning, which is thought to be susceptible to vagueness and ambiguity, is pervasive in economics. But there is a difficulty of … relating variables which are written down and manipulated in mathematical models to things that can be identified and measured in the real world.

The first company… had mistaken quantification for understanding, and confused forecasting and planning. … The public sector is awash with models and metrics, and the mantra that what can be counted counts – and hence that only what is counted counts and what counts can and must be counted – has infected all areas of public and business policy. The second company… had confused aspirations with strategy. … It is today almost obligatory for organizations, small or large, private or public, to adopt some vacuous statement of vision or mission.

The mantra that what can be counted counts – and hence that only what is counted counts and what counts can and must be counted – has infected all areas of public and business policy.

Small worlds / the large world

(Leonard Jimmie) Savage’s intention (of who Milton Friedman said “one of the few people I have met who I would unhesitatingly call a genius”) was to explore a basis for the existence of personal subjective probabilities without implying that the method had universal or even general relevance to decision-making. Indeed, he emphasized that it applied only to ‘small worlds’. The distinction between a small world in which people can solve problems by maximizing expected utility and the large world in which people actually live is crucial. … Savage was careful not to claim that his analysis could be applied outside the narrow confines of his ‘small worlds’. … His caution was not shared by economists who have since been happy not only to adopt the assumption that in a world of uncertainty individuals optimize by maximizing their expected utility, but to claim that the resulting models have direct application to policies appropriate for large worlds.

Prior opinions / limits of knowledge

Having prior opinions about everything is one of the principal characteristics distinguishing the bad decision-maker. … They do not recognize the limits of knowledge – both the limits of their own knowledge, and the limits of all human knowledge of complex and evolving situations.

Having prior opinions about everything is one of the principal characteristics distinguishing the bad decision-maker.

Strategic planning / business plans / businesses

The participants at the strategy weekend were susceptible to bullshit because they did not really believe they had a problem; the offsite debate was no more than a required observance of the modern corporation. … Business leaders have often told us ‘we want someone like you to challenge our ideas’. Our experience is that the statement is almost never true.

Complex business organizations … may most fruitfully be described as collections of capabilities. And businesses and economies advance by developing new capabilities, and by applying existing ones to changing markets and technologies.

IRAC

In many colleges, students of law are taught to follow a structure described as IRAC: issue, rule, analysis, conclusion. The impressive skill of a top lawyer is to identify the issue; to give structure to an array of amorphous facts, frequently presented in a tendentious manner – that is, to establish ‘what is going on here’. … IRAC is a useful acronym for anyone engaged in the search for practical knowledge. .. It leads naturally to the two next stage of effective practical reasoning – communication of narrative and challenge to the prevailing narrative.

Scenarios

Scenarios are a useful ways of beginning to come to terms with an uncertain future. But to ascribe a probability to any particular scenario is misconceived. … Scenario planning is a way of ordering thoughts about the future, not of predicting it.

Investors / financial markets

There is no single approach to financial markets which makes money or explains ‘what is going on here’, no single narrative of the ‘financial world as it really is’. There is a multiplicity of valid approaches, and the appropriate tools, model-based or narrative, are specific to context and to the skills and judgment of the investor. We can … learn from both the contradictory narratives of the world of finance propagated by Gene Fama and Bob Shiller. But we must also recognize the limits to the insights we derive from their small-world models. … Or they are derived from historic data series and assume a non-existent stationarity in the world. Struggling to cope with a large world which they could only imperfectly understand, the proponents of these calculations invented a small world which gave them the satisfaction of clear-cut answers. And financial regulators claiming to monitor risk in the financial system did the same. It is understandable that people who are given a job which they cannot do find instead a more limited ask which they can do.

There is no single approach to financial markets which makes money or explains ‘what is going on here’, no single narrative of the ‘financial world as it really is’.

Radical uncertainty means that such calculations of correlations based on historic data sets represent a fool’s errand. In most cases we simply do not know the variance of the relevant probability distribution, or the covariances. Returns on investment are not random drawings from a known and stationary underlying process. Yet much financial analysis and much financial regulation are predicated on the assumption that they are. … Pay attention to the fundamentals which are relevant to long-term performance of different kinds of assets. Understand that broad asset categories such as ‘emerging market stocks’ and ‘real estate’ are convenient for the analysis of investment consultants, but are insufficiently granular to give insight into the real impact of diversification. … Illustrative numerical simulations may be illuminating, but are never a substitute for asking ‘what is going on here?’

Economics as practical knowledge

Marshall’s conception begins from the problem, not the method. It sees economics as a practical subject, like engineering or medicine. The engineer begins with a project, the doctor with a patient, and their success is defined by the completion of the assignment or the improved health of the patient. Similarly, the scope of economics is defined by the issues it seeks to examine – problems of business and public policy. The success of economics should be measured by the help it gives to the minister of finance and the head of the central bank … to people struggling to invest their savings or establish a business, to households buying their homes or their groceries.

Marshall’s conception begins from the problem, not the method.

Whether you had summoned the plumber to fix a leak or invited an economist to advise on policy issues, you would hope that he or she would begin with a diagnosis. You would then hope that the person had a box of tools from which it was possible to choose the relevant one or ones. A pluralism of ‘small world’ models illuminating specific issues parallels the plumber’s variety of specialist tools. Models should not be judged by the sophistication of the mathematics – in itself neither good or bad – but by the insights that model provides into a particular problem that we are trying to solve. … And there are times when there is no good model to explain what we see.

Economics & data

Like physics, economics demands data, and the world of business and finance provides plentiful data. But such data, we have emphasized, can be interpreted only in the light of some economic theory – and, as we have explained in chapter 6, are generally collected only on the basis, usually unstated, of some underlying theory. The motivating theory need not be correct … The mark of science is not insistence on deductive reasoning but insistence that observation trumps theory, whatever the purported authority supporting the theory… But we are concerned that modern economics teaching puts emphasis on quantitative methods without giving students the opportunity to learn much either about data sources or about principles by which data are compiled. Decisions about politics, finance and business should be made in the light of the best and most extensive data. But while data are essential, it is necessary to be careful in making inferences, and especially causal inferences, about the world based on data alone. … Never rely on data without asking ‘What is the source of this information?’… Here, as elsewhere, useful measurement generally requires some underlying theory or model – in an advanced country, poverty is measured in a very different way. And having asked where data come from, it is also important to ask what model is being used to interpret it.

Adapting to Radical Uncertainty

Acknowledging radical uncertainty does not mean that anything goes. Look to the future and contemplate the ways in which information technology will be deployed in the coming decades, or consider the ways in which the growth of prosperity and political influence in Asia will affect the geopolitical balance. They are all things about which we know something, but not enough; things we see through a glass, darkly. We can construct narratives and scenarios to describe the ways in which technology and global politics might develop in the next twenty years; but there is no sensible way in which we can refine such contingencies. We might, however, talk coherently about the confidence we play in scenarios and the likelihood that they will arise. As we have emphasized, the words ‘confidence’, ‘likelihood’, and ‘probability’ are often used interchangeably, but they have different meanings.

Robust and resilient plans confer positive options – opportunities to take advantage of developments which are not currently foreseen with any specificity or perhaps at all – and avoid negative ones, which close of alternatives and limit future developments to those which can currently be envisaged. Planners in modern cities benefit enormously from the foresight of their predecessors who created positive options.

Entrepreneurship

We value Frank Knight’s insight that people who challenge conventional approaches are the drivers of entrepreneurship, a source of profit opportunities, and a key dynamic of the market economy – and are in economics, as in other subjects, the source of practical knowledge.

The most common profile of the successful entrepreneur today is the individual who draws on his or her past experience in a large organization, and works from inception with a team of like-minded individuals.

People who challenge conventional approaches are the drivers of entrepreneurship, a source of profit opportunities, and a key dynamic of the market economy.

Business plans are represented as forecasts, but they are not. … It is best thought of as a narrative. The exercise of preparing the plan forces the author to translate a vision into words and numbers in order to tell a coherent and credible story.

Von Clausewitz

Clausewitz argued forcefully that good judgment was the distinguishing mark of a successful general. In On War, he describes attempts by some to reduce war to mathematical terms: ‘They wanted to reach a set of sure and positive conclusions, and for that reason considered only factors that could be mathematically calculated.’ … It is only analytically that these attempts at theory can be called advances in the realm of truth; synthetically, in the rules and regulations they offer, they are absolutely useless. They aim at fixed values; but in war everything is uncertain, and calculations have to be made with variable quantities. They direct the inquiry exclusively towards physical quantities, whereas all military action is intertwined with psychological forces and effects. They consider only unilateral actions, whereas war consists of a continuous interaction of opposites.

Pasteur: I am on the edge of mysteries and the veil is getting thinner and thinner. Fortune favors the prepared mind.