Risk Intersectionality 5: Risky risk models

The Chinese conference was Credit Rating and Credit Scoring II. Something that I was quite worried about was going off topic with my presentation, i.e., everything up to this point. From here, it becomes very relevant. Ultimately, analysts need to assess the risks.?While our focus is credit, there are also issues in the insurance industry and elsewhere.?

Expected Loss

For credit, the standard is to determine how the three Expected Loss components are affected, that is, the Probability of Default, Exposure at Default, and Loss Given Default.?Of course, this framework has its origins in the Probability and Payoff components of the Expected Value calculation much loved by gamblers.?For credit losses, it became Probability and Severity, but the latter was split to incorporate elapsed time.?

Typical Scenario

For years, the primary determinant was probability, especially for unsecured consumer lending where 100 percent losses were assumed for decision making—or that is what resulted with almost all credit scores. There is a typical pattern.?The overall risk of a portfolio may change significantly, but the relative risk of its constituent members changes marginally! Where there are shock scenarios, the impact on the relative risk will be greater, but should still recover to a new normal.?

Cupola Coupling

There are some means of assessing intersecting risks.?One is the copula, or cupola.?There is some confusion because the two words are so similar, but a cupola is a fixture to allow light or ventilation and has nothing to do with us.?The term copula has the same root as couple, or copulate. When I worked for the railroad, there were male and female attachments for the coupling.?

Copula

In any event, copulae can be used to represent multivariate joint distributions and are used to model tail risk and for portfolio optimisation. However, as a colleague of mine says, they are practically useless for extreme tails.?They work best for time series that are stationary and continuous, where stationary means a constant mean and variance.?

Irrespective, they are being used in robustness checks, stress testing, Monte Carlo simulations, and for the assessment of panic situations.?

Model Risk Topics

What is perhaps more relevant for us is to review the topics that are foremost in the Model Risk Management universe.?

It should be no surprise that climate change dominates. The other topics are accounting and capital adequacy, the transparency and fairness of machine learning models, information security and the quantification of model risk.?Each is covered in turn.

Environment

Environmental Footprint Assessments

Again, while our focus is credit, analytics plays a role when assessing our impact on the environment—the air we breathe, the water we drink and the land upon which we live.?We are spewing carbon into the atmosphere, polluting the oceans with plastics, creating massive mounds of waste, and causing huge habitat loss for other species with whom we share the planet.?These are general….

Sector-Specific Issues

… there are also sector-specific issues, especially as regards the energy transition. Unfortunately, the process of reducing our reliance on coal is problematic, especially when the costs of oil and gas have skyrocketed, renewables are insufficient to provide baseload power and nuclear is expensive and dangerous. Should the energy transition be successful, fossil fuel reserves could become stranded assets, which would be a godsend to the planet but prove highly problematic for anybody with economies built on massive reserves, or financial exposure to them. Metals and mining companies will also have an issue with climate competitiveness, given their impact on the environment.

Scenario Analysis

In terms of analytics, especially as regards credit, scenario analysis is playing a big role in the assessment of carbon earnings at risk, physical risks, and climate credit analysis more generally.?

Alignments

Of course, there are politics at play. There have been various agreements seeking to limit global warming. 2015 saw the Paris Climate Accord, ’16 the United Nations Sustainable Development Goals. And ’18 The European Union’s Taxonomy Revenue Share. Unfortunately, government actions have not resulted in much.?If anything, it is the actions of private individuals and corporations who are switching to renewables. You just have to look at my house!?

Others

Technology

Moving on to the number 2 Model Risk topic, much is being done to thwart technology’s black hats, players with malicious intents. I was lucky to grow up in the world’s 15th least corrupt country, and it was a shock arriving in another much further down the scale. Privacy was already an issue in the 19th century and is even more so with so much being digital, and analytical means are being used to secure data and guard against misuse. And then there are fraud, money laundering, corruption and terrorism. Both supervised and unsupervised learning play a role, especially in the age of AI/ML.

Accounting and Banking

In credit, it follows that number 3 is the capital adequacy of banks and accounting more generally. Credit scoring focussed exclusively on new-business processing for many years, but has increasing been co-opted... Banks are governed by the Dodd Franks Act in the United States and guidance provided by the Basel Accords elsewhere. Similarly, fair-value accounting requirements for public companies are guided by CECL in the USA and IFRS 9 elsewhere. Where lifetime values are required, there are demands that the models incorporate future projections, but without expending excessive effort given the size and complexity of the portfolios.

AI and ML

Number 4…?Artificial Intelligence and Machine Learning are today’s hottest topics, and both the benefits and risks are many.?A lot of questions must be asked.?Are they being used as intended and are they still appropriate??Do we understand how they work, which becomes more important in shock scenarios??Are people being treated fairly? as seemingly objective models can perpetuate historical patterns of discrimination. Are there any statistical biases that result from the data or the approaches used??Of these, fairness is a hot topic for the general public, especially Americans whose consumer society was built upon credit.

Quantification 1

The last Model Risk topic is its Quantification.?Our usual concerns with credit scoring are their ability to rank cases and the overall accuracy when applied to a portfolio, but there are also obvious other concerns relating to models’ robustness to changes over time and their generalisability to all of the constituent sub-populations. Issues arise if the models are used inappropriately, especially when applied to populations for which they were not intended—which often happens, unfortunately.

Issues

No matter the model, credit or otherwise, there are a variety of issues that must be assessed when prioritising models for remedial action. First and foremost is the materiality of the model… That is, how bad of a shit-storm will it be if things go wrong? Second, is transparency, how well do we understand the model’s workings??And third, is the type of model appropriate for the task?

Quantification 2

Hence, further efforts are needed. Some organisations are using a balanced scorecard approach involving surveys of developers, validators, and users.?Proper quantification can be provided through back-testing, sensitivity analysis, champion-challenger approaches and benchmarking.?But, can it ever be sufficient to handle truly unforeseen shocks?

Moving on

This is the last instalment from the main body of the presentation. The next one is the concluding remarks from the conference presentation. Thereafter, I'll be uploading a post that will like all of these presentations together.