Banking Models and Risk Dynamics: Navigating the Complexities of Financial Quantification

The term model refers to a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into quantitative estimates. A model consists of three components:

1. an information input component, which delivers assumptions and data to the model;
2. a processing component, which transforms inputs into estimates; and
3. a reporting component, which translates the estimates into useful business information.

Models meeting this definition might be used for analysing business strategies, informing business decisions, identifying and measuring risks, valuing exposures, instruments or positions, conducting stress testing, assessing the adequacy of capital, managing client assets, measuring compliance with internal limits, maintaining the formal control apparatus of the bank, or meeting financial or regulatory reporting requirements and issuing public disclosures.

The definition of a model also covers quantitative approaches whose inputs are partially or wholly qualitative or based on expert judgment, provided that the output is quantitative in nature. A model may combine assumptions, data, and hypotheses about the behavior of markets or individuals, and process these inputs into quantitative estimates, forecasted outcomes, or predictions.

Models are simplified representations of real-world relationships among observed characteristics, values, and events. Simplification is inevitable, due to the inherent complexity of those relationships, but also intentional, to focus attention on particular aspects considered to be most important for a given model application. Model quality can be measured in many ways: precision, accuracy, discriminatory power, robustness, stability, and reliability, to name a few. Models are never perfect, and the appropriate metrics of quality, and the effort that should be put into improving quality, depend on the situation. For example, precision and accuracy are relevant for models that forecast future values, while discriminatory power applies to models that rank order risks. In all situations, it is important to understand a model’s capabilities and limitations given its simplifications and assumptions.

Because assumptions are typically simplifications of the actual relationships between inputs and outputs, and hypotheses about behaviour are imprecise, there is some uncertainty associated with a model’s estimate of the outputs, resulting in prediction errors.

Various models may focus on discriminatory power or predictive power as measures of model accuracy. Discriminatory power assesses a model’s rank-ordering property, while predictive power focuses on the model output’s prediction accuracy. A model focusing on discriminatory power need not produce the most accurate prediction, in the same way, a model with the most accurate predictive power need not produce maximum rank ordering.

?In contrast to a model, a quantitative tool not meeting the definition of a model described in the MRM Supervisory Guidance may apply deterministic rules or algorithms( An algorithm is a set of computational rules to be followed to solve a mathematical problem. More recently, the term has been adopted to refer to a process to be followed, often by a computer) ?to process information and produce outcomes defined by the deterministic rules.

For example, a tool can include spreadsheet calculations using algebraic formulas, such as summation, or values for which the output is certain. Outputs produced by quantitative tools that are not models generally do not rely on sensitivity analysis or other methods to develop quantitative estimates, forecasted outcomes, or predictions. The determination by a bank of whether a quantitative tool is considered a model is bank-specific, and a conclusion regarding the tool’s categorization should be based on a consideration of all relevant information. Risk management should be commensurate with the extent and complexity of the quantitative tool used. Risk management for quantitative tools that do not meet the definition of a model described in the MRM Supervisory Guidance may be significantly less robust than risk management for models.

Banks rely heavily on quantitative analysis and models in most aspects of financial decision-making. They routinely use models for a broad range of activities, including underwriting credits; valuing exposures, instruments, and positions; measuring risk; managing and safeguarding client assets; determining capital and reserve adequacy; and many other activities.

The expanding use of models in all aspects of banking reflects the extent to which models can improve business decisions, but models also come with costs. There is the direct cost of devoting resources to develop and implement models properly. There are also the potential indirect costs of relying on models, such as the possible adverse consequences (including financial loss) of decisions based on models that are incorrect or misused. Those consequences should be addressed by active management of model risk.

Models can help increase the automation, transparency, and consistency of bank activities. The number, scope, and complexity of models continue to increase over time. Examples of model uses include

• underwriting and managing credits.
• valuing trading exposures.
• pricing.
• risk hedging.
• managing client assets.
• measuring compliance with

-?? internally established limits.

-?? laws and regulations (including consumer protection-related laws and regulations).

• estimating the allowance for credit losses (ACL) and capital adequacy.
• issuing public disclosures.
• preventing and detecting fraud and money laundering.

The expanded use of models combined with their increasing complexity and value in decision-making underscores the importance of sound model risk management. Additionally, the incorporation of alternative data contributes to model complexity while expanding access to credit and producing benefits for consumers.

Technological and analytical advances are contributing to increased model complexity and use. For example, artificial intelligence (AI), ?including machine learning, ?is used in a? variety of ways. AI is broadly defined as the application of computational tools to address tasks traditionally requiring human analysis. Examples of AI use in banks include fraud detection and prevention, marketing, chatbots, credit underwriting, credit and fair lending risk management, robo-advising (i.e., an automated digital investment advisory service), trading algorithms and automation, financial marketing analysis, cybersecurity, Bank Secrecy Act/anti-money laundering (BSA/AML) suspicious activity monitoring and customer due diligence, robotic process automation, and audit and independent risk management. Some AI may meet the definition of a model noted in the MRM Supervisory Guidance. While AI outputs are not always quantitative in nature, AI is typically based on complex mathematical techniques. Regardless of how AI is classified (i.e., as a model or not a model), the associated risk management should be commensurate with the level of risk of the function that the AI supports.

Risks Associated With the Use of Models

Risk is the potential that events will have an adverse effect on a bank’s current or projected financial condition and resilience. ?There are eight categories of risk for bank supervision purposes: credit, interest rate, liquidity, price, operational, compliance, strategic, and reputation. These risks are not mutually exclusive. Any product or service may expose a bank to multiple risks. Risks may also be interdependent and positively or negatively correlated.

Model use can affect risk in all eight categories of risk. The use of models can increase or decrease risk in each risk category depending on the models’ purpose, use, and the effectiveness of any relevant model risk management. Conceptually, model risk is a distinct risk that can influence aggregate risk across all risk categories. Model risk can increase due to interactions and dependencies among models, such as reliance on common assumptions, inputs, data, or methodologies.

The use of models invariably presents model risk, which is the potential for adverse consequences from decisions based on incorrect or misused model outputs and reports. Model risk can lead to financial loss, poor business and strategic decision-making, or damage to a bank’s reputation. Model risk occurs primarily for two reasons:

?

1. The model may have fundamental errors and may produce inaccurate outputs when viewed against the design objective and intended business uses. The mathematical calculation and quantification exercise underlying any model generally involves the application of theory, choice of sample design and numerical routines, selection of inputs and estimation, and implementation in information systems. Errors can occur at any point from design through implementation. In addition, shortcuts, simplifications, or approximations used to manage complicated problems could compromise the integrity and reliability of outputs from those calculations. Finally, the quality of model outputs depends on the quality of input data and assumptions, and errors in inputs or incorrect assumptions will lead to inaccurate outputs.
2. ?The model may be used incorrectly or inappropriately. Even a fundamentally sound model producing accurate outputs consistent with the design objective of the model may exhibit high model risk if it is misapplied or misused. Models by their nature are simplifications of reality, and real-world events may prove those simplifications inappropriate. This is even more of a concern if a model is used outside the environment for which it was designed. Banks may do this intentionally as they apply existing models to new products or markets, or inadvertently as market conditions or customer behavior changes. Decision-makers need to understand the limitations of a model to avoid using it in ways that are not consistent with the original intent. Limitations come in part from weaknesses in the model due to its various shortcomings, approximations, and uncertainties. Limitations are also a consequence of assumptions underlying a model that may restrict the scope to a limited set of specific circumstances and situations.

Banks should identify the sources of risk and assess the magnitude. Model risk increases with greater model complexity, higher uncertainty about inputs and assumptions, broader use, and larger potential impact. Banks should consider risk from individual models and in the aggregate. Aggregate model risk is affected by interaction and dependencies among models; reliance on common assumptions, data, or methodologies; and any other factors that could adversely affect several models and their outputs at the same time. With an understanding of the source and magnitude of model risk in place, the next step is to manage it properly.

The risks associated with model use can occur at any point during a model’s development, implementation, use, and validation. A bank’s risk profile can increase depending on a model’s complexity, the technologies used to implement models, higher uncertainty about inputs and assumptions, broader model use, larger potential impact on the bank’s financial condition or compliance with laws and regulations, and weaknesses in model governance. It is important to consider risk from individual models and in the aggregate.

Inaccurate measurement of risk or relying on models that are not used as originally intended can result in poor decision-making. Without proper model risk management, model input errors, inaccurate assumptions, and untimely or missing validations can result in risk measurements that are inaccurate or misrepresented, and therefore board and management decisions that are based on inaccurate or irrelevant model outputs. More generally, inadequate governance over models’ development, implementation, use, and validation can increase risk. It is important for a bank’s decision-makers to understand a model’s limitations to avoid using a model in ways not originally intended or if the model has not been validated.

1. Strategic Risk

Strategic risk is the risk to current or projected financial condition and resilience arising from adverse business decisions, poor implementation of business decisions, or lack of responsiveness to changes in the banking industry and operating environment.

The board of directors and senior management are the key decision-makers that drive the strategic direction of the bank and establish a governance framework for using models. The absence of an appropriate governance framework for developing, implementing, using, and validating models poses strategic risk. A bank’s strategic risk can increase if models and associated risk management do not keep pace with strategic changes, the capability of employees, the operating environment, and regulatory requirements. For example, failure to adjust model inputs and assumptions for current and anticipated market conditions, the macroeconomic environment, and consumer behaviors could expose the bank to strategic risk, which may translate into financial losses.

2. Operational Risk

Operational risk is the risk to current or projected financial condition and resilience arising from inadequate or failed internal processes or systems, human errors or misconduct, or adverse external events.?

Operational risk is the primary risk associated with the use of models. Failed or inadequate processes and systems and errors or misconduct by personnel can significantly affect the predictive value of a model. Operational risk can result from fundamental errors in a model when viewed against the design objective and intended business uses without sufficient use of model overlays (A model overlay is a judgmental or qualitative adjustment to model inputs or outputs to compensate for model, data, or other known limitations. A model overlay is a type of override) ?and adjustments when model limitations become apparent. Personnel who do not have sufficient skills and training to develop, implement, use, and validate the bank’s models can increase operational risk. Modelling errors or omissions can occur in the application of theory, data inputs, algorithms, assumptions, shortcuts, simplifications, and approximations, which can lead to inaccurate outputs.

Management’s failure to engage in appropriate model risk management to prevent errors and improper use of models can increase operational risk. For example, operational risk can increase when algorithms are based on biased, insufficient, incomplete, or inaccurate information, or are not properly tested and validated. Models can fail because of inadequate internal controls, such as insufficient processes for controlling the quality of the data inputs. The absence of an appropriate change management process for new technologies, products, or service offerings related to models can also increase operational risk.

Operational risk can increase when the information technology (IT) environment supporting the bank’s models does not have appropriate internal controls. Security weaknesses, including poorly constructed application program interfaces (API) (API is software code that allows two or more programs to communicate with each other) ?and weaknesses in the controls for the access, transmission, and storage of sensitive customer information, could expose a bank to increased operational risk. Weak or lax controls can compromise the confidentiality or integrity of sensitive customer data.

Third-party risk management weaknesses related to a bank’s use of third parties providing models or related products and services could increase operational risk, particularly when management does not fully understand a third-party model’s capabilities, applicability, and limitations. New technologies, products, and services, such as AI and data aggregation, can increase third-party access to banks’ IT systems. When a bank allows third parties to connect to the bank’s models and systems and to access customer information, there can be substantial operational risk. Poorly drafted contracts could increase operational risk. Important considerations include the ability of the third party to resell, assign, or permit access to the bank’s data and IT systems to other entities and how the data will be transmitted, accessed, and used.

3. Reputation Risk

Reputation risk is the risk to current or projected financial condition and resilience arising from negative public opinion. Reputation risk may impair the bank’s competitiveness by affecting its ability to establish new relationships or services or continue servicing existing relationships.

?Inadequate policies and processes, operational breakdowns, or other weaknesses in any aspect of model risk management or governance can increase reputation risk. A bank could incur reputation risk from biased data outcomes, data losses, noncompliance with regulations, fraud, downtime, and insufficient consumer protections. Biased data outcomes can result in potential disparate treatment or disparate impact on borrowers on a prohibited basis. Third-party risk management weaknesses and wrongful acts by third parties could increase reputation risk. A sound corporate culture is the foundation of a sound governance framework and helps form a positive public perception of the bank.

4. Compliance Risk

Compliance risk is the risk to current or projected financial condition and resilience arising from violations of laws or regulations, or from nonconformance with prescribed practices, internal bank policies and procedures, or ethical standards.?

Compliance risk is elevated when banks do not comply with model-related laws and regulations. For example, risk-weighted asset regulations dictate requirements for certain banks’ capital measurement models. ?Compliance risk is also elevated when models result in potential discrimination on a prohibited basis or other violations of consumer protection-related laws and regulations.

A bank’s compliance risk can increase when models used in the bank’s BSA/AML and Office of Foreign Assets Control (OFAC) programs inaccurately reflect the risk of a bank’s business model, products, services, customer base, and geographic footprint. One example is setting and tuning thresholds in a BSA/AML or OFAC model without taking differences in risk levels across lines of business, products, services, customer types, and geographies into account.

A bank’s fair lending compliance risk could increase when a bank’s credit decisioning models include algorithms, variables, or other processes that result in disparate impact on credit applicants or customers based on prohibited factors, such as race, ethnicity, or sex. ?The source of the bias may be obscured by the model’s complexity if management does not properly understand and manage the model. Even when individual variables are not inherently biased on a standalone basis, the complex interactions typical of some models (e.g., models using AI approaches) could lead to unintended impacts or outcomes. Compliance risk can be elevated if management does not understand the requirements of consumer protection-related laws and regulations and the enhanced controls that should be implemented when using alternative data in models.

5. Credit Risk

Credit risk is the risk to current or projected financial condition and resilience arising from an obligor’s failure to meet the terms of any contract with the bank or otherwise perform as agreed. Credit risk exists any time bank funds are extended, committed, invested, or otherwise exposed through actual or implied contractual agreements, whether reflected on or off the balance sheet.

Banks use models to increase efficiency in all stages of lending, including marketing; underwriting; pricing; collateral valuation; risk ratings for obligors, counterparties, and transactions; stress testing individual loans; portfolio monitoring; and risk mitigation. If credit risk models do not incorporate underwriting changes in a timely manner, flawed and costly business decisions could occur. In addition, model error or ineffective model risk management can lead to credit decisions inconsistent with the bank’s policy or risk appetite and higher credit risk exposure than projected. Conversely, models that are well-designed and effectively managed can help management make prudent risk selection and monitor and manage credit risk.

Banks may rely on a model to estimate the ACL, or uncollectible amounts maintained through charges to a valuation reserve adjusted through a bank’s operating income. Models used to estimate ACL do not create credit risk. Rather, the ACL quantifies the credit risk inherent in the bank’s assets. A deficient model can mask credit risk in financial assets, hinder effective identification of higher-risk assets, delay the recognition of credit losses, and result in an inappropriate ACL balance.?

6. Liquidity Risk

Liquidity risk is the risk to current or projected financial condition and resilience arising from an inability to meet obligations when they come due.?

Liquidity risk can increase because of inaccurate or untimely inputs, assumptions, model adjustments, and outputs. Accurate information on the bank’s liquidity position is necessary to monitor liquidity risk. Inaccurate or unreasonable model assumptions related to funding access (e.g., deposit flows, wholesale funding availability, and timing) can increase liquidity risk. Management’s failure to adjust model inputs based on changes in market conditions can increase liquidity risk. For example, inaccurate pricing models may hinder a bank’s ability to liquidate assets quickly for a reasonable price. Examples of some common sources of liquidity risk in modelling are unsupported or unreasonable contingent funding assumptions; stress scenarios that do not consider all relevant legal or regulatory constraints; and inaccurate or unsupported behavioral assumptions (e.g., budgets, loan pipelines, rollover, and embedded optionality).?

7. Interest Rate Risk

Interest rate risk is the risk to earnings or capital arising from movements in interest rates. Interest rate risk arises from differences in the timing of rate changes and the timing of cash flows, from changing rate relationships among yield curves or across maturities, and interest-related embedded options in bank products.

A bank’s interest rate exposure depends on (1) the sensitivity of an instrument’s expected income/expense and economic value to a given change in market rates, and (2) the magnitude and direction of this change in market interest rates. Interest rate risk models depend on assumptions to accurately project cash flows from assets (e.g., prepayments, embedded options, and complex loan terms), liabilities (e.g., non-maturity deposit pricing, decay assumptions, and embedded options), and off-balance-sheet items.

Scenario design is highly dependent on reasonable assumptions for time horizon, rate structure, and magnitude of stress scenarios. Examples of some common interest rate modelling issues are (1) failing to assess potential exposures over a sufficiently wide range of interest rate movements to identify vulnerabilities and stress points; (2) failing to modify or vary assumptions for products with embedded options to reflect individual rate scenarios; (3) basing assumptions solely on past customer behaviour and performance without considering how the bank’s competitive market and customer base may change; and (4) failing to periodically assess the reasonableness and accuracy of assumptions.

8. Price Risk

Price risk is the risk to current or projected financial condition and resilience arising from changes in the value of either trading portfolios or other obligations that are entered into as part of distributing risk. These portfolios typically are subject to daily price movements and are accounted for primarily on a mark-to-market basis.

A bank incurs heightened price risk when trading instruments with prices that are hard to model. Examples include

• instruments that are illiquid or trade infrequently, because of limited data.
• newer instruments, because of limited data.
• instruments whose pricing model assumes a certain relationship between two variables (typically correlation), because that relationship can change.
• instruments with a fair value that depends on accurately modelling human behaviour (e.g., prepayment speeds and deposit betas), because human behaviour is often unpredictable.

Banks use models (primarily value-at-risk and similar expected shortfall models) to measure trading activities’ aggregate price risk. Value-at-risk and similar expected shortfall models determine how much capital a bank with significant trading activity should hold against the bank’s trading book. Management’s improper use of models to manage price risk could result in inadequate capital allocation relative to the size and nature of trading exposures. Compounding the problem, models may depend on other models, as one of the key inputs to a value-at-risk model is the fair value of the trading book, which can depend significantly on pricing models.

Reference

?Comptroller’s Handbook on Model Risk Management, pages 1-11.

?