What are the most important considerations for stress testing in emerging markets?

A key issue with the way stress testing is done now is that usually one or two scenarios are chosen by a country's regulators and there is no indication of how good or bad these scenarios are. What is needed is a data driven approach that is algorithmic whose properties are known. We have developed such an approach and it can be powered a combination of models and the judgement of experts. It guarantees that the best and worst case scenarios are amongst the set generated and allows for what's best for one bank may be worse for another. Expert judgement is a good fallback when data is sparse so that is less of a problem. Scenario analysis requires many scenarios to get a picture of risk. Algorithmic methods are comparable and consistent.

Data quality and availability are indeed critical factors when stress testing in emerging markets. Dealing with Unreliable Data: When faced with unreliable or incomplete data, adopting robust statistical methods becomes essential. Techniques like imputation or sensitivity analysis can help manage these gaps. However, it's important to acknowledge the limitations these methods bring to the accuracy of stress test results. Assumptions and Limitations: Documenting assumptions used to fill data gaps is vital. This transparency is crucial for interpreting the results correctly. Clearly stating these assumptions adds credibility to the stress testing process and helps in understanding the potential margin of error in the conclusions.

Data quality and availability, is the cornerstone of stress testing. Without high-quality data, any analysis is built on shaky ground. This is especially pertinent in emerging markets, where data can be scarce or unreliable, and without good data, it's impossible to accurately assess risk. Also, Scenario design and calibration relies on good data to create realistic scenarios that stress test the financial system. However, the scenarios are hypothetical if the underlying data isn't accurate. Methodology and assumptions are only as strong as the data they're based on. Methodologies must be developed to handle the types of data available in emerging markets, which can be challenging if the data is incomplete or unreliable.

Scenario design and calibration in stress testing, especially in emerging markets, are pivotal for accurate risk assessment. Some aspects to be considered are: Tailoring Scenarios to Market Specifics: Emerging markets have unique characteristics, like fluctuating exchange rates, varying capital flow dynamics, and distinctive sovereign risks. Scenarios must reflect these specific elements accurately. For example, a stress test in a market with a volatile currency should include scenarios that consider extreme currency fluctuations. Consistency and Coherence: It's important that different scenarios are consistent with each other. This ensures that the outcomes of stress tests are comparable and effective for risk management decisions.

In addition to historical episodes, scenarios should be primarily based on forward looking data. The best place to find such data is with the actors in the market who have experience. this can be done with expert judgement. Of course these actors would be experts in the particular markets under study - IE traders of the market instruments, market experts, policy makers and so on. To achieve consistency you need algorithmic methods that can be applied consistently and are data driven - the forward data on risk factors estimated by the wisdom of experts and aggregated into forward-looking distributions that capture future risk. Scenarios should be focused on extremes, not business as usual.

I agree that the methodologies and assumptions should be appropriate for the purpose of the exercise, which is to understand the market factors that cause an institution distress and to understand the bets the institution is taking. Policy should be part of scenario risk factors since their effects are material. However, having 6 wise people sitting around a table generation ideas for a scenario is neither desirable nor scaleable nor consistent. Data driven algorithmic methods that are explainable are more desirable.

It is clear that communication and disclosure of the stress testing results should be clear, timely and comprehensive. However what is usually lacking is the interpretation of the results - what do they really mean? What conclusions can be drawn from them? How do they inform strategy? Unfortunately the way organized stress testing has been executed has become a tick-the-box, painful exercise and not one that is useful for guiding strategy. Stress testing is essential but how to interpret the results is not clear. Using a single stress scenario or even a few can lead to gaming the system and not necessarily to the goal of finding the weaknesses of institutions.