Building Random Forest Scorecards

In the lending industry and credit risk research, a risk practitioner can often encounter Weight-of-Evidence logistic regressions (WOE LR) or various versions of gradient boosted trees (GBDT) used for standardized credit risk assessments. However, implementations of Random Forests for Probability of Default (PD) scoring are not as common, which can be partially attributed to the algorithm's inherently random nature, which may not be well-suited for predictive modeling in high-risk domains, especially perceived as such by regulatory authorities.

To the best of the author's knowledge, there have been no attempts to incorporate Random Forests into scorecard methodologies. To this end, this post aims to bridge this gap by demonstrating how Random Forest scorecards can be developed and highlighting several potential benefits they offer for the risk model development and validation process.

Random forest scorecards

Famous gradient boosting algorithms such as XGBoost or LightGBM include several types of base learners, including linear trees and random forests. In case of linear trees as base estimators, the idea is to estimate a linear regression within each leaf of a decision tree rather than use a constant value in a leaf. Although using this feature can boost performance in some scenarios, it can also break monotonicity of the risk score regardless of monotonic constraints and is much more difficult to interpret.

In contrast, the utilization of random forests as base learners remains relatively unexplored, with even official XGBoost documentation providing limited insights. According to the documentation, a parameter named num_parallel_tree allows the use of a combination of trees (a forest) instead of a single decision tree. Notably, when both num_parallel_tree and num_boost_round (the number of iterations) exceed 1, training incorporates a blend of random forests and gradient boosting.

This interesting combination allows to construct random scoring sub-models that are incorporated into a larger model. To further enhance interpretability and readability of the model outputs, these sub-models and their components can be converted into a scorecard format. This approach is particularly useful when crafting 2-level models, where each tree can perform at most two splits (max_depth=2).

Below is an example featuring four trees forming the first two random forests within the scorecard. In XGBoost's implementation, each unique tree produces log-odds scores, which are subsequently aggregated into a forest by means of summation of margins (please see the technical appendix for more details). We then transform these log-odds scores into a traditional scorecard that assigns higher scores for good risk and lower scores for bad risk accordingly.

Practical applications of Random Forest scorecards

One can think of several practical applications of this modeling technique, including:

• Feature engineering: By applying random subsampling, each tree within a Random Forest model is trained only on subsets of the original data. This capability of decorrelating trees is invaluable for feature engineering leading to more robust and predictive risk models.
• Segmentation: Random Forest scorecards can uncover hidden patterns within the underwriting data. This data-driven approach empowers risk practitioners to fine-tune credit policies and discover unique customer risk segments thus optimizing risk management strategies.
• Model validation: When conducting model validation exercises and other model risk management tasks, Random Forest scorecards offer a versatile toolkit. They facilitate the creation of various model challengers, making it easier to identify opportunities for model improvement and robustness enhancement.

In the example below, we illustrate how such a scoring model can be employed for risk management purposes. Notably, Tree 4, 5, and 8 stand out with individual Gini scores exceeding 50%.

By aggregating the scores from these influential components, we can achieve a Gini score of 70%. This composite score retains 93% of the rank-ordering power of the final model, as indicated by the last bar. Arguably, this approach is much faster and more efficient than a traditional analysis of univariate correlations and exclusion of features with lower rank-ordering power from the combination.

Concluding remarks

Random Forests are not common candidates for credit risk models due to their intrinsic randomness, which impedes interpretability, a critical requirement in high-risk predictive modeling domains such as credit scoring. A non-conventional approach to scorecard building based on Random Forests can enable risk practitioners to explore and discover valuable customer risk segments and craft predictive features for risk management models further optimizing risk strategies.

--

I hope you have enjoyed reading this post!???

The technical appendix with the code can be found in this notebook.

All views expressed are my own.