ESG Ratings: How Many Providers Are Enough?

The Sustainable Finance Disclosure Regulation mandates that asset managers consider sustainability risks in their investment process. One way to do this is purchase ESG ratings from a third-party provider and interpret them alongside financial data. However, studies show that ESG ratings vary wildly among providers. With no reason to believe that any provider is better or worse than others, academic studies suggest averaging (or otherwise combining) multiple ratings to get a "consensus result". However, this approach is impractical for small management companies due to the high costs and infeasibility of acquiring all available ratings.

In this post, I share an approach I used a few years ago to sort through uncertainty when I considered using ESG ratings for integrating sustainability risk.

A fast and frugal analysis

The table below is a correlation matrix of ESG ratings, taken from Capizzi et al. (2021). Each column-row pair is a correlation coefficient, which can be between -1 and 1. I abbreviated the company names to avoid appearing to endorsing one over another, but of course you can work out which is which by looking at the original paper.

The figure below illustrates how different correlation coefficients reflect relationships between variables, which could be helpful to review if you don't spend a lot of time with data.

Let's delve into some mildly technical details. By calculating the eigenvectors and eigenvalues of the correlation matrix, we obtain the results we would get by performing a principal components analysis on the full datasets.

Examining the first component, which accounts for the majority of variability, we see that the coefficients have similar magnitudes and signs. This suggests that averaging the six scores is a viable approach for creating a single ESG rating. This aligns with academic recommendations.

(It's actually a weighted average if you are feeling bit more ambitious, but this is a LinkedIn post so my ambition is rather low.)

To reduce the number of raters in the consensus, we can draw parallels with environmental science, where balancing the number of variables measured taken with cost is crucial. Scientists have developed various heuristics for this. One simplest approaches uses the following steps:

1. Decide the number of raters, n, you can afford
2. Start with the last component, and exclude the rater with the largest coefficient (absolute value), in this case SP.
3. Do this for the components in descending order until only n raters are left. These are the ones to buy.

If we can afford 1 rater, we should buy : MS

If we can afford 2 raters, we should buy: MS and IN

If we can afford 3 raters, we should buy: MS, IN, and TR

If we can afford 4 raters, we should buy: MS, IN, TR, and AR

If we can afford 5 raters, we should buy: MS, IN, TR, AR, and RE.

Ideally, you would take the consensus result of at least three raters. This comes from the observation that there are three important components to variability among raters, and as a practical matter we are associating one rater with each component.

Concluding Thoughts

Variability among ESG raters, complicates their use as objective decision-making tools for investors. This is particularly problematic for small investors that do not have the resources to evaluate or purchase multiple ratings.

There are numerous limitations to my analysis . However, there is a major advantage to this type of fast and frugal approach - the ability to learn and move on without being bogged down by the sunk costs of complex frameworks. Ultimately, I chose not to use any third-party ESG ratings when integrating sustainability risks. Given the evolving nature of ESG markets and our understanding of ESG issues, I believe it is wise to use fast and frugal models to remain adaptable and ready to revise when confronted with new information.

References

Berg F, K?lbel JF, Rigobon R (2022) Aggregate confusion: The divergence of ESG ratings.?Review of Finance?26: 1315-1344.

Capizzi V, Gioia E, Giudici G, Tenca F (2021) The divergence of ESG ratings: An analysis of Italian listed companies.?Journal of Financial Management, Markets and Institutions?9: art2150006.

Carpenter SR (2003) The need for fast-and-frugal models. Pages 455-460 in: Canham CD, Cole JJ, Lauenroth WK, eds. Models in Ecosystem Science. Princeton University Press.

Jolliffe IT (1972) Discarding variables in a principal component analysis. 1: Artificial data. Journal of the Royal Statistical Society C 21: 160-173.

Jolliffe IT (1973) Discarding variables in a principal component analysis. 2: Real data. Journal of the Royal Statistical Society C 22: 21-31.

King JR, Jackson DA (1999) Variable selection in large environmental data sets using principal components analysis. Environmetrics 10:67-77.

Lindenmayer DB, Likens GE (2018) Effective Ecological Monitoring, 2nd ed. CSIRO.