Why PRIIPs got derailed over past performance versus simulations

In September 2016 the European Parliament (EP) dramatically rejected the technical standards of the Key Investor document (KID) laid down by the European Commission (EC). This was the cornerstone of the PRIIPs regulation and it happened with barely three months before an implementation date of 1 January 2017 that had planned for over two years.

There were many causes for this to happen following on from the significant opposition that the PRIIPs RTS proposals has met from many in the industry stretching back at least twelve months. Insurance companies felt disadvantaged and overburdened by extremely complicated rules for calculating costs and risks, while those in the fund management world found the total abandonment of the use of past performance in favour of simulations to be equally unpalatable. The structured product sector has also suffered in that the required calculations can prove onerous and complex and with potentially surprising or unreasonable results.

In order to see what has gone wrong it is important to analyse the roots of this process.

In the European investment market there has long been an emphasis on properly explaining investments to the retail and institutional investor. Examples of this include the current UCITS requirements, the rules on investor documentation that exist in Germany and UK FCA guidance to ensure that information provided to investors is “fair, clear and not misleading”. While all regions have strong regulation today the approach elsewhere can be very different. For example the US market has an emphasis on disclosure rather than genuine explanation. There it is common to see long unwieldy prospectuses with many disclaimers on risks and product features. However this is generally presented in a way that benefits the lawyers that compile them rather than the investors who are supposed to read them.

This European heritage of accessible and meaningful documentation gave rise to the PRIIPs concept which was supposed to extend and improve investor facing literature. However it is important to acknowledge the influence of the EU itself which is obsessed with an approach in everything it touches based on standardisation and detailed prescription and regulation. Whether this is broadly a good thing or not is a matter for debate but it is undeniable that the EU is set up in this way. The pros and cons of how the EU works were passionately debated in the UK during campaigning for the “Brexit” referendum.

The end result on PRIIPs was a regulation which requires generating prescriptive standardised documentation for the vast majority of retail investments across the entire EU. It is an extremely ambitious concept to assess funds, insurance products and structured products under one common regime (the desire for standardisation) and in such a detailed fashion (the desire for prescription).

It is in my view the scope and the detail that caused the fault lines which were so dramatically exposed last month.

The numerical sections of the KID are intended to provide investors with information that tell them about the investment they are considering in order to understand it and provide comparisons with alternatives. The calculations that must be shown are those of risk, costs and performance scenarios. While there has been debate over how PRIIPs should assess risk and costs it is self-evident that a risk calculation should capture all the possible behaviour of the investment’s return and that the cost measure is to be based on a sum of all relevant cost elements. We would expect any robust methodology to produce reasonable results for both.

One of the main reasons for the rejection of the RTS was over the basis of calculating and presenting performance scenarios, this was the central objection of the large and powerful fund management lobby. It is the issue of how to show performance scenarios which I believe were not sufficiently clearly thought through, with ultimately disastrous consequences.

Investors should be able to understand cost and risk numbers for what they are – respectively the objective measure of what is deducted over the life of an investment and a single encapsulation of how much money could be lost.

To give an idea of how a product might perform, the mutual fund industry has presented investors with past performance figures for many years. Typically it shows how a fund has performed over a 1, 3 and 5 year timeframe and it may also include a comparison to a recognised benchmark or other funds in the same sector.

The advantages of such an approach are that performance data is easily to calculate given sufficient track record and the results are easy to verify. The choice of an appropriate benchmark or fund sector is occasionally difficult to determine but by and large it has worked well over the years.

There are two major objections with using past performance for the PRIIPs performance scenarios.

The first is that the results are heavily dependent on market conditions in the previous five years. However if the statistics are presented as past performance figures with suitable disclaimers then they should be easy to intepret. The use of a benchmark or sector average will give context about overall market performance.

Additionally we should anticipate that diligent investors (or their advisers) would refer to KIDs across different products when making an investment decision. This means that the playing field is relatively level at any point in time because all investments are being compared over the same past horizon.

The second shortcoming of using historical past performance is that for investment types other than funds it is generally more difficult to define and calculate such performance because structured products and some insurance products are offered in tranches on a regular basis, the terms of which vary according to market conditions.

Therefore having concluded, with some justification, that reporting simple past performance would not be adequate, The European Commission set about defining a complicated though ingenious simulation basis (involving "resampling" of historical data to create a simulation framework). This was originally formulated to be used for the risk calculation. 

The major problems occurred when this approach was extended to generate performance scenarios, intended to show a favourable, moderate and unfavourable scenario (defined as the 90th, 50th and 10th percentiles respectively). This idea was only made public in the intended final draft RTS made public in April 2016, less than nine months before implementation date.

In my view it is very difficult to really get across to investors the status of these figures. Everyone with any exposure to investments is used to seeing past performance figures with the standard caveat that the “past is no guide to the future”. However the type of projection that had been proposed in the PRIIPs KIDS will inevitably be seen as some kind of forecast, or in the case of the favourable and unfavourable scenarios, virtual best and worst cases.

I do not think that this point was appreciated enough by the EC.

Both the proposals for MiFiD II and previous FCA guidance have striven for a balance between past performance and simulation and a clear distinction between the two.

Analysis and conclusions based on forward looking simulations definitely have a key role in risk management and assessment of investments. However, the first question that gets asked when results from a simulation are put forward is what are the assumptions behind it. There are many ways to set up simulations, but care must be taken to ensure that the results will be reasonable for the purpose that they are intended and that consistency is achieved over the range of investments that are being considered and over time as market conditions change.

In the PRIIPS context the resampling technique was supposed to be an ideal solution comprising both historical data and a simulation framework. It also does not rely on estimating parameters or requiring data that is hard to source, and contained little need for subjectivity or interpretation meaning that any two parties should assess the same investment in the same way.

However the reason that the mutual fund industry objected to these calculations was because of the technical nature of the simulation the result depends purely on the historical volatility of the fund and the current risk free interest rate. Thus all funds in a given volatility range will look the same at a given point in time irrespective of previous performance, such as which quartile the fund may have placed in the past. This represents a flattening of the merits of individual funds that the mutual fund industry was not prepared to accept.

The type of simulations represented by the resampling technique has suffered a blow to its credibility after the events of the last month, it remains to be seen what gets finally agreed.

It is obvious that there can at times be a wide variation between historical performance and future simulations. In particular for the case where historical performance looks worse than the simulation it may be dangerous to show the simulation because it does not highlight risks enough.

In conclusion I think that the differences represented by past performance and simulated results had not been fully appreciated by the EC. To suddenly require the display of future simulations projections in an investment world used to showing past performance was by regulatory standards rushed through and urgently needs a major re-think in order to make sure that investors receive accurate and meaningful information that they can safely interpret.

A version of this article appeared in Investment Week