Know Your Benchmark

A recent TABB Group article provided a great introduction to the futures execution algorithm landscape, including the history of the space, varied requirements of different industry users, buy vs. build argument, and a general outlook on the most popular types of algorithms. A common thread throughout the piece is the importance of transaction cost analysis (TCA) and its usefulness in analyzing trading performance and aiding the trader/portfolio manager in tweaking algorithm behavior or unearthing a need for outright customization. Understanding execution costs is a multi-step process, and a proper understanding of TCA metrics provides useful insights that can be fed back into the trading process and further improve trading performance.

For example, in the graphic below, we illustrate the market’s price path along with a theoretical buy order being filled and identify different potential costs associated with an order that is not apparent when simply looking at average fill prices. The opportunity cost related to differences between signal generation and execution time, market impact costs related to demanding liquidity from the market, and the microstructure costs associated with the effectiveness of the trader are examples of more granular levels of cost breakdown that can help inform trading. Whether a trader is crossing the spread too quickly, demanding too much liquidity from the market, or waiting too long to execute orders are issues which may require different execution solutions.

TCA metrics can be broadly categorized as price-based or volume-based. Price-based metrics typically represent a measure of slippage from a defined benchmark; volume-based metrics tend to represent the ratio of an order’s volume against things such as the average daily volume and interval volume, or more granular aspects such as the passive fill rate.

Price-based Metrics

Price benchmarks set a reference point from which traders can evaluate their average fill price when analyzing their execution costs. Common price benchmarks include things such as arrival price, open price, close price, and volume-weighted average price (VWAP). Interpretation of metrics using these benchmarks differs based on whether a benchmark is pre-trade, post-trade, or intraday.

Pre-Trade Benchmarks

Pre-trade benchmarks, such as the previous closing price or the opening price, may be used to estimate a timing or opportunity cost. Price drift from the time a portfolio manager makes a trading decision, often based on prior day close prices, to the start of trading, represents alpha decay that an execution strategy is not able to capture. This is the first ingredient of overall implementation shortfall and can be measured as the difference in arrival price slippage vs prior close slippage. Significant opportunity costs indicate that time could be well spent trying to identify a trading signal sooner, or delivering the resulting order to the market quicker.

Arrival price, the price in the market at the time an order is entered, is a particularly important pre-trade benchmark[1]. Slippage metrics versus arrival price provide a robust metric for an algorithm’s performance, as it should encompass implicit and explicit costs – both the cost of demanding liquidity, alpha decay over the trading horizon, and total market impact.

Intraday Benchmarks

Various VWAP measurements represent what the weighted average price in the market was during different time intervals. ‘Day VWAP’ measures this average over the entire trading day; ‘interval VWAP’ over the lifetime of an order; and ‘end-of-day VWAP’ the average between the arrival time and the end of the day, which with a large sample size can be used to interpret whether a trader’s market timing is contributing positively or negatively to performance. Slippage from ‘interval VWAP’, the most commonly monitored performance benchmark, represent the cost of demanding liquidity and the quality of child order placement strategies. It is a less comprehensive cost measure than arrival price slippage, but tends to be less volatile between individual orders and provide a cleaner performance view to traders with long horizon strategies.

Analyzing slippage with regards to VWAP requires some caveats. VWAP slippage is not very comparable across instruments due to differences in volatility. Variance in VWAP slippage between individual orders will be driven mostly by whether the execution strategy is pursuing volume driven order placement (as is the case with VWAP and POV algorithms), and if so whether the algorithm’s volume forecasts line up well with the trading day’s actual volume. VWAP can also be heavily skewed by a large market participant. Traders, institutional or otherwise, who make up a large percentage of the volume relative to the VWAP metric they are using as a benchmark may effectively set the VWAP; in such cases, the market impact caused by their own trades reduce the information provided by VWAP in more normal circumstances.

Participation weighted price (PWP) is another intraday benchmark with multiple variants. In this case, the set participation rate is the variable; for example, a 10% PWP is the volume weighted average price that would correspond to the trader’s order making up 10% of the volume following the arrival of the order. Various PWP metrics can be analyzed alongside each other, provided the sample size is appropriate, when trying to determine whether trading horizon should be adjusted.

Post-Trade Benchmarks

Post-trade benchmarks include the close or settlement, next day open, and next day close. The difference in arrival price slippage and post trade slippage can also be used to help analyze the difference in temporary market impact and permanent market impact. Large slippage versus end of day metrics may lead a trader to conclude that they can reasonably trade more passively. Some market participants also focus on close price slippage because their investment objectives are tied to end of day pricing benchmarks, or they may be sensitive to end of day mark to market ramifications. Participants trading significant volumes during short periods near the market close may materially affect the final price, depending on settlement procedures. In these instances, like the problem posed by large percentage participants when comparing against a VWAP metric, a participant may end up effectively setting the settlement price; a trade may look like it achieved a price better than the settlement benchmark, but it may have incurred substantial market impact.

Volume-based Metrics

Market participants should monitor volume-based metrics alongside price-based metrics when analyzing trading performance and execution quality. Volume metrics help put slippage metrics in context; high participation rates correlate to higher arrival price slippage. Volume metrics also help participants analyze capacity constraints of their strategy and optimize the duration of their orders.

Participation rate metrics include: percentage of daily volume, percentage of average daily volume (ADV), and percentage of interval volume. Elevated figures in these metrics would generally indicate a higher cost of market impact during the respective trade; slippage versus arrival price tends to be higher than normal while slippage versus VWAP becomes muted (high participation rates cause the trade to contribute more heavily to the VWAP).

The passive fill rate may be used to determine how adept an algorithm is at avoiding crossing the spread when executing. High passive fill rates may indicate a level of success in avoid avoiding crossing the spread and minimizing the cost of demanding liquidity. However, a high passive fill rate by itself does not indicate high quality execution – passive fill rates can also indicate buying into a falling market, and can reflect adverse selection in market timing, as waiting for a passive fill can cause a trader to execute at a less desirable price as it trends away.

The relative performance measure (RPM) is another metric, particularly useful for comparing trades across products and time. Generally, RPM represents the percentage of trades executed at a worse price than the trader’s fill price during the execution. Whereas certain price metrics may not be readily comparable, RPM provides a simple, robust metric that a trader may use to compare executions. This can be extended to monitor consistency of trading performance, as this metric is less sensitive to volatility differences between products. Inconsistent RPM statistics could highlight potential trading issues warranting further investigation.

The Whole Picture

Transaction cost analysis is a process. No single metric should dominate the analysis, and users should educate themselves on what different combinations of metrics could mean. Traders shouldn’t miss the forest for the trees; ignoring how these different metrics interact could mean missing out on potential improvements to their overall trading process. Some questions you should be able to answer from a composite of different TCA reports include:

• How often is my trading signal correct, but I am not able to put on a position due to a limit price?
• What was the price activity during the life of my order?
• What do my percentage of volume statistics look like? Am I impacting the market too much?
• Is the slippage I’m experiencing acceptable with regards to my trading goals? Is it consistent with my back testing and simulation assumptions?
• Am I comparing metrics appropriately? Is there something obvious that I can work on improving?

RCM-X works with its execution clients to review their transaction cost analysis and make actionable recommendations based on the trade data. Contact Mike Aufmann at 312-870-1537 for more info on how RCM-X can assist with your execution and transaction cost analysis needs.