Probabilities, stale quotes and arbitrage

Introduction

In August of last year the SEC approved a new "displayed" order type called D-Limit, proposed by IEX. The order type was available to trade as of 01 October. You can read more about the order type in IEX's announcement here and a basic description of how the order type (and others offered by IEX) works here. There have been some outspoken views about the intent of the order type, most recently expressed by Nasdaq. IEX countered with a post of their own. For some insightful reading by a participant backing up the innovations offered by IEX you can refer to the amicus brief submitted by XTX Markets to the US Court of Appeals for the District of Columbia Circuit.

The purpose of this post is to take a closer look at the signal the order type is based upon (and other non-displayed order types "D-Peg" and "Primary Peg") rather than join the argument about the prevalence of latency arbitrage or the merits of the order type and what it means for the broader US market and the relevant stakeholders. The IEX should be commended for the research they have published related to the indicator, and i believe the documented process and evolution of the indicator can give readers some useful insights into what a robust and scientific process looks like.

The Crumbling Quote Indicator forms the backbone of the IEX Signal used to determine when a crumbling quote is likely to result in a downward or upward price tick, which would otherwise be considered adverse selection for a resting order were the order to trade at the soon-to-be worse price. The timescales of the events being considered are orders of magnitude less than the time it takes the average person to blink. Think about that for a split second...

For a comprehensive review of the signal, its origins, early development and refinement (as at 2017) it is worth reviewing "The Evolution of the Crumbling Quote Signal" which largely motivated this post together with my own experience.

To understand the solution it is worth first understanding the problem.

Fragmentation

Many countries around the world have multiple venues for executing equity transactions (ie. buying or selling shares in listed companies). As at 2018 the US had the highest number of lit venues at 12 (currently 15), with Canada a close second at 9 (currently 10, although not all listed securities can be traded on each venue). The below table is a subset taken from a table in the paper published by AQR titled Trading Costs which looked at various costs of execution (explicit and implicit) in 20 developed markets.

Having such a large presence of lit venues is often referred to as fragmentation, however in a capitalist system you would expect the number of venues to be in a continuous equilibrium where supply and demand perfectly offset. As markets, regulations and technologies evolve so will the number of venues likely evolve together with the services they offer (for example new order types). For a most thorough explanation of "the problem" the D-Limit (and previously approved non-displayed order types) seeks to solve we will look to Allison Bishop's description from the whitepaper:

"Suppose that trader A has rested a midpoint-pegged order to buy a particular stock on venue V. The purpose of such an order is to float with the midpoint of the NBBO across all of the exchanges, but even this basic statement is a bit misleading. There is no such thing as “the NBBO” in a technical sense. The order is actually pegged to the midpoint of the view of the NBBO as observed by trading venue V. Suppose at a given point in time, the market has been stable for a while (meaning the best bid and offer prices have not changed for several milliseconds), and trading venue V has an accurate, up-to-date view of the NBBO in that symbol as $10.00 by $10.02. So the resting buy order is pegged to the midpoint price of $10.01. Now suppose that the best bid of $10.00 is not available on trading venue V, but instead is only available on trading venue W. Now, a seller comes along and trades with all of the buy interest at $10.00, changing the best available bid to $9.99. The midpoint of the NBBO is now $10.005, but this information does not arrive at trading venue V instantaneously. There is a small window of time in which venue V still believes the midpoint is $10.01, so if a matching sell order arrives at venue V during this window, it can trade with the resting buy order at $10.01. This is bad news for the initiator of the resting order, because the NBBO has already changed in their favor, and this execution at $10.01 goes against the spirit of what a midpoint pegged order is intended to accomplish."

Now its possible the above scenario occurs as a result of a single order looking to sweep multiple price levels in one fell swoop. But it is also possible that the scenario occurs where "an opportunistic trader observed the NBBO change before venue V and managed to submit a sell order to venue V that arrived ahead of the new price information." This is referred to as Latency Arbitrage (or stale quote arbitrage) and IEX believes that it is an undesirable practice in the markets and that it is worth countering. Of course their 350 microsecond speed bump was meant to disadvantage similar higher frequency trading tactics, with 350 microseconds chosen since it took roughly 300 microseconds for IEX to observe a price change on another market, with a 50 microsecond buffer. However, IEX observed an increase in adverse selection where resting orders were trading right before a change in the NBBO. This led IEX to conclude that the speed bump was sufficient for preventing observed stale quote arbitrage but not for anticipated stale quote arbitrage. If sophisticated higher frequency participants were able to anticipate upward or downward price ticks more than 350 microseconds before they occurred, they would be able to "pick-off" stale quotes with some probability.

At this point, IEX determined that instead of increasing the length of the speed bump they would "fight math with math" and so the idea for the Crumbling Quote Indicator was born.

CQI

Being an exchange IEX are heavily regulated. As such, proposed new order types, changes to the CQI coefficients etc are publicly disclosed with significant detail. For an example you can review the originally approved D-Limit document from the SEC in August 2020 or the request to incrementally optimize the indicator in July 2018. These disclosure requirements presumably put IEX at a significant disadvantage to the activities of the opportunistic participants they are trying to foil as these participants can deploy and update models intraday if they so wish, with nothing more than internal approvals. Nevertheless, the extent to which IEX have disclosed the evolution of the CQI is something i appreciate and i believe carries useful insights for others looking to incorporate more scientific and stringent processes into their research and development workflows. Without going into the details about their specific models which can be reviewed in their whitepaper referenced at the beginning of this article, i hope to summarize some of the main points.

Research & Development Workflow

Firstly, IEX had to consider the likelihood that "fighting math with math" would result in a statistical modelling arms race which would be undesirable and potentially not the best use of their time as an exchange (although statistical edges are constantly changing and being refined by participants generally). This was unlikely to occur for 2 reasons. 1 - IEX has a structural 350 microsecond advantage over these opportunistic traders, which should be sufficient in most cases. 2 - An incorrect prediction of an up or down tick by an opportunistic participant is more costly than the opportunity cost to IEX of delaying an order (in the case of D-Peg) or re-pricing an order in the case of D-Limit (especially considering the signal is only on for an average 2 seconds per security per day; again highlighting the timescales at which they are operating, 2 seconds equates to 2 million microseconds).

Next IEX considered a few initial features for predicting an upward or a downward price tick, after filtering for securities with tight spreads and trading above $1. These initial features were the difference in the number of venues at the NBB and NBO at the current time compared with 5 milliseconds earlier. Their initial time horizon to observe for whether or not a tick was observed subsequent to the firing of the signal was 10 milliseconds. To begin with, IEX simply modeled the number of events on a single day, using a brute force approach, where their signal fired and looked at the number of True Positives versus the number of False Positives. As a first pass, the results were quite satisfactory. Across 8,357 symbols for a single day they were able to correctly predict 570,000 crumbling quotes with 860,000 false positive predictions. Those false predictions only equated to roughly 1sec per security per day, where the signal would have been on for an incorrect prediction. This was the model deployed to production for the D-Peg order type from November 2014 to September 2015.

Next IEX tried Logistic Regression using those same features and found the performance more or less in line with the brute force approach. They ended up settling on the Logistic Regression model for several reasons, all of which are worth repeating and i will quote directly from the whitepaper:

"Even though it does not seem to be a big loss in this case, we might still ask: why consider imposing a linear function in the first place? If we can get reasonably reliable estimates of the probabilities of ticks following various feature combinations, why not just use these directly and bypass logistic regression? For one thing, a table of 20,736 probabilities is an inconvenient object to include in a rule filing. But more importantly, it does not give us much human-digestible insight into the structure we are modeling. Some might not see this as a problem: if the model works well, do we really care about understanding why and how it works? Well, we will probably care if it stops working as well. If we cannot succinctly describe and explain our model, it is going to be hard to fix it and improve it over time. It is a common sense rule of good statistical practice: never use an overly complex model when a simpler one will do. Simplicity is a form of protection from over-fitting to training data and overreaction to minor deviations in live data on a day-to-day basis. We should not give it up in exchange for meager returns."

I am not sure who needs to read the above paragraph, but i am sure a future me will appreciate referring to it. Throughout the evolution of the CQI, IEX compared their accuracy using the brute force approach with the logistic regression approach which was first deployed to production in Sep 2015 to August 2016 before receiving 2 more refinements before the paper was published as IEX were hoping to gain approval for the Primary Peg Order Type.

Conclusion

To summarize the main points for R&D workflow purposes, using the example set by IEX and their evolution of the Crumbling Quote Inidicator:

1. Know your problem, and design a solution to fit the problem.
2. Have good and readily available data.
3. Use the same train and test set to compare models.
4. Start with simple models.
5. Make the models simpler if it means negligible deterioration in performance but easier maintenance and explainability.
6. Continue refining features and model specifications as new information becomes available.
7. Lastly, it helps to believe in the greater purpose of what your model is aiming to achieve. In the case of IEX, they believe the CQI contributes to their mission of protecting investors which is a pretty powerful mission.