How can a Portfolio Manager assess the value of AI Signals?Part 1

Portfolio Managers are looking at AI signal generators to help them with their asset selection. But how can you assess the value that such a signal can give to your portfolio? In this 3 part series, I answer this question based on the work we at have done over the last 5 years implementing third-party signal generating algorithms in institutional portfolios.

When we do this, we look at the signal in three stages. The first is about the signal, the second is about how the signal interacts with the PM's own product and in the third we look at how we can best use that signal within the workflow of a practical portfolio management business.

1: The characteristics of the signal in isolation from anything else. What is its hit rate, persistance, p&l statistics when applied individually to the assets in its coverage.

2: How implementing that signal in a portfolio would change a portfolio and what effect it could have on returns. Assess the marginal value of the signal within the constraints of your portfolio definitions.

3: Create a workflow that implements the signal in conjunction with other alpha sources in a systematic way that

So lets address #1.

To start with, what does a signal set look like? Here we are looking at signals that generate trading ideas over a timescale of days and weeks rather than shorter term trading signals..

Typically the signal sets we look at can be presented as a grid of numbers. The dates and assets form the row and column headers, and the numbers in the grid are the value given by the signal for that asset on that date.

Some signals create a value, or indeed a set of values for each asset on each date, and some only generate a number when a signal threshold is breached, leaving other values blank. The example above is a signal that shows a +1 or -1 only when the AI thinks that a trade should be initiated.

An alternative may have daily values for different 'attributes'. This one has three attributes, 2 numbers and a letter for each asset on each day.

But no matter what, within a portfolio context, the signal implementation can only ever be a buy or sell of an asset/risk position: somehow you have to reduce whatever information is in the signal to a binary flag (or do nothing)!

Most signals we have worked with are meant to initiate a trade, i.e. they identify something from their inputs (which can be a huge data set of natural language text, video, internet scrapings etc.) which will lead to a change in perceived value of an asset as other market participants catch on. The signal gives you a time advantage.

But that says nothing about either the persistance of the signal, the target change of price or the timescale over which that change should happen.

So the first thing we do is ASI: Assess the Signal in Isolation.

That is look at the universe of assets that the signal covers, and try to understand what predictive power it has statistically, over what timeframes and with what expectations of return.

Assuming this is positive, we can then assess whether the signal has potential to help a given portfolio manager. Of course it must cover the right assets, but does it operate with a frequency that matches the PM's portfolio, and does its 'alpha' have the right holding period? Is it adding risk where a PM would want it (say at factor level rather than idiosyncratic, or vice-versa), and does it have any reason to persist?

This last question is very tricky with black-box signals: the AI says "buy" or "sell", and all you can do is look at the evolution of market prices post-signal and assess if there is any statistical value.

But if you test three years worth of signal, it is inevitable that the inputs will have changed over that time. Imagine it's based off the text that it scrapes from a news source, that source will not have had the same writers/editors and its own sources over that time.

The alternative to these AI black boxes are 'expert-system' signals: which rely on better defined specific data that you can say is consistent over time (say short-interest or borrow-costs) but that is harder to come by and requires some domain expertise (i.e. market rather than computational AI) to manage the algorithm. Here you can have more confidence that the driver of the signal persists, though of course changes to market structure can alter returns.

But if we can get over these hurdles: find a signal that works, that fits the portfolio and has a reason to persist, the next stage is to value it. How much is it worth to a given PM.

In the next ariticle, we will look at how to do this.