How alternative data is taking centre stage in quantitative investing

Quant used to be a lot of mathematicians analyzing very small amounts of data, and traders making their best bets based largely on historical price data.

Today, quant investing would be practically unrecognisable to the majority of hedge funds of 10-15 years ago.

Acting on the desire to out-perform the reliability of indexed funds, firms in the last few years have invested heavily in alternative data to get the edge. Data has now taken center stage in the ecosystem, creating a fuller picture of the investment landscape with data sources unimaginable to the investors of last century.

The kinds of data some firms are finding applications for are creative, to say the least - satellite imagery, crop yield, shipping container movements, credit reports, sentiment analysis of a firm’s social media - The list goes on.

What does the trading team look like now?

With the need to manage and understand these extensive and varied data, the trading team has doubled in size. Data analysts trawl the web to find new sources of data (for example, public record, government agency, or educational). Many of those team members are also focused on data engineering: building systems and pipelines to pull, process and manage the data. The analysts then clean and normalise the data to ensure there’s no anomalies to skew the modelling, and then machine learning engineers create artificial intelligence programs to reveal the information hidden in the data.

So, far from mathematicians all using roughly the same information and models to predict many of the same market movements, these teams work to source data, build code and teach machine learning algorithms to infer minute patterns that a human couldn’t pick up. That means that different funds, depending what data they have access to, will come up with hugely varied insights and strategies.

The gap widens

The use of data and the new sources of it is becoming widespread across larger hedge funds in particular. This is because bigger firms can benefit from economies of scale when they buy in data sets, with many different teams working the data from different angles and reusing the same data multiple times.

As a result, we’re seeing very large quant firms who have many, many teams of software and data engineers and data analysts, building enormous systems for data collection, storage and retrieval to compute the huge volumes of information produced.

This acceleration is creating a capacity gap. Smaller firms don’t have the same scale of resource, and while they’re agile and unencumbered by legacy processes, they also don’t have the resources to build teams and teams of people focused purely on the data. With more data emerging every day and the landscape rapidly evolving, small firms must innovate so that they’re not left behind.

Are the computers taking over?

None of this is to say that individuals are being replaced by computers. The job of the traders who are using the data to inform their strategies remain largely unchanged, but with quant investing becoming so focused on data, the hunt for new sources that will return more alpha is the key to success in today’s market. As a result, funds are looking to hire from academia, and those with a bachelors or masters-level degree in software with specialisation in data are in high demand, as are those with PhDs in physics for alpha research roles.

If funds were to rely purely on fundamental sources (i.e. price data, earnings reports), they expose themselves to risk by running their analysis on information that everybody has, with strategies that are well documented, to put it mildly. In order to make the most of quant investing now, funds are focusing on innovative uses of alternative data sources. The seachange is clear as we see big firms hoovering up software engineers with academic backgrounds, who’ve worked in any form of data engineering capacity, focused around extracting, transforming and loading information.

One thing is clear: alternative data is not alternative any more. It’s hit the mainstream, and the message is clear - climb aboard or risk obscurity.

What changes have you seen data make to the quant ecosystem? Let us know in the comments below.