Backtesting in the clouds

Imagine a scenario where you want to facilitate backtesting for multiple strategies, written by various developers, some of whom may use different programming languages or environments. This presents a few challenges, primarily around maintaining consistency, version control, and the ease of use of the backtest process for everyone involved. Let's break down concerns and potential solutions:

• Problem: Different developers may prefer or be limited to specific programming languages (e.g., Python, R, C++, Julia, etc.). You can't expect everyone to use Python, and if you were to build a Python-only solution, it would alienate those outside of the ecosystem.

We follow and API-first design and different languages will communicate with the backtest engine using a gRPC messaging protocol. The tool we are going to introduce is going to make it easy and fast to create such APIs. The central backend could be written in Python and clients (in Python, R, Java, etc.) communicate with it by sending input data and receiving results.

• Problem: Even if everyone uses Python, there's still the issue of dependency management, version control, and compatibility. Different users might install different versions of your package or its dependencies, leading to inconsistent results.

The management of versions has been simplified a great deal with dependabot and renovate but both ideas still require a level of discipline and computational hygiene that is often not met in practice. Needless to say that your backtesting engine may come with various other dependencies that are a potential source for conflict.

Our central backend operates on a server within a container. It’s completely isolated from the code of your strategy. If you still refuse containers let me remind you that it is 2025 :-)

• Problem: Since backtests will be run by different individuals at different times, tracking this information (who ran it, when it was run, etc.) becomes important for auditing and ensuring reproducibility.

The backtest service would come with an integrated logging and versioning system. Every time a backtest is initiated, one can log all relevant metadata (time, version, who). The data is stored in a centralized database, which can be queried or visualized. A simple interface (maybe a dashboard) could be created for users to track and analyze backtest histories, metrics, and changes. This could help to support backtest counting following an idea by Marcos Lopez de Prado .

• Problem: Creating a modern server based on Apache Flight is time consuming and challenging.

To create a basic server for your own experiments I have created numpy-flight:

https://pypi.org/project/numpy-flight/

To support the rapid creation of such servers I have also created a [qcradle](https://pypi.org/project/qCradle/) template, e.g.

uvx qcradle https://github.com/tschm/server        

There follow the instructions to create a server which comes complete with required dependencies, a Dockerfile, some tests and basic documentation. Once you have the server in a container it’s trivial to fire it up with any of the established cloud providers.

Conclusion

The idea of using centralized services is not new but it is of elevated relevance in situations where results need to be reproduced, metadata has to be logged and several languages have to be supported.

The approach sketched here addresses the core challenges of backtesting in a multi-language environment:

• gRPC bridges the gap between different programming languages.
• Containerization ensures consistent environments and eliminates dependency conflicts.
• Logging and versioning systems ensure that backtest results are auditable and reproducible.
• Tools like numpy-flight and qcradle make it easier for developers to set up and experiment with backtesting environments