High Performance Financial Technology: Enemy of State?

"... the fastest calculation is always the one you don’t have to do. ... the fastest message is the one you don’t have to send.", Ariel Silahian

Financial transactions are stateful. A quote for a security leads to an order which may result in an execution that has to be confirmed and may require clearing and regulatory reporting. Each of these phases have internal states. Messages constituting financial transactions need to be processed in the context of these transaction states. Financial systems are, thus, inherently stateful. These systems are composed of services which are themselves stateful or stateless. This raises interesting questions about their performance and implementation strategies especially in the context of emerging serverless technology from various cloud offerings.

Stateful vs Stateless

Stateful services cache transaction states which are also written to the database. Caching reduces round trips to persistence thus reducing response time to requests. But this incurs higher memory footprint, longer startup time to prime caches and increased cache management complexity. Response times can be further reduced with asynchronous writes to the database. Higher cache thrashing can result in higher garbage collection and more frequent persistence access which lowers overall performance.

Stateless services, conversely, maintain no state internally. Incoming messages trigger database reads to determine the state in context of which these messages are processed. Corresponding state changes are written back to the database but not stored in the services. Hence, stateless services incur longer response times due to frequent access to the database but have lower complexity, memory footprint and startup times.

Stateless services are generally considered scalable. Theoretically, 2 instances of a stateless service double the throughput. But, if both instances rely on the same database instance, throughput increase is dependent on locking mechanisms to handle contention in the database. Furthermore, financial transactions may consist of multiple messages that have to be processed in order. This requires messages within a single transaction to be routed to the same instance of scaled services leading to uneven loading of these instances and fluctuating throughput. Interestingly, it may be argued that stateful services scaled with sticky load balancing will be more responsive and equally scalable as compared to corresponding stateless services.

Approaching Statelessness

A common approach to achieve statelessness is to externalise server state to out-of-process distributed shared-nothing cache. Services write updates to the cache transactionally and asynchronously to the database. Cache is used as the primary data source with database used to prime the cache during cold starts and for ad hoc analytics but not for real time operations.

While database contention and latency can be avoided with this approach, services will encounter messaging delays and cache latencies. Depending on the caching technology and content organisation (e.g., Distributed Hash Tables - DHT) , contention can be reduced significantly thus enhancing scalability. Message affinity, however, may continue to skew load profiles on individual services.

Cache performance is also affected by the amount of data they contain. Optimum performance is achieved by caching data that is most relevant to the current operation of system. For example, in case of order matching, keeping the cache populated with expired or executed orders will reduce the performance of matching open orders, Most systems have more than one cache each dedicated to specific operation the system supports. Finally, limiting domain specific operations in cache implementations reduces their complexity and can improve their performance. Ideally, caches should be treated as domain agnostic data stores.

Serverless - Scaling in the Cloud

Serverless architectures consist of server side components that are hosted on stateless compute containers that are event triggered, ephemeral and managed by the hosting provider. Implementations of this paradigm, also referred to as Function as a Service (FaaS), are offered by all major cloud vendors with AWS Lambda being the most popular. Serverless architectures rely on automatic and elastic scalability provided by the platform to achieve consistent performance at varying loads. As instances are only created when needed, serverless architectures offer cost saving opportunities.

Almost all FaaS implementations impose limitation on request execution time, code dependencies, memory footprint and instance lifetimes. Because FaaS instances are ephemeral, they have to be stateless. If they need state for execution, that has to either be externalised or embedded in the requests. The latter increases message sizes leading to communication latencies. FaaS instances may also suffer from start-up latencies which can be reduced through careful implementation and reuse of existing instances. Finally, it is challenging, if not impossible, to implement message affinity with FaaS, which may only be emulated with considerably complex implementations.

Financial Technology: Stateful or Stateless?

While most financial systems are stateful, they can have components and subsystems that are stateless and hence optimally scalable. An example is of pricers which are used to price products and to calculate P&L (profit and loss) of the inventory. Trades can be priced independently and concurrently. Certain pricing algorithms are decomposable thus leveraging scalability for performance. The challenge, therefore, is to understand scalability constraints state imposes on processing and operations and to design services and subsystems for optimal performance.