High Performance Financial Systems: Getting the Message

“... the Internet is not something that you just dump something on, it's not a big truck. It's a series of tubes. ... those tubes can be filled, ... when you put your message in, it gets in line it's going to be delayed by anyone that puts into that tube enormous amounts of material..."

This 2006 quote by a controversial US senator Ted Stevens rather ineloquently and superficially but still adequately describes causes of latency and congestion in computer networks.

Text-based Messaging Clogging Financial Systems

Interestingly, this also reflects on a fundamental cause of performance limitations in financial technology. Most messaging formats for both front and back office financial systems are text-based. Two messaging formats dominate the financial industry, FIX (Financial Information Exchange) and XML. Example of latter are FpML (Financial Products Markup Language) and ISO-20022.

The primary advantage of text based protocols is human readability. Messages can be sniffed off the network or read off message queues for debugging. These can be viewed without extensive tooling thus providing efficiency and economy in support.

But this is at the cost of computational and communication efficiency as this leads to verbose messages especially in XML based protocols resulting in higher message latencies. Thus text messages take longer to serialise, deserialise and propagate through networks. For example, date and time in text are usually represented as 2018-09-18T15:32:54Z which is 20 bytes long while in binary it is a long integer which is 8 bytes long, an overhead of over 100%.

Additionally, processing these messages within services requires validating text and parsing values into native types like integers, floating point numbers, dates and timestamps. These conversions have considerable processing overheads leading to performance limitations. Conversely, binary formats being native to the internal representation of data types within software can be mapped directly between the data structures and network streams with little additional overhead.

“The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information.”, Alan J. Perlis

Finally, Industry Gets the Message

With high frequency trading (HFT) and volume processing in exchanges, trading venues and service providers, inefficiencies of text based protocols become all too obvious to be ignored. As a result, highly efficient and specialised binary protocols such as ITCH, OUCH, SoupBinTCP and RashPORT emerged. Operational advantages of these and other binary protocols ignited a debate on the fitness of FIX for HFT where even nanoseconds can make a difference.   

FIX Trading Community adapted the FIX protocol in 2005 for low latency and high volume requirements. FAST (FIX Adapted for STreaming) compresses and encodes FIX messages to reduce latency and throughput. And it has been argued that optimised implementation of FAST can outperform proprietary binary protocols.

Additionally, FIX has been adapted to carry binary traffic encoded using Simple Binary Encoding (SBE), Google Protocol Buffers (GPB) and ASN.1 (Abstract Syntax Notation One). ASN.1 has also been adopted as an alternative encoding scheme for ISO-20022, the universal financial industry message scheme, which was initially XML based.

Adoption of ASN.1 is particularly interesting as this message structuring and encoding standard dates back to 1984. It has seen massive adoption in the telecommunications industry but was dismissed by the financial sector in favour of text based structuring and encoding particularly XML. ISO-20022 recognises a role for both XML and ASN.1 within applications by making the encodings compatible thus using XML for data visualisation and ASN.1 for data communication.

Taking the Message Further

Technology within financial institutions consists of ecosystems or systems of systems. Furthermore, emerging systems are following the microservices architectural pattern. These require substantial communication between services to achieve desired functionality. Even with heavily optimised components, message verbosity can substantially impact end to end performance of these systems.

Most of these systems continue to use text based protocols internally. XML is the most commonly used format followed by FIX and more recently JSON being the preference with the rise of RESTful microservices. All these messaging formats have a significant limiting effect on the overall system performance especially if these systems are composed of several fine grained microservices. Even when JSON is considered less verbose than XML, it is argued to be several orders less efficient than corresponding binary formats.

Developers may be misguided by a notion that RESTful services must communicate in JSON over HTTP(S). Motivating them to use binary messages over TCP may be challenging. REST response compression may be an argument in favour of text messages but with compression, inefficiency in the band is exchanged with the overheads in the box. Furthermore, additional tools may be required to inspect binary messages in flight for diagnosis and troubleshooting. However, this may be an acceptable cost to the performance gains of using binary encoding.

In Conclusion...

"The network is the computer.", John Gage

Messaging formats and protocols that represent information closest to its native representation are most efficient both in computation and communications. This is even more pertinent in modern architectural styles that are highly distributed and depend heavily on messaging between their components.