Memory Tuning in Linux for Ultra-Low Latency Trading

In my previous articles, I covered CPU optimization and Network Optimization using Solarflare NICs for ultra-low latency trading. These optimizations are crucial for reducing execution time and improving trading performance. However, to further optimize your trading systems, memory tuning plays an equally critical role. Efficient memory management ensures your system can handle high-frequency trading workloads with minimal delays.

In this article, I’ll cover a range of memory tuning techniques in Linux that are essential for achieving ultra-low latency in trading environments. Given the number of techniques, I'll keep things brief to cover as much as possible, allowing you to dive deeper into your own research on each topic.

Fast and predictable memory access is crucial to ensuring trading algorithms operate at peak performance. I want to emphasize the importance of "predictability" here. In Random Access Memory, the more predictable the memory access, the faster the performance. Memory bottlenecks, such as page faults or cross-NUMA node access, can lead to unacceptable delays. By fine-tuning memory configurations, we can reduce these latencies and achieve a smoother, faster execution pipeline for trades.

Huge Pages: Reducing Page Table Lookups

One of the first steps in optimizing memory for low latency is enabling huge pages. Typically, Linux uses 4KB memory pages, but for ultra-low latency systems, we can reduce memory management overhead by using huge pages (2MB or 1GB). This decreases the number of page table lookups and TLB (Translation Lookaside Buffer) misses, speeding up memory access.

Benefits of Huge Pages:

- Fewer TLB misses, resulting in faster access to memory.

- Reduced overhead for managing page tables.

We can also use transparent huge pages for automatic management, but for precise control, static huge pages are recommended in trading systems.

NUMA Optimization: Reducing Cross-Node Latency

I briefly spoke about NUMA in CPU optimization article. NUMA (Non-Uniform Memory Access) can significantly impact latency when memory is allocated on a node that is different from the CPU running the process. . In multi-core or multi-processor systems, ensuring that memory is located on the same NUMA node as the executing CPU can greatly reduce access times.

Key NUMA Tuning Techniques:

- Bind processes and memory to the same NUMA node.This reduces cross-node memory access, improving latency.

- Interleave memory across NUMA nodes for workloads spread across multiple cores. This balances memory allocation and reduces bottlenecks.

Disable Swap: Keep Everything in RAM

Swap space introduces significant latency when processes move data between RAM and disk. In low-latency trading systems, even minor delays can affect performance, so it’s crucial to disable swap entirely, ensuring that all operations stay in RAM.

Pre-Allocate Memory: Avoid Allocation Overhead

Memory allocation during runtime can introduce unpredictable latencies. By pre-allocating memory buffers for trading algorithms at application startup, you can avoid the costly overhead of on-demand memory allocation. This ensures that critical processes have memory readily available when needed.

Reduce Dirty Cache Writeback Time

The dirty page writeback mechanism in Linux can introduce delays as the kernel periodically writes modified pages to disk. By reducing the writeback interval, we can ensure that the system writes data more frequently, preventing large bursts of I/O operations at critical times.

Memory Locking: Prevent Paging Out Critical Data

Using memory locking ensures that critical memory pages are not swapped out or paged out by the kernel. This is particularly useful in ensuring that key processes have uninterrupted access to their required memory.

Disable Transparent Huge Pages (if necessary)

While transparent huge pages (THP) can provide performance benefits by automatically managing larger page sizes, they can also introduce latency spikes due to page defragmentation. For ultra-low latency trading, it’s often better to disable THP and manually manage huge pages for more consistent performance.

Adjust VFS Cache Pressure

In ultra-low latency environments, you want the file system cache to be efficiently managed without excessive reclaiming of memory. By reducing the vm.vfs_cache_pressure, you allow the system to keep important file system metadata in memory longer, reducing the overhead of retrieving it.

Optimize Swappiness and Overcommit Memory

By reducing swappiness (how aggressively the kernel swaps out idle processes) and tuning overcommit_memory (which controls memory allocation behavior), you can further minimize the likelihood of paging and ensure that critical trading processes have guaranteed memory access.

Final Thoughts

In high-frequency trading, optimizing memory performance is just as important as tuning your CPU or network stack. By implementing techniques such as huge pages, NUMA optimization, and disabling swap, you can significantly reduce memory-related latencies and improve the overall performance of your trading infrastructure.

Combining memory tuning with CPU and network optimizations allows for a well-rounded, ultra-low latency trading system. In my next article, I’ll dive deeper into storage and I/O tuning to further enhance trading performance.