A TCP option that improves Frontends and Backends latency

99% of network latency issues are caused by the user app logic. App logic here includes libraries and frameworks used by the app. Sometimes however, the 1% could be the kernel. Here is one example where a config to TCP can improve backend and frontend network latency.

When an app writes to a socket connection, whether its the frontend sending a request or the backend sending a response, the raw bytes are copied from the user space process application to the kernel memory where TCP/IP takes place.

Each connection has a receive buffer where data from the other party arrives and a send buffer where the data from the application goes in before it is sent to the network. Both send and receive buffers live in the kernel space.

Data written by the app to the kernel are not sent immediately but instead buffered in the kernel. The kernel hope is to get a full size worth of data to fill TCP segment. This is called an MSS or maximum segment size which is often around 1500 bytes.

The reason of the buffering is segment overhead, Each segment comes with ~40 bytes header, the overhead of sending few bytes with such a large header can lead to inefficient use of network bandwidth. Thus the buffering, classic computer science stuff.

So by default the kernel delays sending the segments out through the network in hopes of receiving more data from the application to fill a full MSS. The algorithm which specifies when to delay and by how much is called Nagle’s algorithm.

You can disable this delay by setting the TCP_NODELAY on the socket option while creating the connection. This causes the kernel to send whatever it has in the send buffer even if it’s a few bytes. This is great because sometimes few bytes is all what we have. This essentially favor low latency over network efficiency.

For the backend, applications can benefit from enabling this option (disabling the delay) especially when writing responses back to the client. This is because responses are sent through the send buffer. Delaying sending segments just because they are not full can lead to slowdowns in writing responses. NodeJS has enabled this option in this PR.

For the frontend, apps can benefit from this option. In 2016 the creator of cURL Daniel spent hours debugging a TLS latency issue only to find out that the kernel was sitting on few bytes of TLS content in segment waiting for a full MSS. That caused the cURL project to set the TCP_NODELAY by default.