Jumbo frames are packets that exceed the standard Ethernet frame size of 1,500 bytes. They can be used in TCP, a protocol that enables reliable and ordered data transmission between devices on a network. But why would you want to use jumbo frames in TCP, and what are the trade-offs? In this article, we will explore the benefits and drawbacks of using jumbo frames in TCP.
One of the main advantages of using jumbo frames in TCP is that they can increase the throughput, or the amount of data transferred per unit of time. This is because jumbo frames reduce the overhead of packet headers and TCP acknowledgments, which consume bandwidth and processing power. By sending fewer but larger packets, jumbo frames can improve the efficiency of data transmission and reduce the latency, or the delay between sending and receiving data.
If you look at the header size (40ish bytes) vs data size of 1460, you see that in the max size the ratio is 97%~ Making the size much bigger make the connection break because of window size issue on the other side, and retransmissions. Together with a lot of comparability issues mentioned below, it’s usually a bad idea.
As @Yoav Karmon says, there isn't much to be gained with increasing frame size. Much more important is to ensure the TCP receive window size is properly tuned for the latency, bandwidth, and throughput congestion. Too large a TCP receive window and you'll end up with TCP buffer bloat (data in TCP buffers for too long). Too little and the available bandwidth over the network path won't be used because the transmitting data will be waiting for acknowledgement from the receiving end. There are newer TCP/IP protocol options that automatically adjust these settings these days, but if writing an extremely light / primitive / performance optimized network stack it might be useful.
Less packets = less interupts and less packet overhead. Can be a a big win on cpu limited hosts. Storage networks (FoCE, RoCE, smb direct) can reallt benefit
What I would like to see, is some regression model that predicts retransmissions probability, with some predictions on the “damage “ it’s doing to the session. Maybe incorporate number of hops , internet connection speed and type (optic, satellite etc…) Usage type: maybe Netflix is ok with some buffering but not live streaming : live sports
Another benefit of using jumbo frames in TCP is that they can lower the CPU usage of the devices involved in the data transfer. This is because jumbo frames reduce the number of interrupts, or signals that notify the CPU of an event that requires its attention. Interrupts are generated every time a packet is received or sent, and they consume CPU cycles and memory. By sending fewer but larger packets, jumbo frames can reduce the interrupt rate and free up CPU resources for other tasks.
Most modern network interfaces have and onboard ASIC that handles the encap/decap work with little need for a cpu. The jumbo frame model does cut down on that work, providing every device in the network path supports the frame size.
One of the main drawbacks of using jumbo frames in TCP is that they can cause fragmentation, or the breaking up of packets into smaller pieces to fit the maximum transmission unit (MTU) of the network. The MTU is the largest packet size that a network can handle without fragmentation. If a device sends a packet that is larger than the MTU of the network, the packet will be fragmented by a router or a switch along the path. Fragmentation can reduce the performance and reliability of data transmission, as it increases the overhead, latency, and error rate.
Another drawback of using jumbo frames in TCP is that they can cause compatibility issues with different network devices and protocols. Not all devices support jumbo frames, and some may have different limits on the maximum frame size. For example, some routers may support up to 9,000 bytes, while some switches may support up to 12,000 bytes. If a device sends a packet that is larger than the supported frame size of another device, the packet will be dropped or truncated, resulting in data loss or corruption. Moreover, some protocols, such as IPsec or VPN, may add extra headers or encryption to the packets, which can exceed the frame size and cause fragmentation or dropping.
To use jumbo frames in TCP, you need to configure both the sender and the receiver devices to support the same frame size. You also need to ensure that all the intermediate devices along the network path support the same or larger frame size. You can use tools such as ping or traceroute to test the MTU of the network and detect any potential bottlenecks or problems. You also need to adjust the TCP maximum segment size (MSS), which is the largest amount of data that a TCP segment can carry. The MSS should be equal to the frame size minus the IP and TCP headers, which are usually 40 bytes.
Supercharge your network for TCP transfers by sending larger data chunks. Think of them as bigger trucks on the data highway! But ensure all devices on the path can handle them.
Using jumbo frames in TCP can be beneficial for certain scenarios, such as transferring large files or streaming high-quality video or audio. However, using jumbo frames in TCP can also have drawbacks, such as fragmentation, compatibility issues, and increased complexity. Therefore, you should weigh the pros and cons of using jumbo frames in TCP and decide whether they are suitable for your network and application needs.