Use Case of Timestamping Control Packets using Linux Socket Options

Introduction

RFC 3918 outlines various benchmarking tests to evaluate the multicast performance of a Device Under Test (DUT) by measuring its throughput, forwarding capabilities, latency, and Internet Group Management Protocol (IGMP) group membership behaviour. One such test is referred to as "Overhead," (Section 6 of RFC 3918) which aims to measure both the "Group Join Delay" and "Group Leave Delay."

Group Join Delay Test: This test measures the time taken by a multicast-capable router to start forwarding multicast frames after an IGMP Join message is sent from a receiver to the router.

• Group Join Delay = RxTS (timestamp of the first multicast frame received) - TxTS (timestamp of the first IGMP Join sent)

Group Leave Delay Test: This test measures the time taken by a router to cease forwarding multicast frames after an IGMP Leave message is sent from the receiver to the router.

• Group Leave Delay = RxTS (timestamp of the last multicast frame received) - TxTS (timestamp of the first IGMP Leave sent)

In this article, we will explore the process of timestamping IGMP packets and multicast data traffic using specific Linux socket options. The timestamping helps in precisely measuring packet transmission delays and processing times during the evaluation. Let's start with the fundamentals & the steps to understand how multiple IGMP hosts can receive IPv4 multicast traffic

Understanding Multicast & IGMP Basics

• Multicast Traffic: In multicast communication, data is sent from one source to multiple destinations (hosts) that are part of a multicast group. The key advantage is that the sender only needs to transmit a single stream of data, regardless of how many hosts want to receive it.
• IGMP: The Internet Group Management Protocol (IGMP) is responsible for managing membership in multicast groups. It allows hosts to join or leave multicast groups on a local network (IPv4). Routers & Switches use IGMP to know which hosts are interested in receiving multicast traffic for specific multicast addresses.
• Multicast Address Range (IPv4): IPv4 multicast addresses are within the range 224.0.0.0 to 239.255.255.255. These addresses are used to identify multicast groups.

Receiving IPv4 Multicast Traffic using IGMP

The topology below illustrates how IPv4 multicast traffic from a source is received by multiple receivers:

1. The IPv4 Multicast Source generates multicast traffic with a destination IP within a multicast group range (for instance, 225.1.2.3) and transmits it to the receivers. The traffic is forwarded by the Querier or Router, assuming all devices are within the same network. In this setup, no configuration of Multicast Routing Protocols (such as PIM or DVMRP) is considered.
2. The IGMP Snooping Switch will not forward the multicast traffic until a forwarding path is established in its Multicast Forwarding Database (MFDB).
3. The Querier periodically sends a General Query message to the Receivers through the IGMP Snooping Switch.
4. Upon receiving the General Query message, the Receivers respond by sending IGMP Join messages to indicate their interest in receiving traffic addressed to 225.1.2.3.
5. The IGMP Snooping Switch learns the IGMP Join messages and stores the information in its Group Membership Table. This action also initiates the creation of a forwarding entry in the MFDB, enabling the traffic to be successfully forwarded to the interested Receivers.

Timestamping IGMP frames

Now, let's explore how IGMP receivers running on Linux OS can be configured to receive multicast traffic using simple socket programming, and how timestamping can be enabled for IGMP Join or Leave messages.

1. Create a datagram socket.
2. Bind the socket to any arbitrary port and set the IP address to INADDR_ANY.
3. Use the socket option IP_ADD_MEMBERSHIP to join the multicast group and enable the reception of multicast data traffic.

The above steps will establish multicast group membership on the specified interface, which can be verified using the “ip maddr” command. This configuration will also enable the IGMP protocol, allowing the interface to respond to General Query messages with IGMP v2/v3 Report messages. Below is the pseudo code for setting up the Group Membership:

int sd = socket(AF_INET, SOCK_DGRAM, 0);
memset((char *)&localSock, 0, sizeof(localSock));
localSock.sin_family = AF_INET;
localSock.sin_port = htons(socket_port);
localSock.sin_addr.s_addr = INADDR_ANY;
if (bind(sd, (struct sockaddr *)&localSock, sizeof(localSock))) {
    close(sd);
    exit(1);
}
group.imr_interface.s_addr = inet_addr("20.20.20.10");
group.imr_multiaddr.s_addr = inet_addr("225.1.2.3");
if (setsockopt(sd, IPPROTO_IP, IP_ADD_MEMBERSHIP, (char *)&group, sizeof(group)) < 0) {
    close(sd);
    exit(1);
}        

To enable timestamping for any packet sent from a socket, the standard mechanism involves using the SO_TIMESTAMPING socket option [Source]. This option is part of the socket programming API in Linux and allows applications to receive timestamps for specific events associated with packet transmission and reception.

When this socket option is enabled, the operating system will automatically attach timestamp information to outgoing packets and incoming packets. These timestamps can provide valuable insights into network performance, such as measuring latency, determining round-trip times, and assessing overall transmission delays. Below is the pseudo code for setting it up:

/* Add HW timestamping if there is a support for accurate results */
int enable = SOF_TIMESTAMPING_SOFTWARE | SOF_TIMESTAMPING_OPT_ID | SOF_TIMESTAMPING_TX_SOFTWARE;
int ok = setsockopt(sd, SOL_SOCKET, SO_TIMESTAMPING, &enable, sizeof(int));
if (ok < 0) {
  return;
}        

However, the code mentioned above may not be able to generate and retrieve the desired transmit (Tx) timestamp for the initial IGMP report sent out, since the kernel implementation (located in net/ipv4/igmp.c) is used when the IP_ADD_MEMBERSHIP socket option is enabled.

One possible solution to this problem is to implement a custom flag (socket option) in the kernel and driver code to activate hardware or software timestamping using the “ioctl” system call [it's implementation is beyond the scope of this article].

Alternatively, a raw socket can be used to send the desired IGMP v2/v3 Report message from user space, with the SO_TIMESTAMPING socket option enabled specifically for that packet. Afterward, the above pseudo code can be applied to enable IGMP within the kernel.

The pseudo code below outlines all the necessary steps for implementing the later approach. Use the code comments as guidance to write the logic for constructing and sending the IGMP Join frame, followed by retrieving the Tx timestamp.

/* Step-1: Open a raw socket */
int sock_raw = socket(AF_PACKET,SOCK_RAW,IPPROTO_RAW);
/* Step-2: Constructing the Ethernet header */
/* Step-3: Constructing the IP header */
/* Step-4: Construct IGMP Header */
/* Step-5: Send the frame */
int send_len = sendto(sock_raw,sendbuff,64,0,(const struct sockaddr*)&sadr_ll,sizeof(struct sockaddr_ll));
if(send_len<0) {
    printf("error in sending....sendlen=%d....errno=%d\n",send_len,errno);
    return;
}
/* Step-6: Prepare the message header for receiving the packet and ancillary data */
memset(&msg, 0, sizeof(msg));
msg.msg_control = control;
msg.msg_iov = &iov;
msg.msg_iovlen = 1;
iov.iov_base = buffer;
iov.iov_len = 2048;
do {
    msg.msg_controllen = 1024;
    got = recvmsg(sock_raw, &msg, MSG_ERRQUEUE);
} while (got < 0 && errno == EAGAIN && check++ < check_max);
if ( got < 0 && errno == EAGAIN ) {
    printf("Gave up acquiring timestamp.\n");
    return;
}
/* Step-7: Extracting and printing the tx timestamp */
for (cmsg = CMSG_FIRSTHDR(&msg); cmsg != NULL; cmsg = CMSG_NXTHDR(&msg, cmsg)) {
    if (cmsg->cmsg_level != SOL_SOCKET)
        continue;
    switch(cmsg->cmsg_type) {
        case SO_TIMESTAMPING:
            udp_tx_stamp = (struct timespec*) CMSG_DATA(cmsg);
            printf("System timestamp ""%" PRIu64 ".%.9" PRIu64 " ""\n", (uint64_t)(udp_tx_stamp[0]).tv_sec, (uint64_t)(udp_tx_stamp[0]).tv_nsec);
            break;
        default:
            /* Ignore other cmsg options */
            break;
    }
}        

Using the method described above, the TxTS (timestamp of the first IGMP Join sent) is calculated. Now, we will explore how to compute the RxTS (timestamp of the first multicast frame received) in a similar manner. This involves capturing the timestamp of the first multicast packet arriving at the receiver. By applying the same socket timestamping mechanism used for TxTS, we can accurately capture RxTS and measure the overall delay between the IGMP Join request and the receipt of the multicast frame.

/* Step-1: Open a raw socket */
int sock_raw = socket(AF_PACKET,SOCK_RAW,IPPROTO_RAW);

/* Step-2: Enable Timestamp */
int enable = 1;
setsockopt(sock_raw, SOL_SOCKET, SO_TIMESTAMP, &enable, sizeof(enable));

struct msghdr msg;
struct iovec iov;
struct sockaddr_storage src_addr;
char buffer[4096];
char control_buffer[1024];  // Ancillary data buffer
struct cmsghdr *cmsg;

/* Step-3: Prepare the message header for receiving the packet and ancillary data */
iov.iov_base = buffer;
iov.iov_len = sizeof(buffer);
msg.msg_name = &src_addr;
msg.msg_namelen = sizeof(src_addr);
msg.msg_iov = &iov;
msg.msg_iovlen = 1;
msg.msg_control = control_buffer;
msg.msg_controllen = sizeof(control_buffer);
msg.msg_flags = 0;

/* Step-4: Receive the first packet and capture ancillary data (timestamp) */
recvmsg(sock_raw, &msg, 0);

/* Step-5: Extracting and printing the rx timestamp */
for (cmsg = CMSG_FIRSTHDR(&msg); cmsg != NULL; cmsg = CMSG_NXTHDR(&msg, cmsg)) {
    if (cmsg->cmsg_level != SOL_SOCKET)
        continue;
    switch(cmsg->cmsg_type) {
        case SO_TIMESTAMPING:
            udp_rx_stamp = (struct timespec*) CMSG_DATA(cmsg);
            printf("System timestamp ""%" PRIu64 ".%.9" PRIu64 " ""\n", (uint64_t)(udp_rx_stamp[0]).tv_sec, (uint64_t)(udp_rx_stamp[0]).tv_nsec);
            break;
        default:
            /* Ignore other cmsg options */
            break;
    }
}        

Once the RxTS (the timestamp of the first received multicast frame) is obtained, the delay can be calculated by determining the difference between the TxTS and the RxTS. This time difference represents the network latency or delay experienced during the transmission of the multicast packet.

Conclusion

The methods described in the previous sections can be applied to retrieve timestamp values for the IGMP Join and IPv4 Multicast frames, enabling the calculation of the "Group Join Delay". Similarly, to calculate the "Group Leave Delay", the same methods can be applied to obtain the timestamps for the first IGMP Leave frame and the last multicast frame. It is important to note that for improved accuracy in calculating these overhead delays, hardware timestamping is recommended.