TCP is a reliable and connection-oriented protocol that uses flags and options to control the flow of data between two endpoints. In this article, you will learn how TCP SYN and FIN flags interact with other TCP flags and options, and how they affect the establishment and termination of TCP connections.
The TCP SYN flag is used to initiate a TCP connection by sending a synchronization request to the remote endpoint. The sender also includes its initial sequence number and other options, such as maximum segment size, window scale, and selective acknowledgment, in the SYN segment. The receiver responds with a SYN-ACK segment, which acknowledges the SYN and includes its own initial sequence number and options. The sender then confirms the SYN-ACK with an ACK segment, and the TCP connection is established. This three-way handshake ensures that both endpoints agree on the parameters of the connection and are ready to exchange data.
When referring to a segment, you're actually talking about the protocol data unit of TCP. Each of these segments are contained in packets which are typically contained in frames. (ethernet or wireless) Understanding this transaction can be really important when troubleshooting different IT and network issues. This is often one of the first things you would check for in a packet capture if there an issue to resolve or review.
The TCP FIN flag is used to terminate a TCP connection by signaling the end of data transmission to the remote endpoint. The sender sends a FIN segment, which indicates that it has no more data to send and requests to close the connection. The receiver acknowledges the FIN with an ACK segment, and enters the FIN-WAIT-2 state, where it can still send data to the sender. The sender enters the TIME-WAIT state, where it waits for a possible retransmission of the ACK or the FIN from the receiver. The receiver sends its own FIN segment when it has no more data to send, and the sender acknowledges it with an ACK segment. The receiver enters the TIME-WAIT state, and the sender enters the CLOSED state. The connection is closed after both endpoints wait for a period of time to ensure that no segments are lost or duplicated in the network.
The TCP RST flag is used to reset a TCP connection by aborting the communication and releasing the resources. The RST flag can be sent by either endpoint in response to an unexpected or invalid segment, such as a SYN segment for an already established connection, an ACK segment for a non-existent connection, or a segment with an incorrect sequence number or checksum. The RST flag can also be sent by an intermediate device, such as a firewall or a router, that blocks or filters the TCP traffic. The RST flag causes the receiver to discard any queued data and enter the CLOSED state, without performing the normal four-way handshake.
In the real world, I've seen this in a number of cases. It can indicate the service you are trying to transmit data to is no longer available (due a service crashing on a server being one example) or as mentioned here an appliance that is in the network path has aborted the connection. So in this case, it can be used in troubleshooting for where an issue lies. Security appliances such as a next generation firewall will send a RST packet to cancel a TCP session before the payload is fully downloaded to an endpoint if it finds that malicious software is inside the download. This is one mechanism for how this TCP flag is used to secure networks in certain circumstances.
The TCP ACK flag is used to acknowledge the receipt of data or control segments from the remote endpoint. The ACK flag is always set in every segment except the initial SYN segment. The ACK segment contains the next expected sequence number from the sender, which indicates how much data has been received and processed by the receiver. The ACK segment can also include other options, such as window size, congestion control, and selective acknowledgment, that inform the sender about the status and capacity of the receiver. The ACK segment can be sent separately or combined with data or control segments, depending on the timing and efficiency of the TCP algorithm.
The TCP URG flag is used to indicate that the segment contains urgent data that should be processed immediately by the remote endpoint. The URG flag is accompanied by a pointer that specifies the offset of the last byte of urgent data from the current sequence number. The receiver can use this pointer to separate the urgent data from the normal data and prioritize its delivery to the application layer. The URG flag is rarely used in modern TCP implementations, as most applications prefer to use separate channels or protocols for urgent data.
TCP options are additional fields that can be appended to the TCP header to provide more functionality and flexibility to the protocol. TCP options can be used to negotiate and communicate various parameters and features of the TCP connection, such as maximum segment size, window scale, selective acknowledgment, timestamp, and multipath TCP. TCP options have variable lengths and are padded with zeros to align with the 32-bit boundary of the TCP header. TCP options are optional and can be ignored by endpoints that do not support them. However, some options, such as maximum segment size and window scale, must be specified in the SYN and SYN-ACK segments to be effective for the duration of the connection.