TCP unpacking and sticking causes and solutions

1. The concept of unpacking and sticking

The picture above shows the general process of TCP protocol transmission.The nature of its data transmission is streaming, and there is no concept of segmentation, So this process may have 3 situations:

1. 正常情况
   The sending end sent two messages, and the receiving end also read two data packets. The first packet contained the complete information of the first message sent by the sender, and the second packet contained the complete second message. In this case, the receiving end only needs to simply read the data from the buffer, which can be processed correctly
2. TCP粘包
   The sender has sent two messages, and the receiver can only read one packet. The complete information of the two messages sent by the sender is contained in one data packet, and the receiver does not know where the first message ends and where the second message starts. It is difficult to process the data correctly.
3. TCP拆包
   The sender sends two messages, and the receiver receives a total of two data packets. One of the data packets contains only part of one message, and the second half of this message and the other message are in another data packet.In short, a message is split into two packets and sent

2. Reasons for unpacking and sticking

TCP协议Transmit data in a stream,The smallest unit of transmission is a segment (segment). HeaderThere is one of the TCP packets Options标识位, and its common identifier MSS(Maximum Segment Size)refers to the maximum length of the content of the packet transmission.This value is usually MTU minus IP头(20 Byte)和TCP头(20 Byte)the size, usually 1460 Byte

MTU(Maximum Transmission Unit)It is the maximum limit of the data transmitted by the data link layer at a time, usually 1500 Byte, data exceeding this size must be divided into multiple segments

To improve performance, the TCP sender will send the data that needs to be sent to the buffer,Wait for the buffer to be full or trigger a flush operation before sending the data in the buffer to the receiver. Similarly, the receiver also has a buffer for receiving data, so TCP sticking and unpacking mainly have the following reasons:

1. The data sent by the sender passes through TCP segmentation of MSS (maximum message length) size, and TCP 数据量 - TCP头部长度 > MSSunpacking will occur at that time
2. If the data written by the sender is larger than the socket buffer size, unpacking will occur
3. The data written by the sender is less than the size of the socket buffer, and the network card sends the data written multiple times together, and sticky packets will occur
4. If the receiver does not read the socket buffer data in time, sticky packets may occur

3. Solution

The underlying TCP protocol itself does not care about the business data of the upper layer, so the occurrence of sticking and unpacking cannot be avoided. It can only be controlled on the application layer data protocol. The commonly used methods are as follows:

1. Use the protocol with message header. The
   message header stores the message start identifier and message length information. When the receiver obtains the message header, it parses out the message length, and then reads the length of the content backwards. The number of bits is not enough to fill in 0
2. Set a fixed-length message, the
   receiving end reads the content of a predetermined length each time as a complete message, and the insufficient space is filled
3. Set the message boundary
   . A newline character can be added at the end of the message to indicate a complete message, so that the receiving end can judge whether the message is complete based on the newline character

There are ready-made tools for TCP sticky packet unpacking in the Netty framework, and it can be realized by adding the corresponding decoder to its pipline. Note that once the corresponding decoder is added, the data must be in compliance with its specifications when sending data, otherwise the message may not be read normally

• LineBasedFrameDecoder is resolved based on line breaks
• DelimiterBasedFrameDecoder is resolved based on the separator
• FixedLengthFrameDecoder specifies the length to solve