Netty series (4) TCP unpacking and sticking

Netty series (4) TCP unpacking and sticking

1. Unpacking and sticking problems

(1) A small Socket Buffer problem

In stream-based transport such as TCP/IP, the received data is first stored in a socket receive buffer. Unfortunately, stream-based transport is not a packet queue, but a byte queue. Even if you send 2 separate packets, the OS doesn't treat it as 2 messages but only as a series of bytes. So there is no guarantee that the data you write remotely will be read accurately. As an example, let's assume that the operating system's TCP/TP stack has received 3 packets:

Due to the generic nature of streaming-based protocols, there is a high probability that data read in your application will be split into the following fragments.

Therefore, a receiver, whether he is a client or a server, should organize the received data into one or more data that are more interesting and can be better understood by the business logic of the program. In the above example, the received data should be structured in the following format:

test:

1. Send data three times on the client side to the server side

   //向服务器发送数据 buf
   f.channel().writeAndFlush(Unpooled.copiedBuffer("ABC".getBytes()));
   f.channel().writeAndFlush(Unpooled.copiedBuffer("DEF".getBytes()));
   f.channel().writeAndFlush(Unpooled.copiedBuffer("GHI".getBytes()));

2. The server side may regard the data transmitted three times as a request, and the result received by the server is as follows

   ABCDEFGHI

(2) Solutions

The solution to the problem of unpacking and sticking, according to mainstream protocols in the industry, there are three solutions, the first three Netty have been implemented:

1. The message length is fixed. For example, the size of each message is fixed at 200 bytes. If it is not enough, fill the blanks with blanks.

2. Add special characters at the end of the package for segmentation, such as adding a carriage return.

3. Divide the message into a fixed-length message header and a message body, and include a field representing the total length of the message in the message header, and then process the business logic. Usually the design idea is to use int32 for the first field of the message header to represent the total length of the message.

Second, the fixed length scheme - FixedLengthFrameDecoder

1. Add the following configuration to Server: [com.github.binarylei.t1]

childHandler(new ChannelInitializer<SocketChannel>() {
    @Override
    public void initChannel(SocketChannel sc) throws Exception {
        //定长拆包：5个字符，不足5位则忽略
        sc.pipeline().addLast(new FixedLengthFrameDecoder(5));
        //设置字符串形式的解码
        sc.pipeline().addLast(new StringDecoder());
        sc.pipeline().addLast(new ServerHandler());
    }
})

1. Receive the requested data in ServerHandler:

public void channelRead(ChannelHandlerContext ctx, Object msg) {
    System.out.println((String)msg);

    //写给客户端
    ChannelFuture f = ctx.writeAndFlush(Unpooled.copiedBuffer(((String)msg).getBytes()));
    //写完成后会自动关闭客户端
    //f.addListener(ChannelFutureListener.CLOSE);
}

1. Data sent by Client:

//向服务器发送数据 buf
f.channel().writeAndFlush(Unpooled.copiedBuffer("aaaaabbbbb".getBytes()));
f.channel().writeAndFlush(Unpooled.copiedBuffer("cccccddd".getBytes()));

1. The results are as follows, it can be seen that 5 characters are processed as a request, and those with less than 5 characters are ignored:

aaaaa
bbbbb
ccccc

3. Fixed separator scheme - DelimiterBasedFrameDecoder

1. Add the following configuration to Server: [com.github.binarylei.t2]

childHandler(new ChannelInitializer<SocketChannel>() {
    @Override
    public void initChannel(SocketChannel ch) throws Exception {
        ByteBuf buf = Unpooled.copiedBuffer("$_".getBytes());
        ch.pipeline().addLast(new DelimiterBasedFrameDecoder(1024, buf));
        //设置字符串形式的解码
        ch.pipeline().addLast(new StringDecoder());
        ch.pipeline().addLast(new ServerHandler());
    }
})

1. Receive the requested data in ServerHandler:

public void channelRead(ChannelHandlerContext ctx, Object msg) {
    System.out.println((String)msg);

    //写给客户端
    ChannelFuture f = ctx.writeAndFlush(Unpooled.copiedBuffer("netty$_".getBytes()));
    //写完成后会自动关闭客户端
    f.addListener(ChannelFutureListener.CLOSE);

}

The results are as follows, and it can be seen that the request is processed three times:

ABC
DEF
GHI

custom protocol

Netty custom protocol please refer to this article

---

Record a little bit every day. Content may not be important, but habits are!