Complete code: https://gitee.com/firewolf/java-io/tree/master/java-io/netty-02-tcppacket

A, TCP stick package / unpacking problem

TCP is a "flow" of the agreement, the so-called flow, which is not a bunch of digital boundaries in the underlying TCP is a connection to the piece, and no boundaries, because they do not know the meaning of service layer, it will be based on the actual situation of the TCP buffer division, resulting in a business layer of view, a full TCP packet may be split into a plurality of packets, also possible to package the plurality of small packets to form a large package.
Stick package and unpacking may occur following:
Here Insert Picture Description
Suppose the client sends two packets D1 and D2 to the server, since the number of bytes read a server is not fixed, so that the following cases may occur:

Twice server to read two complete packet, respectively D1 and D2, stick package and unpacking do not occur;
A server to read the two data packets, D1 and D2 stuck together, a stick package occurred;
Server twice two read data packets, first read the complete packet D1 and D2 D2_1 - part of the package, the second reading of a portion of another packet D2 D2_2. Stick package and unpacking occurred;
Server read twice two packets, the first read D1_1 - D1 packet portion, another portion of the second reading of the full package D1_2 D1 and D2 of the packet, and the occurrence of the stick package unpacking;
In addition, and it may also occur repeatedly stick package unpacking;

Second, the reason for TCP stick package / unpacking issues arising

The reason there are three main problems:

Write byte size written application socket is greater than the size of the send buffer;
For TCP segment size MSS
Ethernet payload larger than the MTU executing IP fragmentation

Third, the solution strategy stick package TCP / unpacking problem

Fixed-length message, such as the length of each packet is 200, when the insufficient, spaces are filled;
A carriage return at the end of the increase divided, such as the TCP protocol;
At the end add special characters segmentation, carriage return line is a special case;
Message into the message header and a message body, message header comprising a message indicating a total length field;
Using a more complex application layer protocol;

Four, TCP stick package / unpacking problems demo

Here's write a simple program, function is the client after the connection server, send a string to the server 20, displays the server receives:

(A) server

package com.firewolf.java.io.netty.packagee.paste.origin;

import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelHandlerAdapter;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelOption;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import java.util.HashMap;
import java.util.Map;

/**
 * 作者：刘兴 时间:2019/5/15
 **/
public class HelloServer {


  public HelloServer(int port) {

    EventLoopGroup bossGroup = new NioEventLoopGroup();
    EventLoopGroup workGroup = new NioEventLoopGroup();
    try {
      ServerBootstrap bootstrap = new ServerBootstrap();
      bootstrap.group(bossGroup, workGroup)
          .channel(NioServerSocketChannel.class)
          .option(ChannelOption.SO_BACKLOG, 1024)
          .childHandler(new ChannelInitializer<SocketChannel>() {
            @Override
            protected void initChannel(SocketChannel socketChannel) throws Exception {
              socketChannel.pipeline()
                  .addLast(new MessageServerHandler());
            }
          });
      ChannelFuture future = bootstrap.bind(port).sync();
      System.out.println("启动服务端监听端口：" + port);
      future.channel().closeFuture().sync();

    } catch (InterruptedException e) {
      e.printStackTrace();
    } finally {
      bossGroup.shutdownGracefully();
      workGroup.shutdownGracefully();
    }

  }

  class MessageServerHandler extends ChannelInboundHandlerAdapter {

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
      ByteBuf buf = (ByteBuf) msg;
      byte[] bytes = new byte[buf.readableBytes()];
      buf.readBytes(bytes);
      String message = new String(bytes);
      System.out.println("有客户端消息：" + message);
      buf.release();
    }
  }


  public static void main(String[] args) {
    new HelloServer(9999);
  }

}

(B) Client

package com.firewolf.java.io.netty.packagee.paste.origin;

import io.netty.bootstrap.Bootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelHandlerAdapter;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelOption;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import java.net.InetSocketAddress;

/**
 * 作者：刘兴 时间:2019/5/15
 **/
public class HelloClient {


  public HelloClient(String host, int port) {
    NioEventLoopGroup group = new NioEventLoopGroup();
    try {
      Bootstrap bootstrap = new Bootstrap();
      bootstrap.group(group)
          .channel(NioSocketChannel.class)
          .option(ChannelOption.TCP_NODELAY, true)
          .handler(new ChannelInitializer<SocketChannel>() {
            @Override
            protected void initChannel(SocketChannel socketChannel) throws Exception {
              socketChannel.pipeline()
                  .addLast(new MessageClientHandler());
            }
          });

      ChannelFuture f = bootstrap.connect(new InetSocketAddress(host, port)).sync();
      System.out.println("连接服务器成功-----");
      f.channel().closeFuture().sync();
    } catch (InterruptedException e) {
      e.printStackTrace();
    } finally {
      group.shutdownGracefully();
    }
  }


  class MessageClientHandler extends ChannelInboundHandlerAdapter {

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {

      //遍历写出100个字符串
      for (int i = 0; i < 20; i++) {
        String message = "helllo," + i + System.getProperty("line.separator");
        ByteBuf buf = Unpooled.buffer(message.getBytes().length);
        buf.writeBytes(message.getBytes());
        ctx.writeAndFlush(buf);
      }
    }
  }


  public static void main(String[] args) {
    new HelloClient("127.0.0.1", 9999);
  }
}

Start the server, and then start the client, the server print the following information:

有客户端消息：helllo,0
helllo,1
helllo,2
helllo,3
helllo,4
helllo,5
helllo,6
helllo,7
helllo,8
helllo,9
helllo,10
helllo,11
helllo,12
helllo,13
helllo,14
helllo,15
helllo,16
helllo,17
helllo,18
helllo,19

We will find, in fact, once the server to accept all the strings, that is to say, these packages do not normally receive, but occurred stick package.

Five, Netty to stick package / unpacking problem solving

To solve the problem of half-read packet unpacking TCP / stick package result, Netty provides many default codec for processing packet half.

(A) LineBasedFrameDecoder

LineBasedFrameDecoder package will identify, cut according carriage (13) inside the package wrap (10) character.

1. The server code

package com.firewolf.java.io.netty.packagee.paste.solve;

import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelHandlerAdapter;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelOption;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.LineBasedFrameDecoder;
import io.netty.handler.codec.string.StringDecoder;
import java.util.HashMap;
import java.util.Map;

/**
 * 作者：刘兴 时间:2019/5/15
 **/
public class HelloServerSolve {


  public HelloServerSolve(int port) {

    EventLoopGroup bossGroup = new NioEventLoopGroup();
    EventLoopGroup workGroup = new NioEventLoopGroup();
    try {
      ServerBootstrap bootstrap = new ServerBootstrap();
      bootstrap.group(bossGroup, workGroup)
          .channel(NioServerSocketChannel.class)
          .option(ChannelOption.SO_BACKLOG, 1024)
          .childHandler(new ChannelInitializer<SocketChannel>() {
            @Override
            protected void initChannel(SocketChannel socketChannel) throws Exception {
              socketChannel.pipeline()
                  //添加编码器
                  .addLast(new LineBasedFrameDecoder(1024))
                  .addLast(new StringDecoder())
                  .addLast(new MessageServerHandler());
            }
          });
      ChannelFuture future = bootstrap.bind(port).sync();
      System.out.println("启动服务端监听端口：" + port);
      future.channel().closeFuture().sync();

    } catch (InterruptedException e) {
      e.printStackTrace();
    } finally {
      bossGroup.shutdownGracefully();
      workGroup.shutdownGracefully();
    }

  }

  class MessageServerHandler extends ChannelInboundHandlerAdapter {


    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
      //由于StringDecoder的功劳，接受消息的时候，我们可以直接转成String
      String message = (String) msg;
      System.out.println("有客户端消息：" + message);
    }
  }


  public static void main(String[] args) {
    new HelloServerSolve(9999);
  }

}

There are two changes to the server

Added to the boot class BootStrap message decoder;
When receiving a message directly to accept String, this is due to the addition StringDecoder has the function.

2. The client code

package com.firewolf.java.io.netty.packagee.paste.solve;

import io.netty.bootstrap.Bootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelHandlerAdapter;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelOption;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.LineBasedFrameDecoder;
import io.netty.handler.codec.string.StringDecoder;
import java.net.InetSocketAddress;

/**
 * 作者：刘兴 时间:2019/5/15
 **/
public class HelloClientSolve {


  public HelloClientSolve(String host, int port) {
    NioEventLoopGroup group = new NioEventLoopGroup();
    try {
      Bootstrap bootstrap = new Bootstrap();
      bootstrap.group(group)
          .channel(NioSocketChannel.class)
          .option(ChannelOption.TCP_NODELAY, true)
          .handler(new ChannelInitializer<SocketChannel>() {
            @Override
            protected void initChannel(SocketChannel socketChannel) throws Exception {
              socketChannel.pipeline()
                  .addLast(new MessageClientHandler());
            }
          });

      ChannelFuture f = bootstrap.connect(new InetSocketAddress(host, port)).sync();
      System.out.println("连接服务器成功-----");
      f.channel().closeFuture().sync();
    } catch (InterruptedException e) {
      e.printStackTrace();
    } finally {
      group.shutdownGracefully();
    }
  }


  class MessageClientHandler extends ChannelInboundHandlerAdapter {

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {

      //遍历写出100个字符串
      for (int i = 0; i < 20; i++) {
        String message = "helllo," + i + System.getProperty("line.separator");
        ByteBuf buf = Unpooled.buffer(message.getBytes().length);
        buf.writeBytes(message.getBytes());
        ctx.writeAndFlush(buf);
      }
    }
  }


  public static void main(String[] args) {
    new HelloClientSolve("127.0.0.1", 9999);
  }
}

We can see that the client does not have any change.

3. Check results

When you turn the print reads as follows:

有客户端消息：helllo,0
有客户端消息：helllo,1
有客户端消息：helllo,2
有客户端消息：helllo,3
有客户端消息：helllo,4
有客户端消息：helllo,5
有客户端消息：helllo,6
有客户端消息：helllo,7
有客户端消息：helllo,8
有客户端消息：helllo,9
有客户端消息：helllo,10
有客户端消息：helllo,11
有客户端消息：helllo,12
有客户端消息：helllo,13
有客户端消息：helllo,14
有客户端消息：helllo,15
有客户端消息：helllo,16
有客户端消息：helllo,17
有客户端消息：helllo,18
有客户端消息：helllo,19

You can see, these messages are separated, there is no question of the stick package and unpacking.
LineBasedFrameDecoder: a carriage return as a packet to be cut;
StringDecoder: converts a string ByteBuf
general case, StringDecoder and LineBasedFrameDecoder be used in conjunction.

（二）DelimiterBasedFrameDecoder

In fact DelimiterBasedFrameDecoder functions and LineBasedFrameDecoder similar, but it will be more powerful than LineBasedFrameDecoder, designated to support our own character cut (string), use and LineBasedFrameDecoder almost exactly the same, here simply put part of the code:

1. Add the server decoder

As used herein _ $ divided:

 socketChannel.pipeline()
                  //添加编码器
                  .addLast(new DelimiterBasedFrameDecoder(1024, Unpooled.copiedBuffer("_$".getBytes())))

2. When sending a message, to bring the end of $ _

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {

      //遍历写出100个字符串
      for (int i = 0; i < 20; i++) {
        String message = "helllo," + i + "_$";
        ByteBuf buf = Unpooled.buffer(message.getBytes().length);
        buf.writeBytes(message.getBytes());
        ctx.writeAndFlush(buf);
      }
    }

（三）FixedLengthFrameDecoder

FixedLengthFrameDecoder fixed length decoder that can decode messages in accordance with a specified length.
Here again, simply put the code (with respect DelimiterBasedFrameDecoder) changes.

1. Add the server decoder

              socketChannel.pipeline()
                  //添加编码器
                  .addLast(new FixedLengthFrameDecoder(20))

Length 20 provided here

2. Client

No change, no longer posted

Results are as follows:

有客户端消息：helllo,0_$helllo,1_$
有客户端消息：helllo,2_$helllo,3_$
有客户端消息：helllo,4_$helllo,5_$
有客户端消息：helllo,6_$helllo,7_$
有客户端消息：helllo,8_$helllo,9_$
有客户端消息：helllo,10_$helllo,11
有客户端消息：_$helllo,12_$helllo,
有客户端消息：13_$helllo,14_$helll
有客户端消息：o,15_$helllo,16_$hel
有客户端消息：llo,17_$helllo,18_$h

We can see, the resulting string length is 20, and the last one due to the length of less than 20, it is not showing up.

VI Summary

Read on their own on top of several decoder source code, we can see that these decoders are inherited ByteToMessageDecoder, the original acquired ByteBuf been processed, that is to say, we can also implement your own decoder through inheritance ByteToMessageDecoder .

Netty-4 TCP packet unpacking sticky problem

A, TCP stick package / unpacking problem

Second, the reason for TCP stick package / unpacking issues arising

Third, the solution strategy stick package TCP / unpacking problem

Four, TCP stick package / unpacking problems demo

(A) server

(B) Client

Five, Netty to stick package / unpacking problem solving

(A) LineBasedFrameDecoder

1. The server code

2. The client code

3. Check results

（二）DelimiterBasedFrameDecoder

1. Add the server decoder

2. When sending a message, to bring the end of $ _

（三）FixedLengthFrameDecoder

1. Add the server decoder

2. Client

VI Summary

Guess you like