Foreword

This article focuses on the integration of Netty is SpringBoot Protobuf related content and the use of data transmission. Protobuf will simply introduce usage, as in the previous article Netty has simply introduced, there is no longer too elaborate.

protobuf

Introduction

protocolbuffer (hereinafter referred to as PB) is a format of data exchange google, it is independent of language, platform-independent. google provides multiple-language realization: java, c #, c ++, go and python, every implementation contains a compiler and library files for the appropriate language.

Because it is a binary format than using xml exchange data much faster. It can be used for data exchange in the data communication between heterogeneous environments or distributed applications. As an excellent efficiency and compatibility are binary data transmission format may be used in many fields such as a network transmission, profile, data storage.

Official Address:

https://github.com/google/protobuf

use

Use here only introduce Java-related use. First we need to create a proto file, we need to define the file transfer in this file.

For example, we need to define a user's information, the field contains the main number, name, age.

Then format the protobuf file is as follows:

Note: As used herein, is proto3, associated comments I have written, there was never too much tells the story. It should be noted that it is important proto file and generate the Java file name can not be the same!

syntax = "proto3";
// 生成的包名
option java_package="com.pancm.protobuf";
//生成的java名
option java_outer_classname = "UserInfo";

message UserMsg {

     // ID
     int32 id = 1;

    // 姓名
     string name = 2;

    // 年龄
      int32 age = 3;

     // 状态
     int32 state = 4;
}

Once you've created the file, we put the file and protoc.exe (software generated Java files) protobuf put under the E disk catalog file folder, and then to enter the dos interface that directory: protoc.exe --java_out = absolute path name of the file.

E.g:

protoc.exe --java_out=E:\protobuf User.proto

Once you have entered, press Enter to see the good in the same directory has been generated Java files, then the file into the project file in the specified path.

Note: protobuf generated files and software testing protobuf file I also integrated into the project, and can be directly obtained.

After the Java file is generated good, let's look at how to use.

Here I'll just paste the code, and write comments in the code, it should be easier to understand something.

Code Example:

// 按照定义的数据结构，创建一个对象
        UserInfo.UserMsg.Builder userInfo = UserInfo.UserMsg.newBuilder();
        userInfo.setId(1);
        userInfo.setName("xuwujing");
        userInfo.setAge(18);
        UserInfo.UserMsg userMsg = userInfo.build();
        // 将数据写到输出流
        ByteArrayOutputStream output = new ByteArrayOutputStream();
        userMsg.writeTo(output);
        // 将数据序列化后发送
        byte[] byteArray = output.toByteArray();
        // 接收到流并读取
        ByteArrayInputStream input = new ByteArrayInputStream(byteArray);
        // 反序列化
        UserInfo.UserMsg userInfo2 = UserInfo.UserMsg.parseFrom(input);
        System.out.println("id:" + userInfo2.getId());
        System.out.println("name:" + userInfo2.getName());
        System.out.println("age:" + userInfo2.getAge());

NOTE: described herein, because binary protobuf is transmitted, it is necessary to pay attention to the corresponding encoding. There is also need to look at using protobuf maximum length of one byte transfer.

Output:

id:1
name:xuwujing
age:18
SpringBoot整合Netty

Note: If you want direct access to the project you can jump directly to the bottom, through the link to download the project code.

Development Readiness
Environmental requirements
JDK：1.8
Netty: 4.0 or more (excluding 5)
Protobuf: 3.0 or above

If Netty unfamiliar, you can look at these articles. Great God - please ignore. ~

https://blog.csdn.net/column/details/17640.html

First of all relevant dependent or Maven:

<properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <java.version>1.8</java.version>
        <netty.version>4.1.22.Final</netty.version>
        <protobuf.version>3.5.1</protobuf.version>
        <springboot>1.5.9.RELEASE</springboot>
        <fastjson>1.2.41</fastjson>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>


    <dependencies>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
            <version>${springboot}</version>
        </dependency>


        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <version>${springboot}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <version>${springboot}</version>
            <optional>true</optional>
        </dependency>


        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-all</artifactId>
            <version>${netty.version}</version>
        </dependency>

        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>${protobuf.version}</version>
        </dependency>

        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>${fastjson}</version>
        </dependency>


    <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
</dependencies>

After adding the appropriate maven dependencies profile piece being nothing can be added, because the port being on a listener only.

Coding

Code module is divided into server and client.

The main achievement of business logic:

After the server starts successfully, the client starts successfully, then the server sends a message protobuf format to the client, and the client to give the appropriate response. After the client and the server connection is successful, the client will send a period of time each heartbeat command to the server, the client tells the server has been saved, if the client does not send a message, the server will shut down at a specified time and the client's connection. When the client can not connect to the server, you will periodically attempt to reconnect, only to reconnection!

Server

The first is to write server-side classes start, the corresponding comments in the code written very detailed, there is no longer much tells the story. But note that in the previous article I wrote Netty, is to start the server directly through the main method, and therefore is a direct new object. And after SpringBoot and integration, we will need to Netty springBoot to manage it, so here it is with the corresponding comments.

code show as below:

@Service("nettyServer")
public class NettyServer {
    private static final int port = 9876; // 设置服务端端口
    private static EventLoopGroup boss = new NioEventLoopGroup(); // 通过nio方式来接收连接和处理连接
    private static EventLoopGroup work = new NioEventLoopGroup(); // 通过nio方式来接收连接和处理连接
    private static ServerBootstrap b = new ServerBootstrap();

    @Autowired
    private NettyServerFilter nettyServerFilter;


    public void run() {
        try {
            b.group(boss, work);
            b.channel(NioServerSocketChannel.class);
            b.childHandler(nettyServerFilter); // 设置过滤器
            // 服务器绑定端口监听
            ChannelFuture f = b.bind(port).sync();
            System.out.println("服务端启动成功,端口是:" + port);
            // 监听服务器关闭监听
            f.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            // 关闭EventLoopGroup，释放掉所有资源包括创建的线程
            work.shutdownGracefully();
            boss.shutdownGracefully();
        }
    }
}

After the completion of the preparation of the server main class, let us set the corresponding filters.

It should Inheritance in Netty ChannelInitializer class, then the override initChannel method, add the appropriate settings, such as the heartbeat timeout setting, the transport protocol is provided, and the corresponding service implementation class.

code show as below:

    @Component
     public class NettyServerFilter extends ChannelInitializer<SocketChannel> {

    @Autowired
    private NettyServerHandler nettyServerHandler;

     @Override
     protected void initChannel(SocketChannel ch) throws Exception {
         ChannelPipeline ph = ch.pipeline();

         //入参说明: 读超时时间、写超时时间、所有类型的超时时间、时间格式
         ph.addLast(new IdleStateHandler(5, 0, 0, TimeUnit.SECONDS));
         // 解码和编码，应和客户端一致
         //传输的协议 Protobuf
         ph.addLast(new ProtobufVarint32FrameDecoder());
         ph.addLast(new ProtobufDecoder(UserMsg.getDefaultInstance()));
         ph.addLast(new ProtobufVarint32LengthFieldPrepender());
         ph.addLast(new ProtobufEncoder());

         //业务逻辑实现类
         ph.addLast("nettyServerHandler", nettyServerHandler);
       }
     }

After the code for the service-related settings are finished, let us write the main business code.

Netty write code using the business layer, we need to inherit ChannelInboundHandlerAdapter or SimpleChannelInboundHandler class, two of them distinguished by the way, right here.

After inheriting SimpleChannelInboundHandler class, will automatically release the resource data out Bytebuffer occupied after receiving the data. And extending this class need to specify the data format.

The inheritance ChannelInboundHandlerAdapter does not automatically release, call ReferenceCountUtil.release manual () and other methods of release. This class does not need to inherit a specified data format. So here, personally recommend the service side inherit ChannelInboundHandlerAdapter, manually release, prevent unprocessed data completely automatically released. And the server may be connected to a plurality of clients, and each client request does not match the data format, then it can be treated accordingly.

The client can inherit SimpleChannelInboundHandler class according to the situation. Good benefits are specified directly transfer data format, you do not need to convert the format.

code show as below:

@Service("nettyServerHandler")
public class NettyServerHandler extends ChannelInboundHandlerAdapter {

    /** 空闲次数 */
    private int idle_count = 1;
    /** 发送次数 */
    private int count = 1;


    /**
     * 建立连接时，发送一条消息
     */
    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        System.out.println("连接的客户端地址:" + ctx.channel().remoteAddress());
        UserInfo.UserMsg userMsg = UserInfo.UserMsg.newBuilder().setId(1).setAge(18).setName("xuwujing").setState(0)
                .build();
        ctx.writeAndFlush(userMsg);
        super.channelActive(ctx);
    }

    /**
     * 超时处理 如果5秒没有接受客户端的心跳，就触发; 如果超过两次，则直接关闭;
     */
    @Override
    public void userEventTriggered(ChannelHandlerContext ctx, Object obj) throws Exception {
        if (obj instanceof IdleStateEvent) {
            IdleStateEvent event = (IdleStateEvent) obj;
            if (IdleState.READER_IDLE.equals(event.state())) { // 如果读通道处于空闲状态，说明没有接收到心跳命令
                System.out.println("已经5秒没有接收到客户端的信息了");
                if (idle_count > 1) {
                    System.out.println("关闭这个不活跃的channel");
                    ctx.channel().close();
                }
                idle_count++;
            }
        } else {
            super.userEventTriggered(ctx, obj);
        }
    }

    /**
     * 业务逻辑处理
     */
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        System.out.println("第" + count + "次" + ",服务端接受的消息:" + msg);
        try {
            // 如果是protobuf类型的数据
          if (msg instanceof UserMsg) {
                UserInfo.UserMsg userState = (UserInfo.UserMsg) msg;
                if (userState.getState() == 1) {
                    System.out.println("客户端业务处理成功!");
                } else if(userState.getState() == 2){
                    System.out.println("接受到客户端发送的心跳!");
                }else{
                    System.out.println("未知命令!");
                }
            } else {
                System.out.println("未知数据!" + msg);
                return;
            }
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            ReferenceCountUtil.release(msg);
        }
        count++;
    }

    /**
     * 异常处理
     */
    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        cause.printStackTrace();
        ctx.close();
    }
}

There is a server-side start classes before starting directly by the main method, but changed here were initiated by springBoot, not very different.

code show as below:

@SpringBootApplication
public class NettyServerApp {

    public static void main(String[] args) {
        // 启动嵌入式的 Tomcat 并初始化 Spring 环境及其各 Spring 组件
        ApplicationContext context = SpringApplication.run(NettyServerApp.class, args);
        NettyServer nettyServer = context.getBean(NettyServer.class);
        nettyServer.run();
    }

}

Here's the code to the appropriate server on the preparation is completed.

Client

In many parts of the client and server side code are similar, I will not elaborate too much, some major out under different codes simply tells.

The first is the main class of the client, and the server is substantially similar, i.e. more listening port and a listener (listening to whether the server is disconnected, for reconnection).

Main code logic to achieve the following:

    public void doConnect(Bootstrap bootstrap, EventLoopGroup eventLoopGroup) {
        ChannelFuture f = null;
        try {
            if (bootstrap != null) {
                bootstrap.group(eventLoopGroup);
                bootstrap.channel(NioSocketChannel.class);
                bootstrap.option(ChannelOption.SO_KEEPALIVE, true);
                bootstrap.handler(nettyClientFilter);
                bootstrap.remoteAddress(host, port);
                f = bootstrap.connect().addListener((ChannelFuture futureListener) -> {
                    final EventLoop eventLoop = futureListener.channel().eventLoop();
                    if (!futureListener.isSuccess()) {
                        System.out.println("与服务端断开连接!在10s之后准备尝试重连!");
                        eventLoop.schedule(() -> doConnect(new Bootstrap(), eventLoop), 10, TimeUnit.SECONDS);
                    }
                });
                if(initFalg){
                    System.out.println("Netty客户端启动成功!");
                    initFalg=false;
                }
                // 阻塞
                f.channel().closeFuture().sync();
            }
        } catch (Exception e) {
            System.out.println("客户端连接失败!"+e.getMessage());
        }
    }

Note: this is achieved by the listener is JDK1.8 wording.

Filter its basic client and server have been this. But note that the transport protocol, encoding and decoding should be the same, there should be less than the time to read and write the heartbeat time set by the server.

Code changes as follows:

   ChannelPipeline ph = ch.pipeline();
        /*
         * 解码和编码，应和服务端一致
         * */
        //入参说明: 读超时时间、写超时时间、所有类型的超时时间、时间格式
        ph.addLast(new IdleStateHandler(0, 4, 0, TimeUnit.SECONDS));

Business logic code for the client.

Several major logic is implemented on time and resolved to send heartbeat data format of the service protobuf sent.

Here more than one server comment, the comment Sharable primarily for multiple channel handler can be more securely share, it is to ensure thread safety.

Nonsense is not to say, the code is as follows:

    @Service("nettyClientHandler")
    @ChannelHandler.Sharable
    public class NettyClientHandler extends ChannelInboundHandlerAdapter {
    @Autowired
    private NettyClient nettyClient;

    /** 循环次数 */
    private int fcount = 1;

    /**
     * 建立连接时
     */
    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        System.out.println("建立连接时：" + new Date());
        ctx.fireChannelActive();
    }

    /**
     * 关闭连接时
     */
    @Override
    public void channelInactive(ChannelHandlerContext ctx) throws Exception {
        System.out.println("关闭连接时：" + new Date());
        final EventLoop eventLoop = ctx.channel().eventLoop();
        nettyClient.doConnect(new Bootstrap(), eventLoop);
        super.channelInactive(ctx);
    }

    /**
     * 心跳请求处理 每4秒发送一次心跳请求;
     *
     */
    @Override
    public void userEventTriggered(ChannelHandlerContext ctx, Object obj) throws Exception {
        System.out.println("循环请求的时间：" + new Date() + "，次数" + fcount);
        if (obj instanceof IdleStateEvent) {
            IdleStateEvent event = (IdleStateEvent) obj;
            if (IdleState.WRITER_IDLE.equals(event.state())) { // 如果写通道处于空闲状态,就发送心跳命令
                UserMsg.Builder userState = UserMsg.newBuilder().setState(2);
                ctx.channel().writeAndFlush(userState);
                fcount++;
            }
        }
    }

    /**
     * 业务逻辑处理
     */
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        // 如果不是protobuf类型的数据
        if (!(msg instanceof UserMsg)) {
            System.out.println("未知数据!" + msg);
            return;
        }
        try {

            // 得到protobuf的数据
            UserInfo.UserMsg userMsg = (UserInfo.UserMsg) msg;
            // 进行相应的业务处理。。。
            // 这里就从简了，只是打印而已
            System.out.println(
                    "客户端接受到的用户信息。编号:" + userMsg.getId() + ",姓名:" + userMsg.getName() + ",年龄:" + userMsg.getAge());

            // 这里返回一个已经接受到数据的状态
            UserMsg.Builder userState = UserMsg.newBuilder().setState(1);
            ctx.writeAndFlush(userState);
            System.out.println("成功发送给服务端!");
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            ReferenceCountUtil.release(msg);
        }
     }
    }

So here's the client code is also written finished.

function test

First, start the server, and then start the client.

If we look at the results as mentioned above.

Server output:

服务端启动成功,端口是:9876
连接的客户端地址:/127.0.0.1:53319
第1次,服务端接受的消息:state: 1

客户端业务处理成功!
第2次,服务端接受的消息:state: 2

接受到客户端发送的心跳!
第3次,服务端接受的消息:state: 2

接受到客户端发送的心跳!
第4次,服务端接受的消息:state: 2

接受到客户端发送的心跳!

Client input results:

Netty客户端启动成功!
建立连接时：Mon Jul 16 23:31:58 CST 2018
客户端接受到的用户信息。编号:1,姓名:xuwujing,年龄:18
成功发送给服务端!
循环请求的时间：Mon Jul 16 23:32:02 CST 2018，次数1
循环请求的时间：Mon Jul 16 23:32:06 CST 2018，次数2
循环请求的时间：Mon Jul 16 23:32:10 CST 2018，次数3
循环请求的时间：Mon Jul 16 23:32:14 CST 2018，次数4

As can be seen by the above-mentioned print information.

Next we look at whether the client can achieve reconnect.

First start the client, and then start the server.

Client input results:

Netty客户端启动成功!
与服务端断开连接!在10s之后准备尝试重连!
客户端连接失败!AbstractChannel$CloseFuture@1fbaa3ac(incomplete)
建立连接时：Mon Jul 16 23:41:33 CST 2018
客户端接受到的用户信息。编号:1,姓名:xuwujing,年龄:18
成功发送给服务端!
循环请求的时间：Mon Jul 16 23:41:38 CST 2018，次数1
循环请求的时间：Mon Jul 16 23:41:42 CST 2018，次数2
循环请求的时间：Mon Jul 16 23:41:46 CST 2018，次数3

Server output:

服务端启动成功,端口是:9876
连接的客户端地址:/127.0.0.1:53492
第1次,服务端接受的消息:state: 1

客户端业务处理成功!
第2次,服务端接受的消息:state: 2

接受到客户端发送的心跳!
第3次,服务端接受的消息:state: 2

接受到客户端发送的心跳!
第4次,服务端接受的消息:state: 2

The results are also said above!