Java IO flow (4) Netty theory [Model | Core Components]

Overview

Netty is a Java open source framework provided by JBOSS. Independent projects are available from Github.
Netty is an asynchronous, event-driven network application framework used to quickly develop maintainable, high-performance network servers and clients ( excerpt from official website )
Netty's so-called asynchronous is for users to use Channel to perform IO operations, and ChannelFuture will be returned immediately. But the IO operation task is submitted to Netty's NIO bottom layer for processing, so we say that Netty's asynchronous event driver and Netty's bottom layer based on NIO (synchronous non-blocking) are not inconsistent.
Netty is mainly aimed at high-concurrency applications on the Client side under the TCP transmission protocol; it essentially belongs to the NIO framework and is suitable for various application scenarios related to server communication.
- The RPC framework of Alibaba's distributed service framework Dubbo uses the Dubbo protocol, and the Dubbo protocol uses Netty as the basic communication component by default to achieve internal communication between nodes.
- Hadoop's RPC framework in the field of big data uses Netty by default for cross-node communication.
Netty official website version recommendation: The current stable version is Netty4.x
Netty supports the following protocols
- TCP/UDP: Netty provides a NIO-based TCP and UDP programming framework that can be used to build high-performance, high-availability network applications.
- HTTP/HTTPS : Netty provides an HTTP/HTTPS programming framework that can be used to develop web servers and clients.
- WebSocket : Netty provides a WebSocket programming framework that can be used to implement two-way communication applications, such as chat rooms, etc.
- SPDY/HTTP2: Netty provides SPDY and HTTP2 programming frameworks that can be used to implement efficient web applications.
- MQTT/CoAP: Netty provides MQTT and CoAP programming frameworks that can be used to build IoT applications

Netty model

Netty has made certain improvements based on the master-slave Reactor multi-thread model. There are multiple Reactors in the master-slave Reactor multi-thread model.

Working principle execution process

Netty abstracts two sets of thread pools, BossGroup and WorkerGroup, both of which belong to NioEventLoopGroup. BossGroup is responsible for receiving client connections, and WorkerGroup is responsible for network reading and writing.
- NioEventLoopGroup belongs to the event loop group. This group contains multiple event loops. Each event loop is a NioEventLoop
- NioEventLoop represents a thread that continuously loops to perform processing tasks. Each NioEventLoop has a selector that is used to monitor the network communication of the socket bound to it.
Each NioEventLoop execution process under BossGroup
- Poll accept event
- Process the accept event, establish a connection with the client to generate NIOSocketChannel, and register it to the selector on NIOEventLoop under the corresponding worker
- Process task queue tasks, namely runAllTasks
Each NioEventLoop execution process under WorkerGroup
- Polling for read and write events
- Handle read and write I/O events in the corresponding NIOSocketChannel
- Process task queue tasks, namely runAllTasks
When each Worker NioEventLoop processes business, it will be handed over to the pipeLine pipeline for processing. The pipeline contains channels, that is, the corresponding channel channels can be obtained through the pipeline. Many Handler processors for processing business are maintained in the pipeline.

TaskQueue task queue

Each NioEventLoop under BossGroup and WorkerGroup maintains a TaskQueue, which can be used to process time-consuming tasks asynchronously. Tasks in TaskQueue have three typical usage scenarios.

The user program customizes a common task, which will be submitted to TaskQueue.

public class NettyServerHandler extends ChannelInboundHandlerAdapter {
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { //读取客户端发送的数据
        //ctx:上下文对象,包含管道pipeline,通道等信息
        //用户程序自定义普通任务
        ctx.channel().eventLoop().execute(new Runnable() {
            @Override
            public void run() {
                try {
                    Thread.sleep(10 * 1000); //模拟耗时长的任务
                    ctx.writeAndFlush(Unpooled.copiedBuffer("Hello, Client！", CharsetUtil.UTF_8)); //将数据写到缓存并刷新
                } catch (Exception e) {
                    System.out.println("Exception Info: " + e.getMessage());
                }
            }
        });
    }
}

User-defined scheduled tasks are submitted to scheduleTaskQueue

        ctx.channel().eventLoop().schedule(() -> {
            try {
                Thread.sleep(5 * 1000);
                ctx.writeAndFlush(Unpooled.copiedBuffer("Hello, Client！", CharsetUtil.UTF_8)); 
            } catch (Exception e) {
                System.out.println("Exception Info: " + e.getMessage());
            }
        },5, TimeUnit.SECONDS);

Non-current Reactor thread calls various methods of Channel ( often used in push task scenarios )

Asynchronous model

Asynchronous is relative to synchronous, which means that when an asynchronous procedure call is issued, the caller cannot get the result immediately. After the component that actually handles the call is completed, it will notify the caller through status, notifications and callbacks.
I/O operations in Netty are asynchronous, including Bind, Write, Connect, etc., which will return a ChannelFuture
Netty's asynchronous model is built on future and callback. Callback is a callback. The core idea of Future is to immediately return a Future ( representing the asynchronous execution result) when processing some time-consuming business, in which the task execution status is detected through the listening method. ), monitor the business processing process through Future and finally obtain the I/O operation results (i.e. Future-Listener mechanism )
- Future-Listener mechanism
  - When the Future object is just created, it is in a non-completed state. The caller can obtain the operation execution status through the returned ChannelFuture and register a listening function to perform the completed operation.
  - Common operating methods
    - isDone: Determine whether the current operation is completed
    - isSuccess: Determine whether the completed current operation is successful or not
    - getCause: Determine the reason why the current operation failed
    - isCancelled: Determines whether the completed current operation has been cancelled.
    - addListener: used to register a listener. When the current operation is completed, the specified listener will be notified (if the Future object is completed, the specified listener will be notified)
  - Future-Listener example
```
ChannelFuture cf = bootstrap.bind(6668).sync(); //绑定指定端口并同步处理
cf.addListener(new ChannelFutureListener() { //为端口添加监听
	@Override
	public void operationComplete(ChannelFuture channelFuture) throws Exception {
		if (cf.isSuccess()){
			System.out.println("监听端口成功!");
		} else{
			System.out.println("监听端口失败!");
		}
	}
});
```

core components

Netty can be divided into network communication layer, event scheduling layer and service orchestration layer according to the different responsibilities of each component.

Network communication layer core components

BootStrap|ServerBootStrap

BootStrap is the client startup class in Netty, responsible for the client startup and connecting to the Netty server; ServerBootStrap is the server startup boot class, responsible for the server startup and listening port; common methods include the following

group: The server is used to set BossGroup and WorkerGroup, while for the client, it is used to set an EventGroup.
channel: used to set the channel type on the server side, such as the server side is set to NioServerSocketChannel type
option: used to add configuration to ServerChannel, such as setting the number of thread queue connections
childOption: used to add configuration to the receiving channel, such as setting the active connection status
handler: This handler acts on BossGroup
childHandler: This handler acts on WorkerGroup and sets business processing classes, including custom handlers.
bind: used on the server side to set the port number exposed to the outside world
connect: used on the client to set the IP and port information to connect to the server.

Channel

Tip: NioSocketChannel and NioServerSocketChannel correspond to the client and server channels respectively. The direct parent classes of the two are inconsistent, so the functions provided to the outside are also different. For example, when a read event occurs, the main logic of NioServerSocketChannel is to establish a new connection, while NioSocketChannel reads the transmitted bytes for business processing.

Channel in Netty is the carrier for completing a series of network communications
Channel can be regarded as a Socket in network programming. It provides an API for performing network asynchronous IO operations, including read|write|bind|connect. The registered listener in ChannelFuture notifies the call result of the success or failure of the IO operation through callbacks, which greatly reduces the cost. Removes the complexity of using the Socket class directly
- Future|ChannelFuture: All IO operations in Netty are asynchronous. You cannot know immediately whether the message is processed correctly, but you can register listening events through Future and ChannelFuture. When the operation succeeds or fails, the registered listening events are automatically triggered. Commonly used methods are as follows
  - channel: returns the channel of the currently ongoing IO operation
  - sync: Wait for the asynchronous operation to complete
Through Channel, you can obtain the status and configuration parameters of the current network connection channel, etc.
Different protocols and different blocking types have different Channel types corresponding to them. Commonly used Channel types are as follows:
- NioSocketChannel: Asynchronous client TCP Socket connection
- NioServerSocketChannel: Asynchronous server-side TCP Socket connection
- NioDatagramChannel: Asynchronous UDP connection

Core components of event scheduling layer

EventLoopGroup

It is a thread pool that is responsible for receiving I/O requests and allocating threads to execute and process the requests. The threads in the bossGroup in the Netty server are used to handle the establishment of new connections. When the connection is established, it is distributed to the workerGroup. Each thread in the workerGroup Will be bound to the only client Channel connection and handle read and write events on the Channel

EventLoop

It is equivalent to a thread in the thread pool

Core components of service orchestration layer

ChannelPipeline

ChannelPipeline is a collection of Handlers, which is responsible for processing and intercepting inbound or outbound events and operations, that is, used to process inbound and outbound operations of Channel.
Each Channel in Netty has a corresponding ChannelPipeline, and the two can obtain each other; and Pipeline maintains a two-way linked list composed of channelHandlerContext , and channelHandlerContext is associated with a ChannelHandler
- If the events are from the client to the server, then we call these events outbound, and the outbound messages will be converted into bytes through the encoder encoding
- On the contrary, the server sends a response message to the client, which is called inbound. The inbound message will be converted from bytes to another format (such as a Java object) by the decoder;
- Both outbound and inbound data will be processed through a series of Handlers in the pipeline
- Note: The linked list position of the codec handler in the pipeline must be before the business handler, otherwise it cannot be called correctly.
When a specified event occurs in the Channel, the event will be propagated along the doubly linked list in the ChannelPipeline and the specified method in each ChannelHandler will be called to complete the corresponding business processing.
Commonly used methods of ChannelPipeline are as follows
- addFirst: Add the business processing class Handler to the first position in the linked list
- addLast: Add the business processing class Handler to the last position in the linked list

Tip: Netty provides users with extension points for custom business logic through the data structure of ChannelPipeline. Users only need to add a ChannelHandler that handles the corresponding business logic to ChannelPipeline. When a specified event occurs, the corresponding The method will perform a callback to implement business processing.

ChannelHandler

**ChannelInboundHandler&&ChannelOutboundHandler are used to handle read&& write events respectively.**

ChannelHandler is the core class of business processing in Netty. When an IO event occurs, the event will be propagated in the ChannelPipeline, and the specific ChannelHandler will be called in turn for business processing.

ChannInboundHandler

A series of callback methods are defined in ChannelInboundHandler. Users can implement this interface and rewrite the corresponding methods to customize business logic
- channelActive: When Channel calls bind() to complete port binding, the channelActive() method will perform a callback
- channelRegistered: A callback will be performed when the Channel is registered with the corresponding Selector.
- channelInactive(ChannelHandlerContext ctx) The channel is inactive (closed). When the close method is called, this event will be triggered, and then the channelUnregistered event will be triggered.
- channelUnregistered(ChannelHandlerContext ctx) is called when the channel is unregistered to the Selector . It is usually triggered when the channel is closed. The channelInactive event is triggered first, and then the channelUnregistered event is triggered.
- channelRead0
  - Server channelRead0
    - When the server Channel is bound to the Selector, it listens to the Accept event. When the client has a new connection, it calls back the channelRead() method and completes the access of the new connection.
  - clientchannelRead0
    - After the server processes the Accept event, it will generate a Channel to communicate with the client. The Channel will also be registered to the corresponding Selector and listen for the read event. When the client sends data to the Channel, the read event will be triggered, calling channelRead() method
- exceptionCaught: If an exception occurs during the processing of each callback method in the current ChannelHandler, the method will be called back.

ChannelHandlerContext

ChannelHandlerContext mainly saves information about Channel channels and ChannelHandler channel handlers. The corresponding relationship between them is as follows
- Correspondence between ChannelPipeline and ChannelHandlerContext: 1:n
- Correspondence between ChannelHandlerContext and ChannelHandler: 1:1
ChannelHandlerContext common methods are as follows
- channel(): Get the current channel
- pipeline(): Get the current pipeline
- handler(): Get the current handler processor
- close(): close the current channel
- writeAndFlush(): writes data into the ChannelPipeline pipeline

other

Unpooled class

Unpooled is a tool class provided by Netty that specifically operates buffers.
copiedBuffer: used to set data and character encoding and return a ByteBuf object
- ByteBuf: The buffer provided in Netty, the bottom layer is an array
- ByteBuf does not need to call the flip method to switch modes. The underlying layer divides byteBuf into three areas through readerIndex|writerIndex|capactiy.
  - 0-readerIndex: indicates the area that has been read
  - readerIndex-writerIndex: readable area
  - writerIndex-capactiy: writable area

Netty’s encoding and decoding mechanism

The encoding and decoding provided by Netty include the following

StringEncoder: Encode string data
ObjectEncoder: Encode Java objects
StringDecoder: Decode string data
ObjectDecoder: Decode Java objects

There is a problem

Netty's bottom layer for decoding and encoding Java objects uses Java serialization technology, but its efficiency is not high and it cannot cross languages. The serialization volume is more than 5 times that of binary encoding, and the performance is too low.

solution

You can use Google's Protobuf, which is a lightweight and efficient structured data storage format for structured data serialization. It is very suitable for data storage or RPC data exchange formats, and it supports cross-language

common problem

NIO already exists, why develop the Netty framework?

NIO class libraries and APIs are complex and troublesome to use
NIO is a single-Ractor single-thread mode. You need to be familiar with threading and network programming to write high-quality NIO programs.
The development workload and difficulty are large, such as problems such as disconnection and reconnection, cache failure, network congestion, etc.
The Epoll bug in NIO itself causes empty polling and causes the CPU to become full.

Netty solves the above problems. It is packaged and optimized based on NIO and has the following advantages:

Elegant Design: Uniform for all transmission types
High performance, higher throughput, lower latency, and reduced resource consumption
Security: Full SSL/TLS support
Active community and short version iteration cycle