Article directory
foreword
The above " BIO, NIO, IO multiplexing model detailed introduction & Java NIO network programming" introduced several IO models and Java NIO, and learned which model to use in network programming can improve system performance and efficiency. Even though Java NIO can help developers write and maintain network applications, due to its complexity and bugs, many powerful and popular network programming frameworks have been born, such as Netty, Undertow, and Grizzly. In normal development, everyone tends to Choose these frameworks for development, and when we learn and understand the underlying principles of network programming, using Java NIO can provide a more direct and in-depth understanding of the underlying operations. This article briefly introduces Netty and implements an HTTP server through Netty.
Introduction to Netty
Netty is a Java-based high-performance network application framework. It encapsulates the complexity of Java NIO and provides a simple and powerful network programming API, enabling developers to more easily build high-performance, scalable network applications. Therefore, it is necessary to understand Java NIO before learning Netty, otherwise it will be foggy.
How powerful can it be using Netty? Including but not limited to the following points:
- High-performance IO processing : If you develop a mature application based on Java NIO, you must pay great attention to issues such as ByteBuffer memory leaks, Channel registration connections, and thread management. Netty, on the other hand, can better handle connection management, threading models, and memory management to provide higher throughput and lower latency.
- Powerful function extension : If we write an HTTP protocol or Websocket protocol based on Java NIO, then we need to consider the format, codec, and Netty provides a wealth of extension points, such as codecs, processors, and interceptors. Development Personnel can build protocols such as HTTP, WebSocket, TCP, and UDP through different configurations, and can also easily add codecs to implement custom protocols.
- Reliability and stability : Netty has good fault tolerance and stability, can handle various network faults and abnormal situations, and provides a variety of fault tolerance and recovery mechanisms, such as disconnection reconnection and heartbeat mechanisms.
In general, using Netty can focus more on business logic when developing web applications. The figure below shows the functions supported by Netty
Netty development history
In order to learn more about Netty, here is a brief introduction to Netty's past and present.
- 2004: The Jboss Netty project, the predecessor of Netty, was launched within JBoss with the goal of providing an extensible and easy-to-use network programming framework.
- 2008: The Netty project was open-sourced within JBoss, and the first public version, Netty 3.0, was released. This version mainly supports TCP and HTTP protocols.
- 2011: Netty 3.2 was released, introducing more features and improvements, including better performance and a more flexible API design.
- 2012: Netty 4.0 is released, which is a major milestone version. In this version, Netty has been completely refactored and optimized, introducing a new API design and more advanced features.
- 2013: Netty 4.0 gained wide recognition and adoption, and became the preferred network programming framework for many large Internet companies and projects. At the end of the same year, 5.0.0.Alpha1 was released, with the goal of improving and optimizing Netty 4.
- 2015: Netty 5 encountered some challenges and technical difficulties during the development process. It was decided to suspend the development of Netty 5 and shift the focus to the improvement and maintenance of Netty 4.
- 2016: Netty 4.1 was released, which was further improved and optimized based on version 4.0, providing better performance and more functions.
At present, many well-known projects have chosen Netty as the basis of network communication, such as the well-known RPC framework Dubbo, gRPC, message queue Kafka, RocketMQ, search engine Elasticsearch, etc., so when learning to understand these projects, Netty will be a bonus item.
Netty Core Components
Because Netty is encapsulated based on Java NIO, it is more abstract. To use Netty for development, you must be familiar with several core components in Netty. The following introduces them one by one:
- Channel (channel) : Like the SocketChannel in Java NIO, data can be read and written.
- EventLoop (event loop) : EventLoop is Netty's event processing mechanism, which is responsible for processing various events, including connection establishment and closure, data reading and writing, etc. It can be understood that the Selector in Java NIO listens to socket events, but Netty is multi-threaded, which is reflected in the following code.
- ChannelHandler (channel processor) : A component used to process events and data in the Channel, such as data encoding and decoding, business logic processing, etc. Netty provides many built-in ChannelHandlers for handling network connections and I/O operations. The following are some commonly used ChannelHandlers:
- ChannelInboundHandler : Used to handle inbound events, such as connection establishment, data reading, etc.
- ChannelOutboundHandler : Used to handle outbound events, such as data writing, connection closing, etc.
- SimpleChannelInboundHandler : inherited from ChannelInboundHandler, which simplifies the logic of message processing.
- SimpleChannelOutboundHandler : inherited from ChannelOutboundHandler, which simplifies the logic of message sending.
- HttpServerCodec : It is responsible for handling the encoding and decoding of HTTP requests and responses.
- HttpObjectAggregator : Combine multiple parts of the HTTP request into a complete FullHttpRequest.
- WebSocketServerProtocolHandler : handles the handshake and frame encoding and decoding of the WebSocket protocol.
- ChannelPipeline (channel pipeline) : ChannelPipeline is an event processor chain used to manage and execute ChannelHandler. Each Channel has a corresponding Pipeline. When data enters or leaves the Channel, it will be processed by a series of ChannelHandlers in the Pipeline.
- ByteBuf (byte buffer) : ByteBuf is a byte container in Netty for efficient storage and transmission of byte data. Compared with Java NIO's ByteBuffer, ByteBuf provides a more flexible API and more efficient memory management.
- Future (asynchronous operation result) : The operations in Netty are all asynchronous, and Future is used to obtain the status and results of the operation.
- Bootstrap (guidance class) : Bootstrap is the class that starts the client, responsible for configuring and starting the related components of the client.
- ServerBootstrap (server bootstrap class) : ServerBootstrap is a class that creates and starts a server, and is used to configure and manage various components of the server.
Implement an HTTP server
Let's take the HTTP protocol as an example, and use Netty to write an HTTP server. Before that, let's use the NIOServer in the previous article to receive the browser's request, which is probably like this:
GET / HTTP/1.1
Host: localhost:8888
Connection: keep-alive
sec-ch-ua: "Google Chrome";v="111", "Not(A:Brand";v="8", "Chromium";v="111"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Accept-Encoding: gzip, deflate, br
Accept-Language: zh-CN,zh;q=0.9
You can see that the message data of an HTTP request has been received, including the request line, request header and request body. At this time, you can also see that the response returned by the browser is: , the sent response is invalid ERR_INVALID_HTTP_RESPONSE. Why? This is because the output format HTTP protocol in NIOServer does not recognize.
So if you use Java NIO to implement an HTTP server, you need to deal with a lot of work, but if you use Netty to implement an HTTP server is very simple, just add the code:
import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.http.*;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.logging.LoggingHandler;
import io.netty.util.CharsetUtil;
import java.util.List;
import java.util.Map;
public class HttpServer {
private final int port;
public HttpServer(int port) {
this.port = port;
}
public void start() throws Exception {
EventLoopGroup bossGroup = new NioEventLoopGroup(1);
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
ServerBootstrap b = new ServerBootstrap();
b.group(bossGroup, workerGroup).channel(NioServerSocketChannel.class).handler(new LoggingHandler(LogLevel.INFO)).childHandler(new ChannelInitializer<SocketChannel>() {
@Override
public void initChannel(SocketChannel ch) throws Exception {
ChannelPipeline p = ch.pipeline();
//将原始的HTTP请求和响应数据进行编解码
p.addLast(new HttpServerCodec());
//将HTTP请求或响应的多个部分合并成一个完整的消息
p.addLast(new HttpObjectAggregator(65536));
p.addLast(new SimpleChannelInboundHandler<FullHttpRequest>() {
@Override
protected void channelRead0(ChannelHandlerContext ctx, FullHttpRequest request) {
//查询URI中的参数：
QueryStringDecoder decoder = new QueryStringDecoder(request.uri());
Map<String, List<String>> params = decoder.parameters();
System.out.println(params);
ByteBuf content = request.content();
//请求主体中的参数
String requestBody = content.toString(CharsetUtil.UTF_8);
System.out.println(requestBody);
String responseContent = "你好, Netty!";
FullHttpResponse response = new DefaultFullHttpResponse(request.protocolVersion(), HttpResponseStatus.OK, Unpooled.wrappedBuffer(responseContent.getBytes()));
response.headers().set(HttpHeaderNames.CONTENT_TYPE, HttpHeaderValues.TEXT_HTML + ";charset=utf-8");
response.headers().set(HttpHeaderNames.CONTENT_LENGTH, response.content().readableBytes());
ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE);
}
});
}
});
ChannelFuture f = b.bind(port).sync();
f.channel().closeFuture().sync();
} finally {
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
public static void main(String[] args) throws Exception {
int port = 8080;
HttpServer server = new HttpServer(port);
System.out.println("Server started on port " + port);
server.start();
}
}
After running this example, you can use a browser or other tools to send HTTP requests to http://localhost:8080 , and an HTTP server is born, which is very simple. Next, explain the code:
The meaning in the code b.group(bossGroup, workerGroup)is that there are two thread groups that will handle IO events in the server, and bossGrouponly one thread is used to monitor the port of the server, receive client connection requests, and assign the connection to the workerGroupEventLoop in for processing; workerGroupresponsible for processing For the I/O events of the received connection, operations such as request decoding, processing business logic, and sending responses are handed over to EventLoop for processing. This is a typical master-slave Reactor mode. By separating connection reception and processing into different thread pools, the performance of network applications can be improved. The model is as follows.
NioServerSocketChannelIt is the Channel type of the specified server, as well as NioDatagramChannel and other types, depending on the application scenario.
.handler(new LoggingHandler(LogLevel.INFO))It is to designate a channel processor for the bossGroup, record the data flow in and out of the Channel, and print the relevant information to the log for easy troubleshooting.
.childHandler()It is to workerGroupconfigure the processor for the EventLoop in it, such as request decoding, processing business logic and sending responses. ChannelPipelineIt is to add a specific channel processor. The processors in the code HttpServerCodecare HttpObjectAggregatorused to process the encoding and decoding of HTTP requests, SimpleChannelInboundHandlerand the business logic and response are performed after receiving the data stream passed through multiple processors.
Summarize
Netty is a very good, powerful and high-performance network communication framework. In this era of rapid Internet development, we need to understand and use such an excellent framework to help us develop applications quickly. When using it, we must know Its principle can also innovate in the business, just like Dubbo, gRPC, and Zookeeper, using Netty to become an excellent framework as it.