Efficient Reactor threading model
There are three commonly used Reactor threading models, as follows:
1) Reactor single-threaded model;
2) Reactor multi-threaded model;
3) Master-slave Reactor multithreading model
The Reactor single-threaded model means that all IO operations are completed on the same NIO thread. The responsibilities of the NIO thread are as follows:
1) As the NIO server, it receives the TCP connection of the client;
2) As a NIO client, initiate a TCP connection to the server;
3) Read the request or response message of the communication peer;
4) Send a message request or response message to the communication peer.
The schematic diagram of Reactor single-threaded model is as follows:
Since the Reactor mode uses asynchronous non-blocking IO, all IO operations will not cause blocking. In theory, a thread can independently handle all IO-related operations. From an architectural perspective, a NIO thread can indeed fulfill its responsibilities. For example, through Acceptor to receive the client's TCP connection request message, after the link is established successfully, the corresponding ByteBuffer is sent to the designated Handler through Dispatch for message decoding. The user Handler can send messages to the client through the NIO thread.
For some small-volume application scenarios, a single-threaded model can be used. However, it is not suitable for applications with high load and large concurrency. The main reasons are as follows:
1) A NIO thread processes hundreds of thousands of links at the same time, and its performance cannot be supported. Even if the CPU load of the NIO thread reaches 100%, it cannot meet the encoding, decoding, reading and sending of massive messages;
2) When the NIO thread is overloaded, the processing speed will slow down, which will cause a large number of client connections to time out, and retransmission will often occur after the timeout, which will increase the load on the NIO thread and eventually cause a large amount of message backlog and processing Overtime, the NIO thread will become the performance bottleneck of the system;
3) Reliability issues: Once the NIO thread runs away accidentally or enters an infinite loop, the communication module of the entire system will be unavailable, unable to receive and process external messages, and cause node failure.
In order to solve these problems, the Reactor multi-threading model has evolved. Let's study the Reactor multi-threading model together.
The biggest difference between the Rector multi-threaded model and the single-threaded model is that there is a set of NIO threads to handle IO operations. Its schematic diagram is as follows:
Features of Reactor multithreading model:
1) There is a dedicated NIO thread-Acceptor thread for monitoring the server and receiving TCP connection requests from the client;
2) Network IO operations-reading, writing, etc. are handled by a NIO thread pool. The thread pool can be implemented using a standard JDK thread pool. It contains a task queue and N available threads. These NIO threads are responsible for reading and writing messages. Decoding, encoding and sending;
3) One NIO thread can process N links at the same time, but one link only corresponds to one NIO thread, preventing concurrent operation problems. In most scenarios, the Reactor multi-threaded model can meet performance requirements; however, in very special application scenarios, a NIO thread is responsible for monitoring
There may be performance issues with handling all client connections. For example, millions of clients connect concurrently, or the server needs to perform security authentication on the client's handshake message. The authentication itself is very lossy. In this type of scenario, a single Acceptor thread may have insufficient performance. In order to solve the performance problem, a third Reactor thread model-the master-slave Reactor multi-threaded model was created.
The feature of the master-slave Reactor thread model is that the server is no longer a single NIO thread for receiving client connections, but an independent NIO thread pool. After Acceptor receives the client TCP connection request processing is completed (may include access authentication, etc.), the newly created SocketChannel is registered to an IO thread of the IO thread pool (sub reactor thread pool), and it is responsible for the read and write of the SocketChannel And codec work. The Acceptor thread pool is only used for client login, handshake and security authentication. Once the link is established successfully, the link is registered to the IO thread of the backend subReactor thread pool, and the IO thread is responsible for subsequent IO operations.
Its threading model is shown below:
Using the master-slave NIO thread model, it can solve the problem of insufficient performance of a server listening thread that cannot effectively handle all client connections. Therefore, in Netty's official demo, it is recommended to use this threading model.
In fact, Netty's threading model is not fixed. By creating different EventLoopGroup instances in the startup auxiliary class and configuring appropriate parameters, the above three Reactor threading models can be supported. It is precisely because Netty's support for the Reactor thread model provides flexible customization capabilities, it can meet the performance requirements of different business scenarios.