Compared with the conventional solution using the HTTP protocol, the first feature provided by TARS is the RPC calling method of asynchronous long connection:
After initiating an asynchronous call, the current thread will not be blocked but will continue to execute. After receiving the response from the server, the callback function will be used to execute the result processing in the callback thread pool. In this way, all processing threads are always in a working state, and will not hang up to cause waste of thread resources. Overall, the processing capacity of the service has been improved.
The asynchronous capability of TARS is mainly realized through the asynchrony of two parts. The first is the asynchrony of the network first packet. The network layer implementation of TARS adopts the Reactor model, and the event-based asynchronous network IO is realized through the event IO provided by nio. The second is the asynchrony of the thread model. Let's look at how TARS makes asynchronous calls from the thread model:
TARS mainly completes the asynchronous call through the process in the above figure. First, the calling thread initiates an asynchronous call, and the calling thread will request the content Join the sending queue of the network thread pool, and then the thread continues to execute. The network thread pool is implemented using the Reactor model, and event IO is implemented through the Selecter provided by nio, so all network threads are event-driven asynchronous IO. After listening to the write event of the corresponding connection, the request is sent, and the read event is read after listening to the read event. Respond and hand over to the callback thread to process the response. In this way, all threads avoid IO blocking and achieve higher utilization efficiency.
Solve the stability problem of the SpringCloud server based on the synchronous thread model
TARS provides pure asynchronous programming, and the ability to protect the server from overloading. The normal processing efficiency of the service can also be guaranteed by ensuring that a large number of requests are received on the server. Secondly, because the caller uses long connections and asynchronous calls, it avoids a large number of new The waste of resources caused by connection and blocking improves the overall stability of the service.
In addition, if the server receives excessive requests, it will often cause thread competition on the server, making the processing capacity of the server lower than the normal processing level. In TARS, queues are used for overload protection. Let's look at the threading model of TARS:
After the network thread receives the request, TARS will add the request to the request queue first, and the worker thread will get the request from the request queue for processing. If a large number of requests arrive in a short period of time, they will only be cached in the request queue and will not affect the worker thread pool. processing capacity. If the worker thread pool gets the request from the queue and finds that it has timed out, it will directly discard the request to avoid processing invalid requests.
Scenes. Asynchronous calls, avoid blocking, improve performance
If we have such a calling relationship in Spring Cloud, service A needs to call service B, and service B needs to rely on service C, which takes much more time to process than service B. For example, in a common business scenario, if the API interface needs to call an order generation service, and the order generation service only needs to generate an order ID, the calculation amount is relatively small, but it also needs to rely on an order writing service, which should be related to inventory Modification and order writing require a series of transaction processing, and the overall time-consuming is much longer than the generation of the order ID, so a large number of thread resources of the order service are wasted in waiting for the order writing service. In this case, TARS can be used to modify the order service and write service, so as to use the asynchronous call write service to improve resource utilization, and use the asynchronous RPC capability provided by TARS to carry out in-depth transformation:
