gRPC completion queue architecture
- grpc uses completion_queue to cache events, typically like rpc requests.
- Use grpc_cq_end_op to add events to cq, use grpc_completion_queue_next or pluck to get events from the queue.
- The difference between next / pluck is that next takes events from the queue in order, and pluck can obtain specific events by specifying tag conditions.
Next, we understand the specific use of completion_queue through the process flow of synchronous RPC requests.
- For grpc_server, the specified number of cqs are used to process RPC requests according to the configuration at startup. The default is 1.
- For each cq, a ThreadManager (thread pool) is started to process, which uses 1 ~ CPU core threads to process epoll requests, and then uses other dynamic threads to process specific RPC methods.
- For each RPC method registered in grpc_server, it will be managed with register_method, each register_method has a request_matcher to track the received RPC request
request_matcher tracks RPC requests as follows:
- Internally maintain a fixed number of tokens (the number of tokens is the same as the number of cq). If there is a token when the request arrives, specify to put it in the cq queue. If the token is currently consumed, it will be placed in the queue to be processed (the request is in the pending state).
- grpc_server will apply for a token every time it receives an RPC request. When actually processing the RPC method, it will return the token, and at the same time determine whether there is a pending status request to be processed.
- Through this mechanism, certain flow control is performed on RPC requests.
The various object relationships mentioned above are shown below:
Let's look at how to use the queue when the RPC request arrives.
The overall relationship involving the objects is as follows:
A cq usually consists of 3 parts
- The grpc_completion_queue structure header is used to store queue management information.
- completion_queue_data, queue data storage. Internally store events through grpc_cq_event_queue. This queue is an MPSC (multi-producer single consumer queue), when there is only one consumer, it is lock-free and has high performance. When there are multiple consumers, the lock will be used for synchronization
- grpc_pollset is used to manage the concerned fd, driving the event loop
When an RPC request arrives, grpc_cq_end_op encapsulates the RPC request into grpc_cq_completion and puts it in the queue. If it is the first event in the queue, it will call grpc_pollset's kick to wake up a worker to complete the task.
