RabbitMQ threshold may be set for the amount of memory and disk, when the threshold is reached, the producer will be blocked until the entry corresponding to normal. In addition to these two thresholds, version 2.8.0 from the beginning, RabbitMQ also introduces flow control (Flow Control) mechanism to ensure stability. Flow control mechanism is used to avoid sending messages too frequently lead to situations difficult to support the server. Memory and disk alarms equivalent to the global flow control, once the cluster will block all Connection triggers, and flow control is for a single Connection, you can call Per-Connection Flow Control or Internal Flow Control.
Principle of flow control
Do not share memory between Erlang processes (except binary type), but communicate by passing messages, each process has its own process mailbox (mailbox). By default, Erlang does not limit the size of the process the mailbox, so when a large number of messages sent to a continuous process, the process will lead to the mailbox is too large, the final out of memory and crash. In RabbitMQ, if the producers continue to send high-speed, and lower consumer spending rate, if there is no flow control, and soon will make the internal processes mailbox size memory threshold is reached.
RabbitMQ using a credit positive based algorithm (credit-based algorithm) flow control mechanism to limit the rate of sending messages to solve this problem. It is the process by monitoring each process mailbox, when a process load is too high and too late to process the message, the process will begin to slowly accumulate email messages, when accumulated to a certain amount, it will block without receiving new upstream messages. So slowly, upstream of the process-mail messages will start to slowly accumulate. When accumulated to a certain amount of time will be blocked to stop receiving messages upstream, and finally it will be responsible for the network packet received suspended process blocked receive new data. The following figure as an example:
Process A receives the message forwarded to the process B, process B receives the message forwarded to the process C. Each process has a pair of credit values for send and receive messages. In the process of Case B, {{credit_from, C}, value} indicates how many messages can be sent to C, each message sent on the value by one, when the process B does not send messages to the Process C is 0, a process is also no longer receive messages of. {{Credit_to, A}, value} then indicates how many received notification message is sent increasing the credit value to the process A, after receiving the notification process A increases {credit_from, B}, value {} corresponding to the value, so that the process A message can be sent continuously. When the upstream transmission rate is higher than the downstream reception rate, will gradually Credit value is depleted, then the process will be blocked, the blocking of the case would have been transmitted to the most upstream. When notified from the upstream process to increase the credit value of the downstream, upstream in the blocked state if at this time the most upstream unblocked state, starts receiving the message further upstream process, a conductive eventually a most upstream unblocks. Thus, the credit-based flow control mechanism of the transmission rate will eventually message sending process is limited within the range of the processing capability of the message handling process.
It will be in the "flow" state when a trigger connected to flow control, which means the state Connection between blocked and unblocked switching back and forth several times per second, so that message transmission rate can be controlled within the range of the server can support You may be ordered by rabbitmqctl list_connections or Web management interface to view the status of Connection.
Connection in a flow state, and in the running state is no different, just tell the state administrator corresponding transmission rate is limited, while the client, the server is smaller than the bandwidth of just under normal circumstances it sees .
Flow control mechanism is not only applied to the Connection, and the same effect on the channel queue. From the Channel Connection, then the queue, and finally the message persistent storage control stream to form a complete chain, the entire process for flow control in an arbitrary chain, as long as the blocking process, the process must be upstream of the obstruction. That is, if a process to achieve the performance bottleneck, will inevitably lead to all processes upstream blocked. So we can use this feature to find out where the flow control mechanism of the bottleneck, the order of a few key processes and their corresponding process messages in the following figure:
Wherein each of the process is as follows:
❤ rabbit_reader: Connection handling process, is responsible for receiving, parsing AMQP protocol packets and so on;
❤ rabbit_channel: Channel ways of handling the process responsible for handling AMQP protocol, routing resolution and so on;
❤ rabbit_amqqueue_process: process queue processing, and is responsible for implementing all logical queues;
❤ rabbit_msg_store: responsible for persistence of messages;
When a Connection in a flow state, but this is not a Channel Connection in the flow state, which means that there are one or more Connection Channel performance bottlenecks appeared, some Channel operation processes (such as processing routing logic) will It enables the server CPU load is too high and lead to such a situation arise. Especially when sending a large number of smaller non-persistent messages, such cases most likely to appear.
When a Connection in a flow state, and this also has a plurality of Connection Channel in a flow state, but there is no flow in a queue corresponding to a state, which means that there are one or more queues having performance bottlenecks. This may be due to the process of the message into a queue cause the server CPU load is too high, or will cause the server I / O load is too high process messages in the queue saved to disk in such a situation arising. Especially when sending a large number of small persistent messages, such cases most likely to appear.
When a Connection in a flow state, and this also has a number of Channel Connection in the flow state, and there are a number of queues in flow state, which means that there has been a performance bottleneck in the message persistence time. Causing the server I / O load is too high and such a situation arising in the course of the queue of messages stored in disk. Especially when sending large amounts of large persistent messages such cases most likely to appear.
Reference: "RabbitMQ combat Guide" edited by ZHU Zhong-hua;