The last article on the basic concepts of RabbitMQ introduction introduced some basic concepts of RabbitMQ, this article will introduce some of the details and some advanced concepts.
1. What to do when the message sent by the message producer is unreachable
RabbitMQ provides the function of sending the message back to the message sender when the message cannot be sent to a queue during the delivery process (for example, no matching queue is found according to its type and routing key). The client using RabbitMQ provides the channel.basicPublish method The two parameters mandatory and immediate (RabbitMQ 3.0 and below), in addition to providing a backup switch can store the messages that cannot be sent for processing, without retransmitting them back to the sender.
1.1 mandatory parameters
Mandatory is defined in the channel.basicPublish method of the client provided by RabbitMQ, as shown below:
When the mandatory parameter of the method is set to true, then when the exchange cannot find a queue that meets the requirements based on its own type and routing key, RabbitMQ will automatically call Basic.Return to send the message back to the sender, which is us The message producer. On the contrary, if set to false, the message will be directly discarded. So the question is, how do we get these unsent messages? RabbitMQ provides us with an event monitoring mechanism to get this kind of messages. You can add a ReturnListener to get this kind of messages that are not sent to the queue through the addReturnListener method, as shown below:
By looking at the source code of the ReturnListener interface, you can see that the interface has only one method. If it is a JDK8+ version, you can use Lambda expressions to simplify some code.
As can be seen, when set mandatory parameters, must also add programming logic ReturnListener listener while producers, so that the code will become more complicated producers, in order to address this situation, RabbitMQ provided 备份交换器to the Messages that have not been successfully routed are stored and processed when we need them.
1.2 immediate parameter
The parameters are also defined in the channel.basicPublish method. The official description is as follows:
This flag tells the server how to react if the message cannot be routed to a queue consumer immediately. If this flag is set, the server will return an undeliverable message with a Return method. If this flag is zero, the server will queue the message, but with no guarantee that it will ever be consumed.
When the immediate parameter is set to true, if the exchange finds a queue that meets the requirements based on its type and routing key, and finds that there are no consumers on all queues, the message will not be stored in the queue and will pass through Basic. The Return command returns the message to the producer. In short, when the immediate parameter is set, when there is a consumer on the queue associated with the message, the message will be sent to the queue immediately, otherwise if there is no consumer on the matching queue, it will be directly The message is passed back to the producer. One thing to note here is that this parameter has been removed from RabbitMQ 3.0+.
2. How to set expiration time (TTL) for messages and queues
TTL is the abbreviation of the initials of time to live. RabbitMQ can set the expiration time of messages and queues. Let's take a look at how to set the expiration time of messages.
1.1 Message TTL setting
RabbitMQ provides two types of setting the expiration time of the message. The first is to set the attribute of the queue. The feature of this method is that the expiration time of all messages in the queue is the same. There is also a smaller granularity setting, which is to set the expiration time for each message separately. This method is more flexible, and the expiration time of each message can be different. At this time, you may ask, if the expiration attribute of the queue and the expiration attribute of the message itself are set at the same time, which one will prevail? The result is that RabbitMQ will compare the values of these two TTLs, whichever is smaller. It is easy to think that if the expiration time is set by the attributes of the queue, it is specified when the queue is declared. The corresponding parameter of the channel.queueDeclare method provided by the client is specified. The sample code is as follows:
Note that the unit of the x-message-ttl parameter is milliseconds. If TLL is not set, it means that the message will not expire. If TTL is set to 0, it means that unless the message can be sent directly to the consumer at this time, the message will be directly discarded.
The method of setting TTL for each message is set when the message is sent. The corresponding client method is the expiration attribute parameter of channel.basicPublish. The specific setting code is as follows:
With this setting method, even if the queue expires, it will not be removed from the queue immediately, because the determination of whether each message expires is performed when it is sent to the consumer. If the message is found to have expired, the message will be deleted. For the first method, the expired messages are moved to the head of the queue, and RabbitMQ only needs to periodically scan from the beginning for expired messages.
1.2 Queue TTL setting
To set the expiration time of the queue, use the x-expires parameter in the channel.queueDeclare method parameter of the client. The unit is also milliseconds, but it should be noted that it cannot be set to 0. The code to set the queue expiration is as follows:
The above code creates a queue with an expiration time of 15 minutes.
3. Introduction to Dead Letter Queue
The full name of the Dead Letter Exchange (DLX) is Dead-Letter-Exchange, also known as the Dead Letter Mailbox. Simply put, when a message becomes a dead message (dead message) due to the message being rejected, the message expires, and the queue reaches the maximum length, it will be resent to an exchange. This exchange is a dead letter exchange. The queue bound to this switch is called a dead letter queue. The dead letter exchange is actually an ordinary exchange, which can be specified on any queue. When a dead letter exchange is set on a queue, if a dead letter appears in the queue, the dead letter message will be resent by RabbitMQ. The letter exchange will go up and route to the dead letter queue. We can monitor this queue to process those dead letter messages. Setting a dead letter exchange for a queue is achieved by setting the x-dead-letter-message parameter in the producer's method of declaring the queue, as shown below:
At the same time, you can also set the routing key of the dead letter interactor through the x-dead-letter-routing-key parameter, instead of setting the routing key that uses the original degree column by default. You can go to the background management interface of RabbitMQ. The dead letter queue with the DLX mark.
The DLX provided by RabbitMQ is a more practical feature. It can be put into the dead letter queue when our messages cannot be correctly consumed by consumers. In the future, we can use the content of this dead letter queue to view abnormal conditions to transform and optimize system.
Four, delay queue introduction
As the name implies, the delayed queue stores the delayed messages that need to wait for a specified time to get. A typical scenario is that the order is cancelled without payment after 30 minutes. It should be noted here that RabbitMQ does not directly provide the delay queue function, but needs to be implemented through the expiration time (TTL) described above and the dead letter queue. For example, in the scenario of canceling an order over time, we can let consumers Subscribe to the dead letter queue, set the normal timeout time of the queue to 30 minutes and bind it to the dead letter queue. When the message is not processed for more than 30 minutes, the message will be sent to the dead letter queue, and then the dead letter queue Consumers can successfully consume the message after 30 minutes.
At the same time, it is also very convenient to expand when we have other timeout configuration requirements. For example, when the producer sends a message, different routing keys can be set, and the message can be sent to different queues bound to the exchange through the routing key, and then These queues are set with different expiration times and corresponding dead letter queues. When the message expires, it will be forwarded to the corresponding dead letter queue by RabbitMQ, so that you can subscribe to the corresponding dead letter queue.
Five, exchange, message and queue persistence
Persistence can improve reliability and prevent data loss under abnormalities such as crashes or restarts. The persistence of RabbitMQ can be divided into three parts from the composition structure, namely exchange persistence, message persistence and queue persistence.
1.1 Exchanger persistence
Exchanger persistence is achieved by setting the durable parameter to true when declaring the exchange. If you do not set the persistence attribute, the data of the exchange will be lost when the RabbitMQ service is restarted. It should be noted that the data of the exchange is lost, and the message will not be lost, but the message cannot be sent to this exchange. The general production environment will set this property to persistence.
1.2 Information endurance
The persistence of the exchange only ensures that the metadata of the exchange itself will not be lost, and it cannot guarantee that the messages stored in it will not be lost. If you need the messages stored in it to not be lost, you need to set up the persistence of the messages. The delivery mode (deliveryMode) is set to 2 to achieve message persistence, as shown below:
The prerequisite for message persistence is that the queue where it is located must also be persisted. If only the persistence of the message is set, the queue disappears after RabbitMQ is restarted, and the message will also be lost. There is something to note here. Although persistence can improve reliability, persistence is to store data on the hard disk, which is much slower than directly operating memory, so there is no need for persistence for those businesses that do not require high reliability.
1.3 Queue persistence
The persistence setting of the queue is similar to the persistence of the exchange. It is also realized by setting the durable parameter to true when it is declared. If it is not set, when RabbitMQ is restarted, the related queue metadata will also be lost, and its corresponding storage The news will also be lost.
After setting up the switch, queue, and message to be persistent, can it be 100% guaranteed that the data will not be lost? In fact, there is no guarantee that 100% of the data will not be lost. For example, if the consumer sets the autoAck parameter to true when subscribing to the consumption queue, the message will hang before it can be processed after receiving the message. At this time, the automatic answer needs to be set to false and the manual ack response is sufficient. Another is that because it is not real-time persistent storage, when RabbitMQ is down during the process of saving messages, data loss will also occur at this time. At this time, it needs to be processed by RabbitMQ's mirror queue mechanism.
Six, summary
This article mainly introduces the setting details and dead letter queues, delay queues, and persistence of some parameters when they are used. There are also some more important points that are not involved, such as the message confirmation mechanism. Knowing this, you must do it yourself. After understanding some basic concepts, you still need to go through specific coding practices to understand them more deeply.