Introduction and use of RabbitMQ dead letter queue in SpringBoot

The dead letter queue can perform other processing on these messages when the messages are not normally consumed to ensure that the messages will not be discarded.

Before talking about the dead letter queue, let's first introduce why we need to use the dead letter queue.

If you want to learn about dead letter docking directly, just jump into the "Dead Letter Queue" section below.

ack mechanism and requeue-rejected attribute

RabbitMQ is integrated in SpringBoot

In the project springboot-demo we see that the configuration content of the application.yaml file is as follows

...

listener:
    type: simple
    simple:
      acknowledge-mode: auto
      concurrency: 5
      default-requeue-rejected: true
      max-concurrency: 100
...

among them

acknowledge-mode

This configuration item is used to indicate the message confirmation mode. There are three configuration modes: none, manual, and auto.

none means that no response will be sent.

Manual means that the listener must notify all messages by calling Channel.basicAck().

Auto means that the container will respond automatically unless the MessageListener throws an exception, which is the default configuration method.

default-requeue-rejected

This configuration item determines whether messages rejected due to exceptions thrown by the listener are put back into the queue. The default value is true.

I had a misunderstanding about this attribute at first. I thought rejected means rejection, so I put the requeue-rejectedconnection back to the queue as rejection. Later I checked the information and understood the function of this attribute before I realized that rejected is an adjective. It should mean rejected. Rejected message

So if this property is configured as true, it means it will be put back into the queue, if it is configured as false, it means it will not be put back into the queue.

Let's take a look at how the acknowledge-mode parameter and default-requeue-rejected parameter use different combinations, and how RabbitMQ processes messages.

The code still uses RabbitApplicationTests in springboot-demo to send messages, and uses the Receiver class to monitor demo-queue messages.

Add a line of code to the Receiver class, which simulates throwing an exception

@Component
public class Receiver {

    @RabbitListener(queues = "demo_queue")
    public void created(String message) {
        System.out.println("orignal message: " + message);
        int i = 1/0;
    }
}

acknowledge-mode=none, default-requeue-rejected=false

This configuration does not confirm whether the message is consumed normally, so no exception is thrown in the console. Through the RabbitMQ management page, I did not see the message that was put back into the queue.

acknowledge-mode=none, default-requeue-rejected=true

The same configuration does not confirm whether the message is normally consumed, so no exception is thrown in the console. And even if the default-requeue-rejected is configured as true, because there is no confirmation, there is no message to be put back into the queue.

acknowledge-mode=manual, default-requeue-rejected=false

This configuration needs to manually confirm whether the message is normally consumed, but there is no manual confirmation in the code. My personal understanding is that because the ack is not received, the message is returned to the queue.

acknowledge-mode=manual, default-requeue-rejected=true

This configuration needs to manually confirm whether the message is normally consumed, but there is no manual confirmation in the code, so the message is put into the queue again, and an exception is also thrown in the console ( this is not very clear, default-requeue- The different effects of setting true and false are rejected, please leave a message below if you have trouble understanding ).

acknowledge-mode=auto, default-requeue-rejected=false

This configuration uses automatic confirmation. From the results, it is automatically confirmed.

From the results printed on the console, it can be seen that the Receiver method has been executed 3 times, the first two messages put back into the queue and the message sent this time, so all 3 messages are consumed.

At the same time, because the default-requeue-rejected is set to false, even if the consumption throws an exception, the message is not returned to the queue.

acknowledge-mode=auto, default-requeue-rejected=true

This configuration also uses automatic confirmation. From the results, it can be seen that no exception is thrown ( this is not very understanding ), and because the default-requeue-rejected is set to true, the message is returned to the queue.

In summary, so many situations are listed to illustrate that in some cases, if the message consumption is wrong, the message is lost due to configuration problems. This is terrible in many cases. For example, the order number paid by the user is terrible if it is directly lost due to throwing exceptions.

Therefore, we need to have an assurance mechanism that can ensure that even failed messages can be preserved. At this time, the dead letter queue comes into play.

Dead letter queue

The whole design idea of ​​the dead letter queue is like this

Producer --> Message --> Exchange --> Queue --> Become a Dead Letter --> DLX Exchange --> Queue --> Consumer

Let's take a look at how to use a dead letter queue through the implementation of a simple dead letter queue on the Internet.

@Bean("deadLetterExchange")
    public Exchange deadLetterExchange() {
        return ExchangeBuilder.directExchange("DL_EXCHANGE").durable(true).build();
    }

    @Bean("deadLetterQueue")
    public Queue deadLetterQueue() {
        Map<String, Object> args = new HashMap<>(2);
//       x-dead-letter-exchange    声明  死信交换机
        args.put("x-dead-letter-exchange", "DL_EXCHANGE");
//       x-dead-letter-routing-key    声明 死信路由键
        args.put("x-dead-letter-routing-key", "KEY_R");
        return QueueBuilder.durable("DL_QUEUE").withArguments(args).build();
    }

    @Bean("redirectQueue")
    public Queue redirectQueue() {
        return QueueBuilder.durable("REDIRECT_QUEUE").build();
    }

    /**
     * 死信路由通过 DL_KEY 绑定键绑定到死信队列上.
     *
     * @return the binding
     */
    @Bean
    public Binding deadLetterBinding() {
        return new Binding("DL_QUEUE", Binding.DestinationType.QUEUE, "DL_EXCHANGE", "DL_KEY", null);

    }

    /**
     * 死信路由通过 KEY_R 绑定键绑定到死信队列上.
     *
     * @return the binding
     */
    @Bean
    public Binding redirectBinding() {
        return new Binding("REDIRECT_QUEUE", Binding.DestinationType.QUEUE, "DL_EXCHANGE", "KEY_R", null);
    }

note

• A direct mode exchange is declared.

• A dead letter queue deadLetterQueue is declared. The queue is configured with some attributes to x-dead-letter-exchangeindicate a dead letter switch and x-dead-letter-routing-keyindicate a dead letter routing key. Because it is in direct mode, this routing key needs to be set.

• A replacement queue redirectQueue is declared, and messages that become dead letters are ultimately stored in this queue.

• Declare the binding relationship, which are the dead letter queue and the binding of the substitute queue and the switch.

So how to simulate a dead letter message? The message sent to DL_QUEUE becomes invalid after 10 seconds and then forwarded to the substitute queue. The code is implemented as follows

public void sendMsg(String content) {
        CorrelationData correlationId = new CorrelationData(UUID.randomUUID().toString());
        MessagePostProcessor messagePostProcessor = message -> {
            MessageProperties messageProperties = message.getMessageProperties();
//            设置编码
            messageProperties.setContentEncoding("utf-8");
//            设置过期时间10*1000毫秒
            messageProperties.setExpiration("5000");
            return message;
        };
        rabbitTemplate.convertAndSend("DL_EXCHANGE", "DL_KEY", content, messagePostProcessor);
    }

The execution results are as follows

The message enters DL_QUEUE first, and it becomes invalid after 5 seconds and is forwarded to REDIRECT_QUEUE.

------------------------------------------------------------------------------------------------------------------

 

# Concept:

• A scenario where the message becomes a dead letter message:

  1. The message is (basic.reject() or basic.nack()) and requeue = falserejected, that is , the message is rejected by the consumer, and the re-entry is false.
     1.1 There is a scenario that needs attention: the consumer sets up automatic ACK. When the number of repeated delivery reaches the set maximum number of retry, the message will also be delivered to the dead letter queue, but the internal principle is still called nack/ reject.
  2. The message expired and the TTL lifetime has passed.
  3. The queue has set the x-max-lengthmaximum number of messages and the messages in the current queue have reached this number. If you post again, the messages will be squeezed out, and the messages that are squeezed out are the messages closest to the end of being consumed.

• The code writing process is:

  1. There is one (n) normal business Exchange, for example user-exchange.
  2. There is one (n) Queue of normal business, for example user-queue. (Since the dead letter queue needs to bind the switch, it is necessary to add two parameters: the switch badmail: x-dead-letter-exchange, dead letter message routing keys: x-dead-letter-routing-key)
  3. Binding of switches and queues for normal business.
  4. Define a dead letter switch, such as for common-dead-letter-exchange.
  5. Bind the normal business queue to the dead letter switch (the queue x-dead-letter-exchangewill be automatically bound when it is set).
  6. Define the dead letter queue user-dead-letter-queueto receive dead letter messages and bind the dead letter switch.

• The business process is:

  1. Normal business messages are delivered to the normal business Exchange, which routes the messages to the bound normal queue according to the routing key.
  2. After the message in the normal business queue becomes a dead letter message, it will be automatically delivered to the dead letter switch bound to the queue (with the configured routing key. If the routing key of the dead letter message is not specified, it will be inherited by default This message is the routing key set during normal business).
  3. After the dead letter exchange receives the message, it routes the message to the specified dead letter queue according to the routing rules.
  4. After the message reaches the dead letter queue, you can monitor the dead letter queue and process the dead letter message.

• 死信交换机, It 死信队列is also an ordinary switch and queue, but we artificially use a certain switch and queue to process dead letter messages.

• flow chart

Picture.png

# Code

1. Configuration

 

spring:
  application:
    name: learn-rabbitmq
  rabbitmq:
    host: localhost
    port: 5672
    username: futao
    password: 123456789
    virtual-host: deadletter-vh
    connection-timeout: 15000
    # 发送确认
    publisher-confirms: true
    # 路由失败回调
    publisher-returns: true
    template:
      # 必须设置成true 消息路由失败通知监听者，而不是将消息丢弃
      mandatory: true
    listener:
      simple:
        # 每次从RabbitMQ获取的消息数量
        prefetch: 1
        default-requeue-rejected: false
        # 每个队列启动的消费者数量
        concurrency: 1
        # 每个队列最大的消费者数量
        max-concurrency: 1
        # 签收模式为手动签收-那么需要在代码中手动ACK
        acknowledge-mode: manual

app:
  rabbitmq:
    # 队列定义
    queue:
      # 正常业务队列
      user: user-queue
      # 死信队列
      user-dead-letter: user-dead-letter-queue
    # 交换机定义
    exchange:
      # 正常业务交换机
      user: user-exchange
      # 死信交换机
      common-dead-letter: common-dead-letter-exchange

1. Queue and switch definition and binding.

 

/**
 * 队列与交换机定义与绑定
 *
 * @author futao
 * @date 2020/4/7.
 */
@Configuration
public class Declare {

        /**
     * 用户队列
     *
     * @param userQueueName 用户队列名
     * @return
     */
    @Bean
    public Queue userQueue(@Value("${app.rabbitmq.queue.user}") String userQueueName,
                           @Value("${app.rabbitmq.exchange.common-dead-letter}") String commonDeadLetterExchange) {
        return QueueBuilder
                .durable(userQueueName)
                //声明该队列的死信消息发送到的 交换机 （队列添加了这个参数之后会自动与该交换机绑定，并设置路由键，不需要开发者手动设置)
                .withArgument("x-dead-letter-exchange", commonDeadLetterExchange)
                //声明该队列死信消息在交换机的 路由键
                .withArgument("x-dead-letter-routing-key", "user-dead-letter-routing-key")
                .build();
    }

    /**
     * 用户交换机
     *
     * @param userExchangeName 用户交换机名
     * @return
     */
    @Bean
    public Exchange userExchange(@Value("${app.rabbitmq.exchange.user}") String userExchangeName) {
        return ExchangeBuilder
                .topicExchange(userExchangeName)
                .durable(true)
                .build();
    }

    /**
     * 用户队列与交换机绑定
     *
     * @param userQueue    用户队列名
     * @param userExchange 用户交换机名
     * @return
     */
    @Bean
    public Binding userBinding(Queue userQueue, Exchange userExchange) {
        return BindingBuilder
                .bind(userQueue)
                .to(userExchange)
                .with("user.*")
                .noargs();
    }

    /**
     * 死信交换机
     *
     * @param commonDeadLetterExchange 通用死信交换机名
     * @return
     */
    @Bean
    public Exchange commonDeadLetterExchange(@Value("${app.rabbitmq.exchange.common-dead-letter}") String commonDeadLetterExchange) {
        return ExchangeBuilder
                .topicExchange(commonDeadLetterExchange)
                .durable(true)
                .build();
    }


   /**
     * 用户队列的死信消息 路由的队列
     * 用户队列user-queue的死信投递到死信交换机`common-dead-letter-exchange`后再投递到该队列
     * 用这个队列来接收user-queue的死信消息
     *
     * @return
     */
    @Bean
    public Queue userDeadLetterQueue(@Value("${app.rabbitmq.queue.user-dead-letter}") String userDeadLetterQueue) {
        return QueueBuilder
                .durable(userDeadLetterQueue)
                .build();
    }

    /**
     * 死信队列绑定死信交换机
     *
     * @param userDeadLetterQueue      user-queue对应的死信队列
     * @param commonDeadLetterExchange 通用死信交换机
     * @return
     */
    @Bean
    public Binding userDeadLetterBinding(Queue userDeadLetterQueue, Exchange commonDeadLetterExchange) {
        return BindingBuilder
                .bind(userDeadLetterQueue)
                .to(commonDeadLetterExchange)
                .with("user-dead-letter-routing-key")
                .noargs();
    }

}

• After the definition is completed, the program is started, and springboot will read the beans of type Queue and Exchange in the Spring container to initialize and bind queues and switches. Of course, you can also create and bind manually in the RabbitMQ management background.

• View management background

   

  

  switch

   

  

  queue

   

  

  Queue and Dead Letter Exchange

# Test

• Message producer

 

/**
 * @author futao
 * @date 2020/4/7.
 */
@Component
public class DeadLetterSender {

    @Autowired
    private RabbitTemplate rabbitTemplate;

    @Value("${app.rabbitmq.exchange.user}")
    private String userExchange;

    public void send() {
        User user = User.builder()
                .userName("天文")
                .address("浙江杭州")
                .birthday(LocalDate.now(ZoneOffset.ofHours(8)))
                .build();
        rabbitTemplate.convertAndSend(userExchange, "user.abc", user);
    }
}

1. Scenario 1.1

The message is (basic.reject() or basic.nack()) and requeue = false, that is, the message is rejected or nacked by the consumer, and the re-entry is false.

The difference between nack() and reject() is: reject() does not support batch rejection, while nack() can.

• Consumer code

 

/**
 * @author futao
 * @date 2020/4/9.
 */
@Slf4j
@Component
public class Consumer {

    /**
     * 正常用户队列消息监听消费者
     *
     * @param user
     * @param message
     * @param channel
     */
    @RabbitListener(queues = "${app.rabbitmq.queue.user}")
    public void userConsumer(User user, Message message, Channel channel) {
        log.info("正常用户业务监听：接收到消息:[{}]", JSON.toJSONString(user));
        try {
            //参数为：消息的DeliveryTag，是否批量拒绝，是否重新入队
            channel.basicNack(message.getMessageProperties().getDeliveryTag(), false, false);
            log.info("拒绝签收...消息的路由键为:[{}]", message.getMessageProperties().getReceivedRoutingKey());
        } catch (IOException e) {
            log.error("消息拒绝签收失败", e);
        }
    }

    /**
     * @param user
     * @param message
     * @param channel
     */
    @RabbitListener(queues = "${app.rabbitmq.queue.user-dead-letter}")
    public void userDeadLetterConsumer(User user, Message message, Channel channel) {
        log.info("接收到死信消息:[{}]", JSON.toJSONString(user));
        try {
            channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
            log.info("死信队列签收消息....消息路由键为:[{}]", message.getMessageProperties().getReceivedRoutingKey());
        } catch (IOException e) {
            log.error("死信队列消息签收失败", e);
        }
    }
}

• It can be seen that the normal message finally reaches the dead letter queue after being NACKed, and the routing key has changed.

  

  Dead letter message

  
  1. Scenario 1.2

The consumer has set up automatic sign-off. When the number of repeated delivery reaches the set maximum number of retry, the message will also be delivered to the dead letter queue, but the internal principle is still called nack/ reject.

• Some configuration needs to be changed in application.yml

 

spring:
  application:
    name: learn-rabbitmq
  rabbitmq:
    listener:
      simple:
        # 每次从RabbitMQ获取的消息数量
        prefetch: 1
        default-requeue-rejected: false
        # 每个队列启动的消费者数量
        concurrency: 1
        # 每个队列最大的消费者数量
        max-concurrency: 1
        # 自动签收
        acknowledge-mode: auto
        retry:
          enabled: true
          # 第一次尝试时间间隔
          initial-interval: 10S
          # 两次尝试之间的最长持续时间。
          max-interval: 10S
          # 最大重试次数(=第一次正常投递1+重试次数4)
          max-attempts: 5
          # 上一次重试时间的乘数
          multiplier: 1.0

• Consumer code

 

/**
 * @author futao
 * @date 2020/4/9.
 */
@Slf4j
@Configuration
public class AutoAckConsumer {

    /**
     * 正常用户队列消息监听消费者
     *
     * @param user
     */
    @RabbitListener(queues = "${app.rabbitmq.queue.user}")
    public void userConsumer(User user) {
        log.info("正常用户业务监听：接收到消息:[{}]", JSON.toJSONString(user));
        throw new RuntimeException("模拟发生异常");
    }

    /**
     * @param user
     */
    @RabbitListener(queues = "${app.rabbitmq.queue.user-dead-letter}")
    public void userDeadLetterConsumer(User user) {
        log.info("接收到死信消息并自动签收:[{}]", JSON.toJSONString(user));
    }
}

• Test Results:

   

  

  image.png

   

  

  image.png

• It can be seen from the test results that if the message is not normally consumed, it will be retried, and if it has not been consumed normally in the end, it will be delivered to the dead letter queue.

initial-interval, max-intervalI don’t know what these two parameters do. The result of the test now is that the shortest time will always be used as the next delivery time...

2. Test scenario 2

The message expired and the TTL lifetime has passed.

• You need to modify the queue definition and set the expiration time of the queue message x-message-ttl.

 

    /**
     * 用户队列
     *
     * @param userQueueName 用户队列名
     * @return
     */
    @Bean
    public Queue userQueue(@Value("${app.rabbitmq.queue.user}") String userQueueName,
                           @Value("${app.rabbitmq.exchange.common-dead-letter}") String commonDeadLetterExchange) {
        return QueueBuilder
                .durable(userQueueName)
                //声明该队列的死信消息发送到的 交换机 （队列添加了这个参数之后会自动与该交换机绑定，并设置路由键，不需要开发者手动设置)
                .withArgument("x-dead-letter-exchange", commonDeadLetterExchange)
                //声明该队列死信消息在交换机的 路由键
                .withArgument("x-dead-letter-routing-key", "user-dead-letter-routing-key")
                //该队列的消息的过期时间-超过这个时间还未被消费则路由到死信队列
                .withArgument("x-message-ttl", 5000)
                .build();
    }

• The user-queueconsumer comments, the message can not be consumed until the message in the queue reaches the set time of survival.

  

  ttl

   

• According to the log, you can see that the message will be delivered to the dead letter queue after 5S.

   

  

  image.png

• Note: You can set the message expiration time for the queue, then all messages posted to this queue will automatically have this attribute. You can also set a specified expiration time for each message before the message is delivered. (When both are set, the shorter value is taken by default)

The following test sets the specified expiration time for each message:

• Modify the message producer:

 

/**
 * @author futao
 * @date 2020/4/7.
 */
@Slf4j
@Component
public class DeadLetterSender {

    @Autowired
    private RabbitTemplate rabbitTemplate;

    @Value("${app.rabbitmq.exchange.user}")
    private String userExchange;

    public void send(String exp) {
        User user = User.builder()
                .userName("天文")
                .address("浙江杭州")
                .birthday(LocalDate.now(ZoneOffset.ofHours(8)))
                .build();
        log.info("消息投递...指定的存活时长为:[{}]ms", exp);
        rabbitTemplate.convertAndSend(userExchange, "user.abc", user, new MessagePostProcessor() {
            @Override
            public Message postProcessMessage(Message message) throws AmqpException {
                MessageProperties messageProperties = message.getMessageProperties();
                //为每条消息设定过期时间
                messageProperties.setExpiration(exp);
                return message;
            }
        });
    }
}

image.png

• It can be seen from the test results that each message is delivered to the dead letter queue at the specified time.

[Pit] Important note!!!: RabbitMQ's detection of message expiration: It will only detect whether the message that will be consumed recently has reached the expiration time, and will not detect whether the non-terminal message has expired. The problem is that the non-terminal message has expired, but because the terminal message has not expired, the non-terminal message is in a blocking state, so the non-terminal message will not be detected as having expired. Make the business produce results that are seriously inconsistent with expectations.

• Test the above questions: (The expiration time of the first message is set to 10S, and the second message is set to 5S)

   

  

  image.png

• It can be seen from the test result that the survival time of a message with id 1 is 10S, and the survival time of a message with id 2 is 5S. But only when the first message (id=1) expires and the message with id=2 reaches the end of the queue, will it be detected that it has expired.

3. Test scenario 3

The queue has set the x-max-lengthmaximum number of messages and the messages in the current queue have reached this number. If you post again, the messages will be squeezed out, and the messages that are squeezed out are the messages closest to the end of being consumed.

• Modify queue definition

 

  /**
     * 用户队列
     *
     * @param userQueueName 用户队列名
     * @return
     */
    @Bean
    public Queue userQueue(@Value("${app.rabbitmq.queue.user}") String userQueueName,
                           @Value("${app.rabbitmq.exchange.common-dead-letter}") String commonDeadLetterExchange) {
        return QueueBuilder
                .durable(userQueueName)
                //声明该队列的死信消息发送到的 交换机 （队列添加了这个参数之后会自动与该交换机绑定，并设置路由键，不需要开发者手动设置)
                .withArgument("x-dead-letter-exchange", commonDeadLetterExchange)
                //声明该队列死信消息在交换机的 路由键
                .withArgument("x-dead-letter-routing-key", "user-dead-letter-routing-key")
                //队列最大消息数量
                .withArgument("x-max-length", 2)
                .build();
    }

image.png

• Deliver messages to the queue

   

  

  image.png

• It can be seen from the results that when the third message is delivered, RabbitMQ will remove the message on the most consumed end of the queue and deliver it to the dead letter queue.

  

  image.png

   

  A maximum of two messages will always be kept in the queue.

# Other:

• The configurable items of Queue can be viewed in the management background of RabbitMQ:

   

  

  image.png

• Source code: https://github.com/FutaoSmile/springboot-learn-integration/tree/master/springboot-learn-rabbitmq

# Related:

SpringBoot RabbitMQ realizes reliable message delivery

# EVERYTHING:

• Consumer current limit protection
• Delay queue