RabbitMQ implements delay queue through TTL and DLX

Rabbitmq implemented using the delay queue delay (timing) of the task
dynamic timing task many students first reaction is achieved by the spring timer task polling schedule database, this scheme has disadvantages following points:
(1) consume system memory Because timed tasks have been occupying processes in the system, it consumes more memory
(2) increases the pressure on the database. This mentions two aspects. One is that the database connection is occupied for a long time, and the query base is large
(3) there is a large time error
but we use third-party plug-ins such as rabbitmq to achieve.

1. Introduction

RabbitMQ itself does not directly support the delay queue function, but it can simulate the delay queue function through TTL and DLX. Use RabbitMQ's two features to curve the delay queue: Time To Live (TTL) and Dead Letter Exchanges (DLX)

1.TTL

TTL is an attribute of a message or queue in MQ. It indicates the maximum survival time of a message or all messages in a queue or queue, in milliseconds. If a message has a TTL attribute set, or enters a queue with a TTL set, if the message is not consumed within the TTL time, the message becomes a dead letter. If the TTL of the message and the TTL of the queue are configured, it will be worse. The lower value will be used.
The TTL of the message is the key to the delay task.

How to set TTL (2 ways):

1. One is to set the "x-message-ttl" attribute of the queue when creating the queue.
2. The other is to set TTL for each message.

Note : In the
first type, if the TTL of the queue is set, if the message expires, it will be entered into the dead letter queue;
and the second type will not be discarded immediately even if the message expires, because whether the message expires is determined before it is delivered to the consumer of. In addition, if TTL is not set, it means that the message will never expire, and if the message expires, it will become a dead letter.

2.Dead Letter Exchanges

DLX is a dead letter exchange.
If a message meets the following conditions, it will enter the dead letter routing. Remember that this is a routing instead of a queue. A routing can correspond to many queues.
①. A message is rejected by Consumer, and the parameter of the reject method is false. In other words, it will not be placed in the queue again and used by other consumers.
②. The TTL of the above message has arrived, and the message has expired.
③. The length limit of the queue is full. Messages that are ranked first are discarded or thrown on the dead letter routing.

Dead Letter Exchange is actually an ordinary exchange, no different from creating other exchanges. It's just that if a message expires in a certain queue (delay queue) that sets up Dead Letter Exchange, it will automatically trigger the forwarding of the message and send it to Dead Letter Exchange.

Note :
Even if a message expires earlier than other messages in the same queue, the expired messages will not enter the dead letter queue first. They are still consumed by consumers in the order of storage. If the expiration time of the first incoming message is 1 hour, then the consumer of the dead letter queue may wait 1 hour to receive the first message. Only when the expired message reaches the top of the queue (the head of the queue), will it be truly discarded or enter the dead letter queue.

Therefore, when considering using RabbitMQ to implement a delayed task queue, you need to ensure that the delay time of each task in the business is consistent. If you encounter different task types that require different delays, you need to establish a separate message queue for each message with a different delay time.

Second, the idea of ​​implementing delay queues

Delay queue is how long you want the message to be processed.
TTL can just make the message become a dead letter after the delay. On the other hand, the message that becomes the dead letter will be delivered to the dead letter queue, so that only the consumer needs to consume the messages in the dead letter queue all the time, because The messages inside are all messages that you want to be processed immediately.

Three, SpringBoot+RabbitMQ implements delay queue

1.RabbitMQConfig configuration class

Create delay queue, dead letter queue, delay switch, dead letter switch
We use an ordinary queue as a delay queue, set TTL, when the message expires, it will enter the dead letter queue.

1. Bind the delay queue to the delay switch and set the routing key
2. Bind the dead letter queue to the private letter exchange bing to set the routing key
3. And create a new Map in the delay queue, set the dead letter switch bound to the current queue, the dead letter routing key of the current queue, and the TTL of the queue.

view:

 /**
     * 实现延时队列的思路：
     * 生产者生产消息
     * 通过 延时队列路由key  经过  延时交换机   发送到  延时队列
     * 延时队列中的消息过期以后
     * 通过 死信队列路由key 经过  死信交换机  进入死信队列
     * 消费者监听死信队列进行消费
     */

    // 延时交换机
    @Bean("delayExchange")
    public DirectExchange delayExchange(){
    
    
        return new DirectExchange("delayExchange");
    }

    // 死信交换机
    @Bean("deadLetterExchange")
    public DirectExchange deadLetterExchange(){
    
    
        return new DirectExchange("deadLetterExchange");
    }

    /**
     * 死信队列A            用于接收延时10s处理的消息
     * @return
     */
    @Bean("deadLetterQueueA")
    public Queue deadLetterQueueA(){
    
    
        return new Queue("deadLetterQueueA");
    }

    /**
     * 延时队列A        延时6s
     * 并绑定到对应的死信交换机
     * @return
     */
    @Bean("delayQueueA")
    public Queue delayQueueA(){
    
    
        Map<String, Object> args = new HashMap<>(2);
        // x-dead-letter-exchange    这里声明当前队列绑定的死信交换机
        args.put("x-dead-letter-exchange", "deadLetterExchange");
        // x-dead-letter-routing-key  这里声明当前队列的死信路由key
        args.put("x-dead-letter-routing-key", "deadLetterQueueAroutingkey");
        // x-message-ttl  声明队列的TTL  单位是毫秒 1000*6
        args.put("x-message-ttl", 6000);
        return QueueBuilder.durable("delayQueueA").withArguments(args).build();
    }


    /**
     * 将延时队列A与延时交换机绑定   并指定延时队列路由
     * @param queue
     * @param exchange
     * @return
     */
    @Bean
    public Binding delayBindingA(@Qualifier("delayQueueA") Queue queue,
                                 @Qualifier("delayExchange") DirectExchange exchange){
    
    
        return BindingBuilder.bind(queue).to(exchange).with("delayQueueAroutingkey");
    }

    /**
     * 将死信队列 与 死信交换机绑定   指定死信队列路由
     * @param queue
     * @param exchange
     * @return
     */
    @Bean
    public Binding deadLetterBindingA(@Qualifier("deadLetterQueueA") Queue queue,
                                      @Qualifier("deadLetterExchange") DirectExchange exchange){
    
    
        return BindingBuilder.bind(queue).to(exchange).with("deadLetterQueueAroutingkey");
    }

2. Consumer

Consumers can directly monitor the dead letter queue! The messages in the dead letter queue are delayed messages.

@Component
@Slf4j
public class RabbitMQConsumer {
    
    

    @RabbitListener(queues = "deadLetterQueueA")
    public void receiveA(Message message, Channel channel) throws IOException {
    
    
        String msg = new String(message.getBody());
        log.info("当前时间：{},死信队列A收到消息：{}", new Date().toString(), msg);
		channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
    }
}

3. Producer

In the control layer, RabbitTemplate is called to send messages to the delay queue.

@RestController
public class RabbitController {
    
    
    @Autowired
    RabbitTemplate rabbitTemplate;
     @GetMapping("/delayQueue")
    public void test(){
    
    
        rabbitTemplate.convertAndSend("delayExchange","delayQueueAroutingkey",LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE_TIME));
        System.out.println("延时队列已发送");
    }
}

test

The test was successful.

Reference article [RabbitMQ] takes you through RabbitMQ delay queue.
This article is very well written! pay tribute!
Finally, please pay attention to my official account and exchange and learn more Java experience and technology together.