Rabbitmq dead letter queue implements delayed messages

Get into the habit of writing together! This is the 11th day of my participation in the "Nuggets Daily New Plan · April Update Challenge"

foreword

In e-commerce development, we often encounter such functional requirements. For example, when an order is not paid after 30 minutes, the order needs to be closed; when an order is not clicked to confirm receipt after 7 days, the system will automatically confirm it. We can summarize this type of task as a system event that is executed by triggering a certain time condition. We can put aside the middleware of rabbitmq and think about how to design this task.

First of all, the triggering of these events should not affect our main process. For example, the user generates an order, but even if the user does not pay, he can still do other things, such as browsing products or participating in other promotions. That is, this type of event is independent, non-blocking, and asynchronous relative to our user main thread. So can we implement this event by starting another thread? We can design a thread pool and use delayed threads to execute events.

But this seems to have some problems. For example, if the event time triggered by our task is relatively long, then each small task needs a thread resource to maintain, then our thread pool needs to consider the thread resource occupation and rejection strategy. In addition, due to this design, our threads are completely bound to the java service. Under normal conditions, if the thread is shut down or restarted, these threads in memory will be lost and destroyed.

Therefore, we need an asynchronous thread-like design, which preferably runs independently of our Java service, which can reduce the harm caused by downtime and restart. So we thought of the message queue rabbitmq, which delivers the events that need to be executed after a period of time to the queue, and then accepts the messages of the queue after a period of time.

text

  In rabbitmq, dead letter queues are generally used to implement the function of delay queues

​ 

First of all, what is a dead letter queue, here we need to understand the concept of TTL in rabbitmq.

Generally speaking, TTL means that the expiration time is the same as the life cycle of a cache in redis. In rabbitmq, we allow us to set TTL on messages and queues, mainly by setting x-message-ttl. When applied to a queue, it is effective for all messages in the queue

Map<String, Object> args = new HashMap<String, Object>();
args.put("x-message-ttl", 60000);
channel.queueDeclare("myqueue", false, false, false, args);
复制代码

上面就代表的是一个设置消息过期时间为60s的队列，那么所有这个队列的消息都只能在队列中存储最多60s，超过则被丢弃。 我们也可以对单独的消息设置过期时间

byte[] messageBodyBytes = "Hello, world!".getBytes();
AMQP.BasicProperties properties = new AMQP.BasicProperties.Builder()
                                   .expiration("60000")
                                   .build();
channel.basicPublish("my-exchange", "routing-key", properties, messageBodyBytes);
复制代码

上面表示这条消息在队列中最多存储60秒，如果没有消费者消费，则会被丢弃。 如果把过期参数设置在队列上，那么超过过期时间，这个队列就会自动删除。

Map<String, Object> args = new HashMap<String, Object>();
args.put("x-expires", 1800000);
channel.queueDeclare("myqueue", false, false, false, args);
复制代码

上面表示这个队列在30m后会被过期删除。

下面接着死信队列来讲，在rabbitmq中超过ttl的消息叫做死信，那么一个都是死信消息的队列则被叫做死信队列。所以延时队列就是利用了这个特性。首先声明一个死信队列，这个队列不能有消费者监听，这样可以保证一到过期时间，消息会被丢弃。接下来我们要做的就是接受这些死信重新把它们投递到队列中消费。

rabbitmq提供了一种Dead Letter Exchanges，死信交换，它的作用就是“死信”的重发布。也就是当消息过期时，消息并不是简单丢弃，而是重新投递到一个交换器中进行新的消费。rabbitmq给我们提供了两个参数用于重发布。分别是x-dead-letter-exchange这个是重新投递的交换器，x-dead-letter-routing-key这个是重新投递的路由，可以指向到具体的队列，方便我们做监听。

下面是延时消息的代码设计。

/*
 * @param
 * @return org.springframework.amqp.core.TopicExchange
 * @description 定义一个死信交换器
 * @author zhou
 * @create 2021/7/12 12:27
 **/
@Bean
public TopicExchange deadExchange() {
    return new TopicExchange("dead_exchange", false, false);
}

/**
 * @param
 * @return org.springframework.amqp.core.Queue
 * @description 死信队列
 * @author zhou
 * @create 2021/7/12 12:31
 **/
@Bean
public Queue deadQueue() {
    HashMap<String, Object> args = new HashMap<>();
    //要发送的交换器
    args.put("x-dead-letter-exchange", "dlx_exchange");
    //路由键
    args.put("x-dead-letter-routing-key", "web.expire");
    return new Queue("dead_queue", false, false, false, args);
}

/**
 * @param
 * @return org.springframework.amqp.core.Binding
 * @description 建立绑定关系
 * @author zhou
 * @create 2021/7/12 12:38
 **/
@Bean
public Binding deadBinding(TopicExchange deadExchange, Queue deadQueue) {
    return BindingBuilder
            .bind(deadQueue)
            .to(deadExchange)
            .with("dlx");
}
复制代码

首先我们声明一个普通的交换器，一个死信队列（记住这个队列没有消费者监听），队列中声明了死信交换出去的指定交换器和指定的路由。然后将这个普通交换器和死信队列通过一个路由绑定。

public void delayInfo(String message) {
    LocalDateTime now = LocalDateTime.now();
    rabbitTemplate.convertAndSend("dead_exchange", "dlx", message,
            m -> {
                m.getMessageProperties().setExpiration(String.valueOf(5000));
                return m;
            }
    );
    log.info("发送延时时间:[{}]", now.toString());
}
复制代码

这里投递了一条标记TTL的消息，我们设置为5S。同时指定它投递到我们之前声明的普通交换器和死信队列中。

@RabbitListener(bindings = @QueueBinding(
        value = @Queue(value = "dlx_queue", durable = "false", autoDelete = "true", exclusive = "false"),
        exchange = @Exchange(value = "dlx_exchange", type = ExchangeTypes.TOPIC),
        key = {"web.expire"}
))
public void delayListener(String message) {
    log.info("delay info:[{}]", message);
}
复制代码

这里是我们的消费者部分。首先我们需要声明一个交换器，它的名称要和之前x-dead-letter-exchange中写的一致。然后我们要声明一个队列，队列的名字可自定义，但是队列和交换器之间绑定的路由需要和x-dead-letter-routing-key中写的一致。

下面我们发送一条消息,可以看到下面的日志输出，之间间隔的时间差不多是5s。

Such a simple rabbitmq delay queue implemented by dead letter queue is completed.