One, what is the dead letter queue
Dead letter, the corresponding word on the official website is "Dead Letter", it can be seen that the translation is indeed very simple and rude. So what is a dead letter?
"Dead letter" is a message mechanism in RabbitMQ. When you are consuming messages, if the message in the queue appears in the following situation
1, the message is negatively confirmed, use channel.basicNack or channel.basicReject, and the request attribute is Set to false.
2. The survival time of the message in the queue exceeds the set TTL time.
3. The number of messages in the message queue has exceeded the maximum queue length.
Then the news will become a "dead letter."
"Dead letter" messages will be specially processed by RabbitMQ. If the dead letter queue information is configured, the message will be thrown into the dead letter queue. If it is not configured, the message will be discarded.
Second, how to configure the dead letter queue
It can be roughly divided into the following steps:
1. Configure the service queue and bind it to the service switch
2. Configure the dead letter switch and routing key for the service queue
3. Configure the dead letter queue for the dead letter switch
Note that it is not directly declaring a public dead letter queue, and then all dead letter messages go to the dead letter queue by themselves. Instead, a dead letter switch is configured for each service queue that needs to use dead letters. Here, the dead letter switches of the same project can share one, and then a separate routing key is assigned to each service queue.
With the dead letter switch and routing key, next, just like configuring the service queue, configure the dead letter queue, and then bind it to the dead letter switch. In other words, the dead letter queue is not a special queue, but a queue bound to the dead letter switch. The dead letter switch is not a special switch, it is only a switch used to accept dead letters, so it can be any type [Direct, Fanout, Topic]. Generally speaking, each service queue is assigned a unique routing key, and a dead letter queue is configured to monitor accordingly, that is, a dead letter queue is generally configured for each important service queue.
The deployment and construction of the RabbitMQ environment is omitted here, and the direct connection switch is taken as an example.
(1) Provider
1. Pom introduces GA coordinates
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.4.1</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.mq</groupId>
<artifactId>rabbitmq-consumer</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>rabbitmq-consumer</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.amqp</groupId>
<artifactId>spring-rabbit-test</artifactId>
<scope>test</scope>
</dependency>
<!-- 糊涂工具包-->
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.5.2</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<excludes>
<exclude>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</project>
2. Modify the yml file
server:
port: 8001
spring:
application:
name: rabbitma-provider
#rabbitmq配置
rabbitmq:
host: localhost
port: 5672
username: guest
password: guest
virtual-host: LCHost #虚拟host 可以不设置,使用server默认host
publisher-confirm-type: correlated #确认消息已发送到交换机 必须配置这个才会确认回调
publisher-returns: true #确认消息已发送到队列
listener:
simple:
# retry:
# enabled: true #开启重试
# max-attempts: 6 #最大重试间隔
# initial-interval: 3000 #重试间隔
default-requeue-rejected: false
acknowledge-mode: manual
Remember to set the default-requeue-rejected attribute to false here.
3. Add two config classes, a dead letter queue and a business queue
Dead letter switch and queue
package com.mq.rabbitmqprovider.config;
import com.rabbitmq.client.AMQP;
import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.DirectExchange;
import org.springframework.amqp.core.Queue;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
/**
* 死信队列
* @author LiuCheng
* @date 2020/12/30 14:13
*/
@Configuration
public class DeadRabbitMQConfig {
/**
* 死信交换机
*/
public static final String DEAD_LETTER_EXCHANGE = "deadLetterExchange";
/**
* 死信队列路由KEY
*/
public static final String DEAD_LETTER_ROUTING_KEYA = "deadLetterRoutingkeyA";
/**
* 死信队列名称
*/
public static final String DEAD_LETTER_QUEUEA = "deadLetterQueueA";
@Bean
public DirectExchange deadExchange(){
return new DirectExchange(DEAD_LETTER_EXCHANGE);
}
@Bean
public Queue deadQueue(){
return new Queue(DEAD_LETTER_QUEUEA);
}
@Bean
public Binding bindingDeadLetterExchangeA(){
return BindingBuilder.bind(deadQueue()).to(deadExchange()).with(DEAD_LETTER_ROUTING_KEYA);
}
}
Business switches and queues
package com.mq.rabbitmqprovider.config;
import org.springframework.amqp.core.*;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.HashMap;
import java.util.Map;
/**
* 业务交换机以及队列
* @author LiuCheng
* @data 2020/12/30 14:31
*/
@Configuration
public class BusinessRabbitMQConfig {
public static final String BUSINESS_EXCHANGE = "businessExchange";
public static final String BUSINESS_QUEUE = "businessQueue";
public static final String BUSINESS_ROUTE_KEY = "businessRouteKey";
@Bean
public DirectExchange businessExchange(){
return new DirectExchange(BUSINESS_EXCHANGE);
}
@Bean
public Queue businessQueue(){
Map<String,Object> map =new HashMap<>();
//这里声明当前队列绑定的死信交换机
map.put("x-dead-letter-exchange",DeadRabbitMQConfig.DEAD_LETTER_EXCHANGE);
//这里声明当前队列的死信路由key
map.put("x-dead-letter-routing-key",DeadRabbitMQConfig.DEAD_LETTER_ROUTING_KEYA);
return QueueBuilder.durable(BUSINESS_QUEUE).withArguments(map).build();
// return QueueBuilder.durable().withArguments(map).build();
}
@Bean
public Binding bindBusinessExchange(){
return BindingBuilder.bind(businessQueue()).to(businessExchange()).with(BUSINESS_ROUTE_KEY);
}
}
4. Control layer
package com.mq.rabbitmqprovider.controller;
import cn.hutool.core.date.DateUtil;
import cn.hutool.json.JSONNull;
import cn.hutool.json.JSONObject;
import cn.hutool.json.JSONUtil;
import com.mq.rabbitmqprovider.config.BusinessRabbitMQConfig;
import com.mq.rabbitmqprovider.config.DeadRabbitMQConfig;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
/**
* @author LiuCheng
* @data 2020/12/30 15:03
*/
@RestController
@RequestMapping("/mq/business")
public class BusinessController {
@Autowired
private RabbitTemplate rabbitTemplate;
@GetMapping("/send")
public String send(String msg){
String messageId= String.valueOf(UUID.randomUUID());
String creatTime= DateUtil.now();
Map<String,Object> map = new HashMap<>();
map.put("messageId",messageId);
map.put("messgageName",msg);
map.put("creatTime",creatTime);
rabbitTemplate.convertAndSend(BusinessRabbitMQConfig.BUSINESS_EXCHANGE,BusinessRabbitMQConfig.BUSINESS_ROUTE_KEY, JSONUtil.toJsonStr(map));
return "ok";
}
}
Start the provider, check the data in the queue list and find that there are two more queues, one for the businessQueue and one for the dead letter queue deadLetterQueueA
Second, consumers
The pom file and the yml file are the same as the provider, except that the port has changed
1. Business queue consumers
package com.mq.rabbitmqconsumer.rabbimqConsumer;
import cn.hutool.json.JSONUtil;
import com.rabbitmq.client.Channel;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitHandler;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;
import java.io.IOException;
import java.util.Map;
/**
* 业务消费者
* @author LiuCheng
* @data 2020/12/30 15:13
*/
@Component
public class BusinessReceiver {
@RabbitListener(queues = "businessQueue")
public void process(Message message, Channel channel) throws IOException {
try {
// int i=10/0;
System.out.println("业务消费者接受到的信息是 : "+ JSONUtil.parse( new String (message.getBody(),"UTF-8")));
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
} catch (Exception e) {
System.out.println("业务消费者发生异常");
channel.basicNack(message.getMessageProperties().getDeliveryTag(),false,false);
e.printStackTrace();
}
}
}
Start the consumer, request the provider's business class interface, and the consumption is normal. Now
let go of the exception int i=10/0, send the request again, the console reports an error, and there will be a message in the dead letter queue
2. Dead letter queue consumers
package com.mq.rabbitmqconsumer.rabbimqConsumer;
import cn.hutool.json.JSONUtil;
import com.rabbitmq.client.Channel;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;
import java.io.IOException;
import java.util.Map;
/**
* 死信队列消费者
*
* @author LiuCheng
* @data 2020/12/30 15:47
*/
@Component
public class DeadReceiver {
@RabbitListener(queues = "deadLetterQueueA")
public void process(Message message, Channel channel) throws IOException {
System.out.println("死信队列消费者 : "+ JSONUtil.parse( new String (message.getBody(),"UTF-8")));
// 必须手动确认否则消息一直存在队列中，每次启动会被消费一次
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}
}
After startup, it is found that the messages in the dead letter queue are consumed. Then
look at the request sent to the provider, the consumer is abnormal, and the dead letter queue consumes the process
业务消费者发生异常
java.lang.ArithmeticException: / by zero
at com.mq.rabbitmqconsumer.rabbimqConsumer.BusinessReceiver.process(BusinessReceiver.java:25)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:171)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120)
at org.springframework.amqp.rabbit.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:53)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:221)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandlerAndProcessResult(MessagingMessageListenerAdapter.java:149)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:134)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1632)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1551)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer$$Lambda$499/11521473.invokeListener(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:93)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:329)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:225)
at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:116)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at org.springframework.amqp.rabbit.listener.$Proxy68.invokeListener(Unknown Source)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1539)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1530)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1474)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:966)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:912)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:83)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.mainLoop(SimpleMessageListenerContainer.java:1287)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1193)
at java.lang.Thread.run(Thread.java:745)
死信队列消费者 : {
"messageId":"382eccb7-0870-4ac5-910f-21b9ca4498de","creatTime":"2020-12-30 17:00:17","messgageName":"hello"}
As you can see, the Consumer of the dead letter queue has received this message, so the process ends here.
Third, the changes in dead letter news
So what happens after the "dead letter" is dropped into the dead letter queue?
If the queue is configured with the parameter x-dead-letter-routing-key, the routing key of the "dead letter" will be replaced with the value corresponding to this parameter. If it is not set, the original routing key of the message is retained.
For example,
if the routing key of the original message is testA, it is sent to the business Exchage, and then delivered to the business queue QueueA. If the queue does not have the configuration parameter x-dead-letter-routing-key, the message becomes After the dead letter, the original routing key testA will be retained. If this parameter is configured and the value is set to testB, then after the message becomes a dead letter, the routing key will be replaced with testB and then thrown to the dead letter switch.
In addition, because it was thrown to the dead letter exchange, the Exchange Name of the message will also be replaced with the name of the dead letter exchange.
In the header of the message, many weird fields will also be added. Modify the above code to add a line of log output to the consumers of the dead letter queue:
log.info("死信消息properties：{}", message.getMessageProperties());
Then run it again to get the information added in the dead letter message Header
在这里插入代码片死信消息properties：MessageProperties [headers={
x-first-death-exchange=dead.letter.demo.simple.business.exchange, x-death=[{
reason=rejected, count=1, exchange=dead.letter.demo.simple.business.exchange, time=Sun Jul 14 16:48:16 CST 2019, routing-keys=[], queue=dead.letter.demo.simple.business.queuea}], x-first-death-reason=rejected, x-first-death-queue=dead.letter.demo.simple.business.queuea}, correlationId=1, replyTo=amq.rabbitmq.reply-to.g2dkABZyYWJiaXRAREVTS1RPUC1DUlZGUzBOAAAPQAAAAAAB.bLbsdR1DnuRSwiKKmtdOGw==, contentType=text/plain, contentEncoding=UTF-8, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, redelivered=false, receivedExchange=dead.letter.demo.simple.deadletter.exchange, receivedRoutingKey=dead.letter.demo.simple.deadletter.queuea.routingkey, deliveryTag=1, consumerTag=amq.ctag-NSp18SUPoCNvQcoYoS2lPg, consumerQueue=dead.letter.demo.simple.deadletter.queuea]
There seems to be a lot of information in the header, but in fact there is not much, but the value is relatively long. Here is a brief description of the value in the Header:
x-first-death-exchange: the name of the dead letter exchange that was thrown into the first time
x-first-death-reason: It is the first reason for a dead letter, rejected: the message was rejected by the queue when it re-entered the queue, because the default-requeue-rejected parameter was set to false. expired: The message has expired. maxlen: The number of messages in the queue exceeds the maximum capacity of the queue
x-first-death-queue: The name of the queue before becoming a dead letter for the first time
x-death: The list of information that has been put into a dead letter exchange. Each time the same message enters a dead letter exchange, the information in this array will be updated
Fourth, the application scenario of dead letter queue
With the above information, we already know how to use the dead letter queue, so in what scenarios is the dead letter queue generally used?
Generally used in more important business queues to ensure that messages that are not correctly consumed are not discarded. Generally, the possible causes of consumption exceptions are processing exceptions due to errors in the message information itself, abnormal parameter verification during processing, or due to network Query exceptions caused by fluctuations, etc. When an exception occurs, of course, you can't get the original message through the log every time, and then let the operation and maintenance help to re-deliver the message (yes, it was done before ==). By configuring the dead letter queue, you can temporarily store the incorrectly processed messages in another queue. After the subsequent troubleshooting is clear, you can write the corresponding processing code to process the dead letter messages, which is much better than manually recovering the data.
Five, summary
There is actually no mystery about the dead letter queue, it is just an ordinary queue bound to the dead letter exchange, and the dead letter exchange is just an ordinary exchange, but a special exchange for handling dead letters.
Summarize the life cycle of dead letter messages:
1. The business message is put into the business queue
2. The consumer consumes the message in the business queue. Due to an abnormality in the processing, the nck or reject operation is performed.
3. The message that is nck or rejected is delivered to the dead letter exchange by RabbitMQ
4. The dead letter exchange puts the message into the corresponding dead letter queue
5. The consumers of the dead letter queue consume the dead letter messages
The dead letter message is a guarantee made by RabbitMQ. In fact, we can also not use the dead letter queue. Instead, when the message consumption is abnormal, the message will be actively delivered to another exchange. After you understand this, these exchanges You can cooperate with Queue as you want. For example, pull messages from the dead letter queue, and then send emails, text messages, and nail notifications to notify developers of their attention. Or re-deliver the message to a queue and set the expiration time to delay consumption.