Table of contents
1. springboot integrated release confirmation mechanism
1.1 Confirmation Mechanism Legend
In the production environment, due to some unknown reasons, RabbitMQ restarted. During the restart of RabbitMQ, the delivery of producer messages failed, resulting in message loss, which required manual processing and recovery. So how can RabbitMQ's message be delivered reliably? This article will introduce in detail how we deal with this kind of problem!
1. springboot integrated release confirmation mechanism
1.1 Confirmation Mechanism Legend
First, after the producer publishes the message, it is backed up in the cache. If the message is successfully published and confirmed to the switch, the message will be deleted from the cache; if the release fails, a scheduled task will be set to obtain the message from the cache again and publish it to the switch until it is successfully published. to the switch.
1.2 Example code
- A switch: confirm.exchange, the switch type is direct, and the routingKey associated with the queue is key1;
- A queue: confirm.queue;
- A consumer: confirm.consumer;
server:
port: 8280
spring:
rabbitmq:
host: 47.105.220.36
port: 5672
username: admin
password: admin
publisher-confirm-type: correlated
Added publisher-confirm-type: correlated on the original basis, let’s talk about the following parameters:
（1）The default is none , which disables publish confirmation mode(2) correlated is a callback method that will be triggered after the message is successfully published to the exchange
(3) simple has two effects: one is that the same as correlated , the callback method will be triggered after the message is successfully published;
The second is to use the rabbitTemplate to call the waitForConfirms or waitForConfirmsOrDie method to wait for the broker node to return the sending result after the message is successfully published, and determine the logic of the next step according to the returned result. The point to note is that if the waitForConfirmsOrDie method returns false, the channel will be closed, and the next Unable to send message to broker;
1. Write a configuration class: used to declare switches and queues, and bind switches and queues
/**
* desc：配置类，发布确认(高级)
*/
@Configuration
public class ConfirmConfig {
//交换机
public static final String CONFIRM_EXCHANGE_NAME = "confirm_exchange";
//队列
public static final String CONFIRM_QUEUE_NAME = "confirm_queue";
//routingKey
public static final String CONFIRM_ROUTING_KEY = "key1";
//声明交换机
@Bean("confirmExchange")
public DirectExchange confirmExchange(){
return new DirectExchange(CONFIRM_EXCHANGE_NAME);
}
//声明队列
@Bean("confirmQueue")
public Queue confirmQueue(){
return QueueBuilder.durable(CONFIRM_QUEUE_NAME).build();
}
//绑定
@Bean
public Binding queueBindingExchange(@Qualifier("confirmQueue") Queue confirmQueue,
@Qualifier("confirmExchange") DirectExchange confirmExchange){
return BindingBuilder.bind(confirmQueue).to(confirmExchange).with(CONFIRM_ROUTING_KEY);
}
}2. Producer (Controller layer)
/**
* desc:高级消息发布 消息生产者
*/
@Slf4j
@RequestMapping("/confirm")
@RestController
public class ProductController {
@Autowired
private RabbitTemplate rabbitTemplate;
//开始发消息,测试确认
@GetMapping("/sendMessage/{message}")
public void sendMessage(@PathVariable("message") String message){
//指定消息 id 为 1
CorrelationData correlationData1 = new CorrelationData("1");
rabbitTemplate.convertAndSend(ConfirmConfig.CONFIRM_EXCHANGE_NAME,
ConfirmConfig.CONFIRM_ROUTING_KEY,message+"key1",correlationData1);
log.info("发送消息内容:{}",message+"key1");
//指定消息 id 为 2
CorrelationData correlationData2 = new CorrelationData("2");
String CONFIRM_ROUTING_KEY = "key2"; //这里routing_key我们故意设置错误
rabbitTemplate.convertAndSend(ConfirmConfig.CONFIRM_EXCHANGE_NAME,
CONFIRM_ROUTING_KEY,message+"key2",correlationData2);
log.info("发送消息内容:{}",message+"key2");
}
}3. Consumer (listening confirm.queue queue)
/**
* desc:接受消息
*/
@Slf4j
@Component
public class Consumer {
@RabbitListener(queues = ConfirmConfig.CONFIRM_QUEUE_NAME)
public void receiveConfirmMessage(Message message){
String msg = new String(message.getBody());
log.info("接受到的队列confirm.queue消息:{}",msg);
}
}4. The callback interface after the message producer publishes the message
As long as the producer publishes a message, the switch will call confirm the method of this class regardless of whether it receives the message or not.
/**
* desc:回调接口
*/
@Slf4j
@Component
public class MyCallBack implements RabbitTemplate.ConfirmCallback {
//注入
@Autowired
private RabbitTemplate rabbitTemplate;
@PostConstruct
public void init(){
//注入
rabbitTemplate.setConfirmCallback(this);
}
/**
* 交换机不管是否收到消息都调用 回调方法
* 1. 发消息 交换机接收到了 回调
* @param correlationData 保存回调信息的Id及相关信息
* @param ack 交换机收到消息 为true
* @param cause 未收到消息的原因
*
*/
@Override
public void confirm(CorrelationData correlationData, boolean ack, String cause) {
String id = correlationData!=null?correlationData.getId():"";
if(ack){
log.info("交换机已经收到了ID为:{}的消息",id);
}else {
log.info("交换机还未收到ID为:{}的消息，由于原因:{}",id,cause);
}
}
}5. Result analysis
Visit URL: http://localhost:8280/confirm/sendMessage/ Hello
It can be seen that the producer has sent two messages, the RoutingKey of the first message is "key1", and the RoutingKey of the second message is "key2". Both messages have been successfully received by the switch, and the confirmation callback of the switch has also been received. , but the consumer only receives one message, because the RoutingKey of the second message is inconsistent with the BindingKey of the queue, and no other queue can receive this message, all the second messages are discarded directly.
Discarded messages are unknown to the exchange and need to be resolved to tell the producer that the message delivery failed.
2. Rollback message
When only the producer confirmation mechanism is enabled, the switch will directly send a confirmation message to the message producer after receiving the message. If the message is found to be unroutable, the message will be discarded directly. At this time, the producer does not know the message was dropped for this event.
So how do I get messages that cannot be routed to help me find a way to deal with them? At least let me know, so I can handle it myself. By setting the mandatory parameter, the message can be returned to the producer when the destination is unreachable during message delivery.
2.1 Introduction
To get the rollback message, first enable this function in the configuration file, and then need a custom class to implement RabbitTemplate.ReturnsCallback the interface , and when initializing, use the custom class as the processing class of the rollback message, enable it at the same time Mandatory, and set it to true.
1. Open it in the configuration file or open it in the code
spring:
rabbitmq:
template:
mandatory: truerabbitTemplate.setMandatory(true);2. Modify the callback interface
/**
* desc:回调接口
*/
@Slf4j
@Component
public class MyCallBack implements RabbitTemplate.ConfirmCallback,RabbitTemplate.ReturnsCallback {
//注入
@Autowired
private RabbitTemplate rabbitTemplate;
@PostConstruct
public void init(){
//注入
rabbitTemplate.setConfirmCallback(this);
rabbitTemplate.setReturnsCallback(this);
}
/**
* 交换机不管是否收到消息都调用回调方法
* 1. 发消息 交换机接收到了 回调
* @param correlationData 保存回调信息的Id及相关信息
* @param ack 交换机收到消息 为true
* @param cause 未收到消息的原因
*
*/
@Override
public void confirm(CorrelationData correlationData, boolean ack,String cause) {
String id = correlationData!=null?correlationData.getId():"";
if(ack){
log.info("交换机已经收到了ID为:{}的消息",id);
}else {
log.info("交换机还未收到ID为:{}的消息，由于原因:{}",id,cause);
}
}
//可以在当消息传递过程中不可达目的地时将消息返回给生产者
//该方法只有不可达目的地的时候 才会被触发进行回退
/**
* 当消息无法路由的时候的回调方法
* message 消息
* replyCode 编码
* replyText 退回原因
* exchange 从哪个交换机退回
* routingKey 通过哪个路由 key 退回
*/
@Override
public void returnedMessage(ReturnedMessage returned) {
log.error("消息{},被交换机{}退回，退回原因:{},路由key:{}",
new String(returned.getMessage().getBody()),returned.getExchange(),
returned.getReplyText(),returned.getRoutingKey());
}
}3. Effect demonstration
Visit URL: http://localhost:8280/confirm/sendMessage/ Hello
With the mandatory parameter and the fallback message, we gain the ability to perceive undeliverable messages, and have the opportunity to discover and process when the producer's message cannot be delivered.
But sometimes, we don't know how to deal with these messages that cannot be routed. At most, we can make a log, then trigger an alarm, and then handle it manually. It is very inelegant to process these unroutable messages through logs, especially when the producer's service has multiple machines, manually copying logs will be more troublesome and error-prone. At the same time, setting the mandatory parameter will increase the complexity of the producer, and it is necessary to add logic to handle these returned messages. How to do it if you don't want to lose messages and don't want to increase the complexity of the producer?
Reminiscent of when setting up a dead letter queue, we will be able to set a dead letter switch for the queue to store messages that fail to process, but these unroutable messages have no chance to enter the queue at all, so the dead letter queue cannot be used to save messages. In RabbitMQ, there is a backup switch mechanism that can deal with this problem well. What is a backup switch?
3. Backup switch
The backup switch can be understood as the "spare tire" of the switch in RabbitMQ. When we declare a corresponding backup switch for a switch, we create a backup switch for it. When the switch receives a non-routable message, it will pass this The message is forwarded to the backup switch, which is forwarded and processed by the backup switch. Usually the type of the backup switch is Fanout, so that all messages can be delivered to the queue bound to it, and then we bind a queue, so that all messages that cannot be routed by the original switch will enter this queue. Of course, we can also build an alarm queue and use independent consumers to monitor and alarm. Next, let's actually do it:
3.1 Code Architecture Diagram
3.2 Example code
1. Modify the configuration class of advanced confirmation release (add backup related settings)
/**
* desc：配置类，发布确认(高级)
*/
@Configuration
public class ConfirmConfig {
//交换机
public static final String CONFIRM_EXCHANGE_NAME = "confirm_exchange";
//队列
public static final String CONFIRM_QUEUE_NAME = "confirm_queue";
//routingKey
public static final String CONFIRM_ROUTING_KEY = "key1";
//关于备份的相关操作
//备份交换机
public static final String BACKUP_EXCHANGE_NAME = "backup_exchange";
//备份队列
public static final String BACKUP_QUEUE_NAME = "backup_queue";
//报警队列
public static final String WARNING_QUEUE_NAME = "warning_queue";
//声明交换机,并且设置该交换机的备份交换机
@Bean("confirmExchange")
public DirectExchange confirmExchange(){
return ExchangeBuilder.directExchange(CONFIRM_EXCHANGE_NAME)
.durable(true).withArgument("alternate-exchange",BACKUP_EXCHANGE_NAME).build();
}
//声明队列
@Bean("confirmQueue")
public Queue confirmQueue(){
return QueueBuilder.durable(CONFIRM_QUEUE_NAME).build();
}
//绑定
@Bean
public Binding queueBindingExchange(@Qualifier("confirmQueue") Queue confirmQueue,
@Qualifier("confirmExchange") DirectExchange confirmExchange){
return BindingBuilder.bind(confirmQueue).to(confirmExchange).with(CONFIRM_ROUTING_KEY);
}
//声明备份交换机
@Bean("backupExchange")
public FanoutExchange backupExchange(){
return new FanoutExchange(BACKUP_EXCHANGE_NAME);
}
//声明备份队列
@Bean("backupQueue")
public Queue backupQueue(){
return QueueBuilder.durable(BACKUP_QUEUE_NAME).build();
}
//声明报警队列
@Bean("warningQueue")
public Queue warningQueue(){
return QueueBuilder.durable(WARNING_QUEUE_NAME).build();
}
//绑定 备份队列绑定备份交换机
@Bean
public Binding backupQueueBindingBackupExchange(@Qualifier("backupQueue") Queue backupQueue,
@Qualifier("backupExchange") FanoutExchange backupExchange){
return BindingBuilder.bind(backupQueue).to(backupExchange);
}
//绑定 报警队列绑定备份交换机
@Bean
public Binding warningQueueBindingBackupExchange(@Qualifier("warningQueue") Queue warningQueue,
@Qualifier("backupExchange") FanoutExchange backupExchange){
return BindingBuilder.bind(warningQueue).to(backupExchange);
}
}2. Alert consumers
/**
* decs:报警消费者
*/
@Slf4j
@Component
public class WarningConsumer {
//接收报警信息
@RabbitListener(queues = ConfirmConfig.WARNING_QUEUE_NAME)
public void receiveWarningMsg(Message message){
String msg = new String(message.getBody());
log.error("报警发现不可路由消息:{}",msg);
}
}Since confirm.exchange the switch has been written before, when changing the configuration, the original one needs to be deleted, otherwise an error will be reported.
Access address: http://localhost:8280/confirm/sendMessage/ Hello
When the Mandatory parameter and the backup switch can be used together, if both are enabled at the same time, where will the message go? Who has the higher priority? The above results show that the answer is that the backup switch has a higher priority.