Table of contents
1.1. Producer message confirmation (producer perspective)
1.2. Message persistence (message perspective)
1.3. Consumer message confirmation (consumer perspective)
1.4. Failure retry mechanism (processing mechanism after failure)
1. Message reliability
1.1. Producer message confirmation (producer perspective)
1.1.1. Theory
On the producer side, RabbitMQ provides a message confirmation mechanism to ensure that the producer's messages arrive in the queue.
Specifically, after the producer sends the message to MQ, a result will be returned to the producer, indicating whether the message was successfully processed. There are two specific responses:
- publish-confirm response
- The message is successfully delivered to the switch and ack is returned.
- If the message is not delivered to the switch (for example, the switch does not exist, or the switch name is written incorrectly), nack is returned.
- publish-return response
- If the message is delivered to the switch but is not routed to the queue (for example, the specified queue name is written incorrectly), an ack is returned, along with the reason for the routing failure.
Finally, after the callback on the producer side receives the response, it executes different "strategies" according to different acks (similar to when you go to buy a book, and then you decide what to do after you get the book).
Ps: When the confirmation mechanism sends a message, a globally unique id needs to be set for each message to distinguish different messages and avoid ack conflicts.
1.1.2. Practice
a) Add configuration in application.yml of publisher microservice:
spring:
rabbitmq:
publisher-confirm-type: correlated
publisher-returns: true
template:
mandatory: true
Configuration instructions:
- publish-confirm-type: Enable publisher-confirm. Two types are supported here.
- Simple (not recommended, similar to dead waiting, occupying resources): wait for the confirm result synchronously until timeout.
- Correlated (recommended): Asynchronous callback, define ConfirmCallback, MQ will call back this ConfirmCallback when returning the result.
- publish-returns: Turn on the publish-return function, which is also based on the callback mechanism, but defines ReturnCallback.
- template.mandatory: Defines the strategy when message routing fails. If true, ReturnCallback is called; false: The message is discarded directly.
b) Each RabbitTemplate can only be configured with one ReturnCallback , so it needs to be configured during the project startup process:
@Slf4j
@Configuration
public class CommonConfig implements ApplicationContextAware {
@Override
public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
// 获取RabbitTemplate
RabbitTemplate rabbitTemplate = applicationContext.getBean(RabbitTemplate.class);
// 设置ReturnCallback
rabbitTemplate.setReturnCallback((message, replyCode, replyText, exchange, routingKey) -> {
log.info("消息发送失败，应答码{}，原因{}，交换机{}，路由键{},消息{}",
replyCode, replyText, exchange, routingKey, message.toString());
});
}
}
Ps: ApplicationContextAware is the notification interface to be executed when the Spring container starts. Specific notifications are implemented through the setApplicationContext method.
c) The producer sends a message, specifies the ID, and the message ConfirmCallback
@Test
public void testSendMessage2SimpleQueue() throws InterruptedException {
// 消息体
String message = "hello, spring amqp!";
// 消息ID，需要封装到CorrelationData中
CorrelationData correlationData = new CorrelationData(UUID.randomUUID().toString());
// 添加callback
correlationData.getFuture().addCallback(
result -> {
if(result.isAck()){
// ack，消息成功
log.debug("消息发送成功, ID:{}", correlationData.getId());
}else{
// nack，消息失败
log.error("消息发送失败, ID:{}, 原因{}",correlationData.getId(), result.getReason());
}
},
ex -> log.error("消息发送异常, ID:{}, 原因{}",correlationData.getId(),ex.getMessage())
);
// 发送消息
rabbitTemplate.convertAndSend("amq.direct", "simple", message, correlationData);
}
1.2. Message persistence (message perspective)
1.2.1. Theory
MQ stores messages in memory by default. By turning on the persistence function (setting durable = true), you can persist messages to files to ensure that messages are not lost.
Ps: The prerequisite for messages to be persistent is that the switch (not necessarily, but preferably) and the queue are persistent.
1.2.2. Practice
a) Switch persistence
@Bean
public DirectExchange simpleExchange(){
// 三个参数：交换机名称、是否持久化、当没有queue与其绑定时是否自动删除
return new DirectExchange("simple.direct", true, false);
}
b) Queue persistence
@Bean
public Queue simpleQueue(){
// 使用QueueBuilder构建队列，durable就是持久化的
return QueueBuilder.durable("simple.queue").build();
}
c) Message persistence
public void testDurableMessage() {
//1.构造一个持久的消息
Message message = MessageBuilder.withBody("hello".getBytes())
.setDeliveryMode(MessageDeliveryMode.PERSISTENT)
.build();
rabbitTemplate.convertAndSend("simple.queue", message);
}
Ps: delivery_mode = 2 means that the message must be persisted.
1.3. Consumer message confirmation (consumer perspective)
1.3.1. Theory
RabbitMQ supports the consumer confirmation mechanism, that is, the consumer can send an ack receipt to MQ after processing the message, and MQ will delete the message after receiving the ack receipt.
SpringAMQP allows three confirmation modes to be configured:
- Manual: Manual ack, you need to call the API to send ack after the message code executed by the consumer ends.
- auto: automatic ack. Spring monitors whether there is an exception in the consumption code executed by the consumer. If there is no exception, ack will be returned; if an exception is thrown, nack will be returned. Then the message will be re-added to the queue and sent to the consumer, and then an exception will occur again. ...,Infinite loop.
- none: Turn off ack. MQ assumes that the consumer will successfully process the message after it obtains it, so the message will be deleted immediately after delivery.
1.3.2. Practice
Here you only need to configure the following application.yml file and add the following configuration:
spring:
rabbitmq:
listener:
simple:
prefetch: 1
acknowledge-mode: none # none，关闭ack；manual，手动ack；auto：自动ack
1.4. Failure retry mechanism (processing mechanism after failure)
1.4.1. Theory
As mentioned just now, SpringAMQP provides three confirmation modes for consumer consumption confirmation. Among them, auto mode, when the consumer encounters an exception when executing the consumption code, the message will be re-added to the queue and then sent to the consumer again. Exceptions and infinite loops cause mq's message processing to surge and bring unnecessary pressure.
Assume that the consumption task is as follows:
@Component
public class SpringRabbitListener {
@RabbitListener(queues = "simple.queue")
public void listenSimpleQueue(String msg) {
System.out.println("消费者接收到消息：" + msg);
System.out.println("开始消费！");
System.out.println(1/0);
System.out.println("消费完成！");
}
}
We can use Spring's retry mechanism to retry locally when an exception occurs in the consumer, instead of joining the mq queue without restrictions . We only need to configure the consumer's configuration file as follows:
spring:
rabbitmq:
listener:
simple:
prefetch: 1
retry:
enabled: true # 开启消费者失败重试
initial-interval: 1000 # 初始的失败等待时长为1秒
multiplier: 3 # 下次失败的等待时长倍数，下次等待时长 = multiplier * last-interval
max-attempts: 4 # 最大重试次数
stateless: true # true无状态；false有状态。如果业务中包含事务，这里改为false
After turning on the retry mode, if the number of retries is exhausted and the message still fails, the MessageRecoverer interface is required to handle it . It contains three different implementations:
- RejectAndDontRequeueRecoverer (default mode): After the retries are exhausted, directly reject and discard the message. This is the default
- ImmediateRequeueMessageRecoverer: After the retries are exhausted, nack is returned and the message is re-enqueued.
- RepublishMessageRecoverer (recommended method): After the retries are exhausted, the failed message is delivered to the specified switch, and then the switch delivers it to the specified queue.
The third method above is more recommended, as shown below:
1.4.2. Practice
Here we test the following recommended solution RepublishMessageRecoverer
a) First define the switch, queue, and binding relationship used to receive failed messages, and finally define the RepublishMessageRecoverer (injected in Bean mode, overriding Spring's default solution):
@Configuration
public class ErrorMessageConfig {
@Bean
public DirectExchange errorMessageExchange() {
return new DirectExchange("error.direct");
}
@Bean
public Queue errorQueue() {
return new Queue("error.queue", true);
}
@Bean
public Binding errorBinding() {
return BindingBuilder.bind(errorQueue()).to(errorMessageExchange()).with("error");
}
@Bean
public MessageRecoverer republishMessageRecoverer(RabbitTemplate rabbitTemplate) {
return new RepublishMessageRecoverer(rabbitTemplate, "error.direct", "error");
}
}
b) Define the consumption tasks performed by consumers
@Component
public class SpringRabbitListener {
@RabbitListener(queues = "simple.queue")
public void listenSimpleQueue(String msg) {
System.out.println("消费者接收到消息：" + msg);
System.out.println("开始消费！");
System.out.println(1/0);
System.out.println("消费完成！");
}
}
c) Start the consumer as follows:
d) Check the specific information in the failure queue (exception stack information and information information)
