During the use of message queues, there are many practical problems that need to be considered:

1. Message reliability

From sending the message to receiving it by the consumer, it will manage multiple processes:

Each of these steps can lead to message loss, common reasons for loss include:

Lost on send:
- The message sent by the producer was not delivered to the exchange
- The message did not reach the queue after it arrived at the exchange
MQ is down, the queue will lose the message
The consumer crashes without consuming the message after receiving the message

For these problems, RabbitMQ gives solutions respectively:

Producer Confirmation Mechanism
mq persistence
Consumer Confirmation Mechanism
Failure retry mechanism

2. Producer message confirmation

RabbitMQ provides a publisher confirm mechanism to avoid loss of messages sent to MQ. This mechanism must assign a unique ID to each message. After the message is sent to MQ, a result will be returned to the sender, indicating whether the message is processed successfully.

There are two ways to return results:

publisher-confirm, the sender confirms
- The message is successfully delivered to the switch and returns ack
- The message was not delivered to the exchange, return nack
publisher-return, sender receipt
- The message was delivered to the exchange, but not routed to the queue. Return ACK and the reason for routing failure.

2.1. Modify configuration

First, modify the application.yml file in the publisher service and add the following content:

spring:
  rabbitmq:
    publisher-confirm-type: correlated
    publisher-returns: true
    template:
      mandatory: true

illustrate:

publish-confirm-type: Enable publisher-confirm, here supports two types:
- simple: Synchronously wait for the confirm result until timeout
- correlated: Asynchronous callback, define ConfirmCallback, this ConfirmCallback will be called back when MQ returns the result
publish-returns: Enable the publish-return function, which is also based on the callback mechanism, but defines the ReturnCallback
template.mandatory: Defines the policy when message routing fails. true, call ReturnCallback; false: discard the message directly

2.2. Define the Return callback

Each RabbitTemplate can only configure one ReturnCallback, so it needs to be configured when the project is loaded:

Modify the publisher service and add one:

@Slf4j
@Configuration
public class CommonConfig implements ApplicationContextAware {
    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        // 获取RabbitTemplate
        RabbitTemplate rabbitTemplate = applicationContext.getBean(RabbitTemplate.class);
        // 设置ReturnCallback
        rabbitTemplate.setReturnCallback((message, replyCode, replyText, exchange, routingKey) -> {
            // 投递失败，记录日志
            log.info("消息发送失败，应答码{}，原因{}，交换机{}，路由键{},消息{}",
                     replyCode, replyText, exchange, routingKey, message.toString());
            // 如果有业务需要，可以重发消息
        });
    }
}

2.3. Define ConfirmCallback

ConfirmCallback can be specified when sending a message, because the logic of each business processing confirm success or failure is not necessarily the same.

In the cn.itcast.mq.spring.SpringAmqpTest class of the publisher service, define a unit test method:

public void testSendMessage2SimpleQueue() throws InterruptedException {
    // 1.消息体
    String message = "hello, spring amqp!";
    // 2.全局唯一的消息ID，需要封装到CorrelationData中
    CorrelationData correlationData = new CorrelationData(UUID.randomUUID().toString());
    // 3.添加callback
    correlationData.getFuture().addCallback(
        result -> {
            if(result.isAck()){
                // 3.1.ack，消息成功
                log.debug("消息发送成功, ID:{}", correlationData.getId());
            }else{
                // 3.2.nack，消息失败
                log.error("消息发送失败, ID:{}, 原因{}",correlationData.getId(), result.getReason());
            }
        },
        ex -> log.error("消息发送异常, ID:{}, 原因{}",correlationData.getId(),ex.getMessage())
    );
    // 4.发送消息
    rabbitTemplate.convertAndSend("task.direct", "task", message, correlationData);

    // 休眠一会儿，等待ack回执
    Thread.sleep(2000);
}

3. Message Persistence

The producer confirms that the message can be delivered to the RabbitMQ queue, but after the message is sent to RabbitMQ, if there is a sudden downtime, the message may also be lost.

To ensure that messages are safely stored in RabbitMQ, the message persistence mechanism must be enabled.

switch persistence
queue persistence
message persistence

3.1, switch persistence

The switch in RabbitMQ is non-persistent by default, and it will be lost after mq restarts.

In SpringAMQP, switch persistence can be specified by code:

@Bean
public DirectExchange simpleExchange(){
    // 三个参数：交换机名称、是否持久化、当没有queue与其绑定时是否自动删除
    return new DirectExchange("simple.direct", true, false);
}

In fact, by default, switches declared by Spring AMQP are persistent.

You can see the mark on the persistent switch in the RabbitMQ console D:

3.2, queue persistence

The queue in RabbitMQ is non-persistent by default, and it will be lost after mq restarts.

In SpringAMQP, switch persistence can be specified by code:

@Bean
public Queue simpleQueue(){
    // 使用QueueBuilder构建队列，durable就是持久化的
    return QueueBuilder.durable("simple.queue").build();
}

In fact, by default, queues declared by Spring AMQP are persistent.

You can see the mark on the persistent queue in the RabbitMQ console D:

3.3, message persistence

When using SpringAMQP to send messages, you can set the properties of the message (MessageProperties) and specify the delivery-mode:

non-persistent
Persistence

Specify with java code:

By default, any message sent by Spring AMQP is persistent, without specifying it.

4. Consumer message confirmation

RabbitMQ is a mechanism that burns after reading . RabbitMQ confirms that the message will be deleted immediately after being consumed by the consumer.

RabbitMQ confirms whether the consumer has successfully processed the message through the consumer receipt: after the consumer obtains the message, it should send an ACK receipt to RabbitMQ to indicate that it has processed the message.

Imagine this scenario:

1) RabbitMQ delivers messages to consumers
2) After the consumer gets the message, it returns ACK to RabbitMQ
3) RabbitMQ delete message
4) The consumer is down, and the message has not been processed

In this way, the message is lost. Therefore, the timing of the consumer returning ACK is very important.

SpringAMQP allows configuration of three confirmation modes:

manual: manual ack, you need to call the api to send ack after the business code ends.
auto: automatic ack, the listener code is monitored by spring to see if there is an exception, if there is no exception, it will return ack; if an exception is thrown, it will return nack
none: close ack, MQ assumes that the consumer will successfully process the message after getting it, so the message will be deleted immediately after delivery

From this we can see:

In none mode, message delivery is unreliable and may be lost
The auto mode is similar to the transaction mechanism. When an exception occurs, it returns nack, and the message is rolled back to mq; if there is no exception, it returns ack
manual: According to the business situation, judge when to ack

Generally, we can use the default auto.

4.1, demo none mode

Modify the application.yml file of the consumer service and add the following content:

spring:
  rabbitmq:
    listener:
      simple:
        acknowledge-mode: none # 关闭ack

Modify the method in the SpringRabbitListener class of the consumer service to simulate a message processing exception:

@RabbitListener(queues = "simple.queue")
public void listenSimpleQueue(String msg) {
    log.info("消费者接收到simple.queue的消息：【{}】", msg);
    // 模拟异常
    System.out.println(1 / 0);
    log.debug("消息处理完成！");
}

The test can find that when the message processing throws an exception, the message is still deleted by RabbitMQ.

4.2. Demo auto mode

Change the confirmation mechanism to auto again:

spring:
  rabbitmq:
    listener:
      simple:
        acknowledge-mode: auto # 关闭ack

Break the point at the abnormal position and send the message again. When the program is stuck at the break point, you can find that the message status is unack (undetermined status):

After an exception is thrown, because Spring will automatically return nack, the message returns to the Ready state and is not deleted by RabbitMQ:

5. Consumption failure retry mechanism

When the consumer has an exception, the message will continue to requeue (re-enter the queue) to the queue, and then resend to the consumer, and then the exception again, requeue again, infinite loop, causing the message processing of mq to soar, bringing unnecessary pressure:

5.1, local retry

We can use Spring's retry mechanism to use local retry when an exception occurs in the consumer, instead of unlimited requeue to the mq queue.

Modify the application.yml file of the consumer service and add the content:

spring:
  rabbitmq:
    listener:
      simple:
        retry:
          enabled: true # 开启消费者失败重试
          initial-interval: 1000 # 初识的失败等待时长为1秒
          multiplier: 1 # 失败的等待时长倍数，下次等待时长 = multiplier * last-interval
          max-attempts: 3 # 最大重试次数
          stateless: true # true无状态；false有状态。如果业务中包含事务，这里改为false

Restart the consumer service and repeat the previous test. It can be found:

After retrying 3 times, SpringAMQP will throw an exception AmqpRejectAndDontRequeueException, indicating that the local retry triggers
Check the RabbitMQ console and find that the message has been deleted, indicating that SpringAMQP returned ack at the end, and mq deleted the message

in conclusion:

When local retry is enabled, if an exception is thrown during message processing, it will not be requeed to the queue, but will be retried locally by the consumer
After the maximum number of retries is reached, Spring will return ack and the message will be discarded

5.2. Failure Strategy

In the previous test, after reaching the maximum number of retries, the message will be discarded, which is determined by the internal mechanism of Spring.

After the retry mode is turned on, the number of retries is exhausted. If the message still fails, the MessageRecovery interface is required to handle it. It contains three different implementations:

RejectAndDontRequeueRecoverer: After the retries are exhausted, directly reject and discard the message. This is the default
ImmediateRequeueMessageRecoverer: After retries are exhausted, nack is returned, and the message is re-queued
RepublishMessageRecoverer: After the retries are exhausted, deliver the failure message to the specified exchange

A more elegant solution is RepublishMessageRecoverer. After a failure, the message will be delivered to a designated queue dedicated to storing exception messages, and subsequent processing will be done manually.

1) Define the switch and queue for processing failed messages in the consumer service

@Bean
public DirectExchange errorMessageExchange(){
    return new DirectExchange("error.direct");
}
@Bean
public Queue errorQueue(){
    return new Queue("error.queue", true);
}
@Bean
public Binding errorBinding(Queue errorQueue, DirectExchange errorMessageExchange){
    return BindingBuilder.bind(errorQueue).to(errorMessageExchange).with("error");
}

2) Define a RepublishMessageRecoverer, associate queues and switches

@Bean
public MessageRecoverer republishMessageRecoverer(RabbitTemplate rabbitTemplate){
    return new RepublishMessageRecoverer(rabbitTemplate, "error.direct", "error");
}

Full code:

@Configuration
public class ErrorMessageConfig {
    @Bean
    public DirectExchange errorMessageExchange(){
        return new DirectExchange("error.direct");
    }
    @Bean
    public Queue errorQueue(){
        return new Queue("error.queue", true);
    }
    @Bean
    public Binding errorBinding(Queue errorQueue, DirectExchange errorMessageExchange){
        return BindingBuilder.bind(errorQueue).to(errorMessageExchange).with("error");
    }

    @Bean
    public MessageRecoverer republishMessageRecoverer(RabbitTemplate rabbitTemplate){
        return new RepublishMessageRecoverer(rabbitTemplate, "error.direct", "error");
    }
}

6. Summary

How to ensure the reliability of RabbitMQ messages?

Enable the producer confirmation mechanism to ensure that the producer's message can reach the queue
Enable the persistence function to ensure that the message will not be lost in the queue before it is consumed
Turn on the consumer confirmation mechanism as auto, and the spring will complete the ack after confirming that the message is processed successfully
Enable the consumer failure retry mechanism and set MessageRecoverer. After multiple retries fail, the message will be delivered to the abnormal switch and handed over to manual processing.

RabbitMQ --- message reliability