RabbitMQ: How to ensure 100% of messages are consumed

We know that messages go through three steps from the producer to the consumer:

• The producer sends a message to RabbitMQ
• RabbitMQ sends messages to consumers
• The consumer consumes this message

Each of these three steps may lead to message loss. Message loss is not terrible, but we don't know if it is lost, so there must be some measures to ensure the reliability of the system. The reliability here is not necessarily 100% not lost. Disk damage, fire in the computer room, etc. can lead to data loss. Of course, this is a very small probability of occurrence. It can be reliable if 99.99999% of messages are not lost.

Let's analyze the problems and solutions in detail.

Reliable delivery on the production side

Reliable delivery on the production side, that is, the production side must ensure that messages are correctly delivered to RabbitMQ. There are many reasons for the loss of messages delivered by the production side. For example, a network failure occurs during the network transmission of the message, and the message is lost; or when the message is delivered to RabbitMQ, RabbitMQ hangs, and the message may also be lost, and we do not know it happened. what. In response to the above situation, RabbitMQ itself provides some mechanisms.

Transaction Messaging

There are three methods related to the transaction mechanism in RabbitMQ: txSelect() , txCommit() , txRollback() , txSelect is used to set the current channel to transaction mode, txCommit is used to commit the transaction, and txRollback is used to roll back the transaction . After opening the transaction, we can publish the message to the broker proxy server. If the txCommit submission is successful, the message must have reached the broker. If the broker crashes abnormally or throws an exception for other reasons before txCommit is executed, then we can Catch the exception and roll back the transaction through txRollback .

try {
    channel.txSelect()
    channel.basicPublish(EXCHANGE_NAME, ROUTING_KEY, MessageProperties.PERSISTENT_TEXT_PLAIN,  msg.toByteArray(StandardCharsets.UTF_8))
    channel.txCommit()
}catch (e: Exception){
    channel.txRollback()
    // 进行消息重发
    channel.basicPublish(EXCHANGE_NAME, ROUTING_KEY, MessageProperties.PERSISTENT_TEXT_PLAIN,  msg.toByteArray(StandardCharsets.UTF_8))
}
复制代码

Transactions can indeed solve the problem of message confirmation between the producer and the broker. Only when the message is successfully accepted by the broker can the transaction commit succeed. Otherwise, we can re-send the message while catching the exception and performing the transaction rollback operation.

Transactional messaging is generally not used because it can seriously degrade performance. So is there a better way to ensure that the producer knows that the message has been sent correctly, without basically causing performance loss? From the perspective of the AMQP protocol, there is no better way, but RabbitMQ provides a better solution, which is to set the channel channel to confirm mode.

confirm message confirmation mechanism

什么是confirm消息确认机制？顾名思义，就是生产端投递的消息一旦投递到RabbitMQ后，RabbitMQ就会发送一个确认消息给生产端，让生产端知道我已经收到消息了，否则这条消息就可能已经丢失了，需要生产端重新发送消息了。

通过下面这句代码来开启确认模式：

channel.confirmSelect() // 开启生产者确认模式
复制代码

然后异步监听确认和未确认的消息：

channel.addConfirmListener(object: ConfirmListener{
    override fun handleAck(deliveryTag: Long, multiple: Boolean) {
        // 已收到消息，做一些处理
    }

    override fun handleNack(deliveryTag: Long, multiple: Boolean) {
        // RabbitMQ因为自身内部错误导致消息丢失，就会发送一条nack消息
        // 做一些其他处理，比如消息重发等
    }
})
复制代码

这样就可以让生产端感知到消息是否投递到RabbitMQ中了，当然这样还不够，稍后我会说一下极端情况。

消息持久化

那消息持久化呢？我们知道，RabbitMQ 收到消息后将这个消息暂时存在了内存中，那这就会有个问题，如果RabbitMQ 挂了，那重启后数据就丢失了，所以相关的数据应该持久化到硬盘中，这样就算 RabbitMQ 重启后也可以到硬盘中取数据恢复。那如何持久化呢？

message 消息到达 RabbitMQ 后先是到 exchange 交换机中，然后路由给 queue 队列，最后发送给消费端。

所以需要给 exchange、queue 和 message 都进行持久化。

exchange持久化：

// 第三个参数 true 代表这个 exchange 持久化
channel.exchangeDeclare(EXCHANGE_NAME, EXCHANGE_TYPE, true)
复制代码

queue持久化：

// 第二个参数 true 表示这个 queue 持久化
channel.queueDeclare(QUEUE_NAME, true, false, false, null)
复制代码

message持久化：

// 第三个参数MessageProperties.PERSISTENT_TEXT_PLAIN表示这条消息持久化
channel.basicPublish(EXCHANGE_NAME, ROUTING_KEY, MessageProperties.PERSISTENT_TEXT_PLAIN,  msg.toByteArray(StandardCharsets.UTF_8))
复制代码

这样，如果RabbitMQ收到消息后挂了，重启后会自行恢复消息。

到此，RabbitMQ提供的几种机制都介绍完了，但这样还不足以保证消息可靠性投递RabbitMQ中，上面我也提到了会有极端情况，比如RabbitMQ收到消息还没来得及将消息持久化到硬盘时，RabbitMQ挂了，这样消息还是丢失了。

所以除了RabbitMQ提供的一些机制外，我们自己也要做一些消息补偿机制，以应对一些极端情况。接下来我就介绍其中的一种解决方案——消息入库。

消息入库

消息入库，顾名思义就是将要发送的消息保存到数据库中。

首先发送消息前先将消息保存到数据库中，有一个状态字段 status=0，表示生产端将消息发送给了 RabbitMQ 但还没收到确认；在生产端收到确认后将 status 设为 1，表示 RabbitMQ 已收到消息。这里有可能会出现上面说的两种情况，所以生产端这边开一个定时器，定时检索消息表，将 status=0 并且超过固定时间后（可能消息刚发出去还没来得及确认这边定时器刚好检索到这条 status=0 的消息，所以给个时间）还没收到确认的消息取出重发（第二种情况下这里会造成消息重复，消费者端要做幂等性），可能重发还会失败，所以可以做一个最大重发次数，超过就做另外的处理。

这样消息就可以可靠性投递到RabbitMQ中了，而生产端也可以感知到了。

消费端不丢失

既然已经可以让生产端100%可靠性投递到RabbitMQ了，那接下来就改看看消费端的了，如何让消费端不丢失消息。

默认情况下，以下3种情况会导致消息丢失：

• 在RabbitMQ将消息发出后，消费端还没接收到消息之前，发生网络故障，消费端与RabbitMQ断开连接，此时消息会丢失；
• 在RabbitMQ将消息发出后，消费端还没接收到消息之前，消费端挂了，此时消息会丢失；
• 消费端正确接收到消息，但在处理消息的过程中发生异常或宕机了，消息也会丢失。

其实，上述3中情况导致消息丢失归根结底是因为RabbitMQ的自动ack机制，即默认RabbitMQ在消息发出后就立即将这条消息删除，而不管消费端是否接收到，是否处理完，导致消费端消息丢失时RabbitMQ自己又没有这条消息了。

所以就需要将自动ack机制改为手动ack机制。

消费端手动确认消息：

channel.basicConsume(
    QUEUE_NAME,
    false,
    DeliverCallback { consumerTag, delivery ->
        try {
            //接收到消息，做处理
            //手动确认
            channel.basicAck(delivery.envelope.deliveryTag, false)
        } catch (e: java.lang.Exception) {
            //出错处理，这里可以让消息重回队列重新发送或直接丢弃消息
        }
    },
    CancelCallback {}
)
复制代码

In this way, when the autoAck parameter is set to false, for the RabbitMQ server, the messages in the queue are divided into two parts: one is the message waiting to be delivered to the consumer; the other is the message that has been delivered to the consumer, but has not been received by the consumer. A message to confirm the signal. If RabbitMQ has not received the confirmation signal from the consumer, and the consumer that consumes the message has been disconnected or down (RabbitMQ will perceive it by itself), RabbitMQ will arrange for the message to re-enter the queue (put it at the head of the queue), Waiting for delivery to the next consumer, of course, it may still be the original consumer. Of course, the consumer also needs to ensure idempotency.

Well, at this point, the whole link from the production end to RabbitMQ to the consumer end can ensure that the data is not lost.