RabbitMQ Learning (6): Release Confirmation

1. The principle of release confirmation

The producer sets the channel to confirm mode. Once the channel enters the confirm mode, all messages published on the channel will be assigned a unique ID (starting from 1). Once the message is delivered to all matching queues, the broker A confirmation will be sent to the producer (including the unique ID of the message), which makes the producer know that the message has arrived at the destination queue correctly. If the message and the queue are durable, the confirmation message will be written to the disk After sending it out, the delivery-tag field in the confirmation message returned by the broker to the producer contains the serial number of the confirmation message. In addition, the broker can also set the multiple field of basic.ack, indicating that all messages up to this serial number have been received. deal with.

The biggest advantage of the confirm mode is that it is asynchronous. Once a message is published, the producer application can continue to send the next message while waiting for the channel to return confirmation. When the message is finally confirmed, the producer application can pass the callback method To process the confirmation message, if RabbitMQ loses the message due to its own internal error, it will send a nack message, and the producer application can also process the nack message in the callback method.

2. Publish the confirmed strategy

2.1 How to enable release confirmation

Release confirmation is not enabled by default. If you want to enable it, you need to call the method confirmSelect. Whenever you want to use release confirmation, you need to call this method on the channel.

Channel channel = connection.createChannel();
channel.confirmSelect();

2.2 Single confirmation release

This is a simple confirmation method. It is a synchronous confirmation method, that is, after publishing a message, only after it is confirmed and published, subsequent messages can continue to be published. waitForConfirmsOrDie(long) is only available when the message is confirmed. Time to return, if the message is not acknowledged within the specified time frame then it will throw an exception.

One of the biggest disadvantages of this confirmation method is that the release speed is extremely slow. Because the publication of all subsequent messages will be blocked if the published message is not confirmed, this method provides a maximum throughput of no more than hundreds of published messages per second. Of course for some applications this may be sufficient.

Core code:

for (int i = 0; i < MESSAGE_COUNT; i++) {
    String message = i + "";
    channel.basicPublish("", queueName, null, message.getBytes());
    //服务端返回 false 或超时时间内未返回，生产者可以消息重发
    boolean flag = channel.waitForConfirms();
    if(flag){
        System.out.println("消息发送成功");
    }
}

2.3 Batch confirmation release

The above method is very slow. Compared with waiting for a single confirmation message, publishing a batch of messages first and then confirming them together can greatly improve throughput. Of course, the disadvantage of this method is: when a failure causes a problem in the release, it is unknown Which message has a problem, we must keep the entire batch in memory to record important information and then republish the message. Of course, this solution is still synchronous, and it also blocks the release of messages.

Core code:

for (int i = 0; i < MESSAGE_COUNT; i++) {
    String message = i + "";
    channel.basicPublish("", queueName, null, message.getBytes());
    outstandingMessageCount++;
    //每发batchSize条消息，就确认一次
    if (outstandingMessageCount == batchSize) {
        channel.waitForConfirms();
        outstandingMessageCount = 0;
    }
}
//为了确保还有剩余没有确认消息 再次确认
if (outstandingMessageCount > 0) {
    channel.waitForConfirms();
}

2.4 Asynchronous confirmation release

Although the programming logic of asynchronous confirmation is more complicated than the previous two, it is the most cost-effective, regardless of reliability or efficiency. It uses the callback function to achieve reliable delivery of messages. This middleware also uses function callbacks to ensure whether The delivery is successful, let us explain in detail how the asynchronous confirmation is realized.

We no longer confirm immediately after receiving the message, but hand over the task of confirmation to the broker. Regardless of whether the message is consumed normally, the message queue will receive the message.

Core code:

/**
 * 确认收到消息的一个回调
 * 1.消息序列号
 * 2.true 可以确认小于等于当前序列号的消息
 * false 确认当前序列号消息
*/
ConfirmCallback ackCallback = (sequenceNumber, multiple) -> {
    if (multiple) {
        //返回的是小于等于当前序列号的未确认消息 是一个 map
        ConcurrentNavigableMap<Long, String> confirmed = outstandingConfirms.headMap(sequenceNumber, true);
        //清除该部分未确认消息
        confirmed.clear();
    }else{
        //只清除当前序列号的消息
        outstandingConfirms.remove(sequenceNumber);
    }
};
ConfirmCallback nackCallback = (sequenceNumber, multiple) -> {
    String message = outstandingConfirms.get(sequenceNumber);
    System.out.println("发布的消息"+message+"未被确认，序列号"+sequenceNumber);
};

/**
 * 添加一个异步确认的监听器
 * 1.确认收到消息的回调
 * 2.未收到消息的回调
*/
channel.addConfirmListener(ackCallback, null);
long begin = System.currentTimeMillis();
for (int i = 0; i < MESSAGE_COUNT; i++) {
    String message = "消息" + i;
    /**
     * channel.getNextPublishSeqNo()获取下一个消息的序列号
     * 通过序列号与消息体进行一个关联
     * 全部都是未确认的消息体
    */
    outstandingConfirms.put(channel.getNextPublishSeqNo(), message);
    channel.basicPublish("", queueName, null, message.getBytes());
}

2.5 How to handle asynchronous unacknowledged messages

The best solution is to put unacknowledged messages on a memory-based queue that can be accessed by the publishing thread, such as ConcurrentLinkedQueue. This queue passes messages between confirm callbacks and the publishing thread.

2.6 Comparison of the above three release confirmation speeds

• published separately

Message synchronization waits for confirmation, simple, but very limited throughput

• Batch release

Messages are synchronized in batches and wait for confirmation, simple and reasonable throughput, once a problem occurs, it is difficult to deduce which message has a problem

• asynchronous processing

Optimal performance and resource usage, with good control in case of errors, but slightly harder to implement