RabbitMQ: a magic wand for efficient message delivery, an article to help you build a reliable distributed system (Part 1)

1. What is MQ

1.1 The concept of MQ

MQ is an acronym for Message Queue ( Message Queue), which is a technology for passing messages between applications. Typically used in 分布式系统or 异步通信where the sender puts a message on the queue and the receiver gets the message from the queue.

This asynchronous communication pattern allows senders and receivers to communicate without requiring a real-time connection, improving application performance and scalability.

Message queues can be used in many different application scenarios, such as handling a large number of concurrent requests, implementing asynchronous processing of tasks, decoupling and building loosely coupled systems, implementing logging and monitoring, and more.

Let's use a scenario to better understand MQ:

Order Processing System:

An e-commerce website that has an order processing system. When a user submits an order, the system needs to save the order information into the database, and send emails and SMS notifications to the customer and warehouse administrator.

Without the use of message queues, the order processing system would need to wait for the database operation to complete before sending notifications. When the system lags or responds slowly under high load conditions .

Now, suppose a message queue is used to handle this scenario. When a user submits an order, the system puts the order information into the message queue.

Then, another system (or process) gets the order information from the message queue and saves it to the database.

Once the order information is saved in the database, another system (or process) will get the order information from the message queue and send notification emails and SMS.

In this way, the order information can be quickly put into the message queue, and then the response can be returned to the user immediately, and more processors can be easily added to process the order information in the message queue.

The following is an example to illustrate the three major characteristics of MQ: traffic peak reduction, application decoupling, and asynchronous processing .

1.2 MQ flow peak elimination

When the system is facing a peak period, it can put the request into the message queue, and then gradually consume the requests in the message queue according to the system's processing capacity and resource conditions, which is MQ traffic peak reduction.

In the order system, if the client requests the server 10,000 orders per second, and the server can only carry 5,000 orders per second, the database may be overloaded and the server may go down.

These order requests can be put into the message queue, 5000 times per second, and sequentially queued to access the database, or multiple message queues can be set up to process different types of order requests to further disperse the request pressure.

1.3 Application decoupling of MQ

The application decoupling of MQ refers to reducing the degree of coupling between applications. Different applications are no longer directly dependent, but decoupled through message queues.

When an application generates a message, it only needs to send the message to the message queue, and other applications can obtain the message by subscribing to the message queue and process it accordingly, thus achieving loose coupling between applications.

Suppose there is an order system of an e-commerce website. In the traditional application architecture, there is usually a tight coupling relationship between the order system, inventory management system, and logistics management system. A failure of a subsystem may lead to abnormal order results.

Through use MQ, the order system puts the order information into the message queue, and then the inventory management system, payment system, and logistics management system respectively subscribe to the message queue and perform corresponding operations.

In this way, the systems are no longer directly dependent, but are decoupled through message queues.

1.4 Asynchronous processing of MQ

In the traditional synchronous processing mode, when an application sends a task, it must wait for the task to complete and obtain the result, which will cause the application to respond slowly and cannot handle high concurrent requests.

By using the message queuing technology, the execution of the task and the return of the result are decoupled and unbound, so that the application that sends the task can return immediately without waiting for the completion of the task and the return of the result.

When A calls B's service, A does not need to wait for B to return the result before doing its own thing, just join the message queue, and after B completes the task, send a message to A through MQ.

1.5 Classification of MQ and how to choose

Common message queuing software includes, Apache Kafka、RabbitMQ、ActiveMQ，RocketMQetc., and these four are briefly introduced below.

1.5.1 ActiveMQ

ActiveMQis a Javamessaging middleware based on , which implements Java Message Service (JMS)the specification for passing messages in distributed applications.

ActiveMQSupports a variety of messaging protocols, including AMQP、MQTT、OpenWire和STOMPetc., with features such as persistence, distributed clustering, message selection, and security.

ActiveMQAdvanced features such as message transformation, message storage, transactional sessions, message monitoring and debugging are also provided for Springintegration with frameworks and other Java applications.

1.5.2 Apache Kafka

Apache KafkaIt is a distributed publish-subscribe messaging system for processing large-scale data streams and real-time data processing.

KafkaProcessing data through distributed storage, distributed processing, and distributed collaboration can support data processing at a speed of millions per second, and is mainly used in big data scenarios.

It can be integrated with Hadoop、Storm、Sparkother big data processing platforms.

1.5.3 RabbitMQ

RabbitMQis an open source messaging middleware that implements the Advanced Message Queuing Protocol (AMQP) for messaging in distributed applications.

RabbitMQ is a reliable, scalable, cross-platform solution that can link different applications, services or systems over the network and provide an asynchronous, reliable, and transactional messaging mechanism.

The architecture of RabbitMQ is based on the Erlang language, which has concurrency and scalability, and can distribute message brokers on multiple servers, thereby improving the scalability and availability of the system.

1.5.4 RocketMQ

RocketMQIt is Alibaba's open source distributed message middleware, which has the characteristics of high availability, high reliability, and high throughput. It supports publish/subscribe mode and point-to-point mode, and also provides a variety of messaging modes, such as synchronous sending, asynchronous sending and sequential sending.

RocketMQIt also provides advanced functions such as message transaction mechanism and message trace, as well as integration with big data frameworks such as Apache Storm and Apache Flink. RocketMQ is a mature, reliable, and enterprise-level messaging middleware that has been widely used in e-commerce, finance, games, and other fields.

1.5.5 Differences between the four MQs

The main differences have been Kafka、RabbitMQ、ActiveMQ、RocketMQcompared, you can refer to the table below:

features	Kafka	RabbitMQ	ActiveMQ	RocketMQ
delivery mode	publish-subscribe, peer-to-peer	publish-subscribe, peer-to-peer	publish-subscribe, peer-to-peer	publish-subscribe, peer-to-peer
protocol support	Own protocol Kafka Protocol, HTTP	AMQP、STOMP、MQTT、HTTP、WebSockets	JMS、AMQP、STOMP、MQTT、OpenWire	Own protocol, MQTT, HTTP
scalability	high	medium	high	high
reliability	high	medium	high	high
throughput	high	medium	medium	high
real-time	high	Low	Low	high
storage mechanism	disk storage	memory or disk storage	memory or disk storage	disk storage
message assurance	at least once, at most once, exactly once	at least once, at most once, exactly once, not guaranteed	at least once, at most once, exactly once	At-least-once, at-most-once, ordered messages
Partition support	support	not support	not support	support
performance comparison	high	Low	Low	high
Application Scenario	Stream processing, log aggregation, messaging	Data processing, task distribution, message passing	Data processing, messaging, enterprise integration	Streaming data processing, messaging, online transactions
advantage	High throughput, high reliability, high scalability	High message reliability and support for multiple messaging modes	Support multiple protocols, high reliability, stable message delivery	High throughput, high reliability, high scalability
shortcoming	Low real-time performance and complex ecosystem	Low real-time performance and general scalability	Less real-time, lower throughput	The ecosystem is relatively small and some functions are weak

1.6 The core concept of MQ

1.6.1 Four core concepts of MQ

The four core concepts of MQ ( Message Queue, message queue) are:

1. Exchange : Mainly used to route messages to specified queues or topics.

2. Queue : A buffer for storing messages. Message publishers send messages to the queue, and message consumers take messages from the queue for processing.

3. Producer : The application in MQ that sends messages to the queue, that is, the producer of the message.

4. Consumer : The application in MQ that takes messages out of the queue for processing, that is, the consumer of the messages.

The producer sends the message to the queue, which is distributed through the switch in the middle, and the consumer takes the message from the queue for processing, which realizes the decoupling between applications and improves the reliability, scalability and maintainability of the system.

1.6.2 Six core parts of MQ

The following are the six core parts of RabbitMQ:

1. Hello World: The simplest getting started example for RabbitMQ, demonstrating how to send and receive simple "Hello World" messages in RabbitMQ.

2. Work Queues: Also known as the task queue pattern, it is used to distribute tasks among multiple consumers and balance the load. Multiple consumers subscribe to the same queue at the same time, and one message will only be received and processed by one consumer.

3. Publish/Subscribe: Also known as the publish/subscribe pattern, it is used to broadcast messages to multiple consumers. The publisher sends the message to an exchange, and the exchange routes the message to all queues bound to it, thereby realizing the broadcast of the message.

4. Routing: Also known as routing mode, it is used to route messages to specified queues. The sender sends a message with a routing key (Routing Key) to the switch, and the switch will route the message to the specified queue according to the routing key.

5. Topics: Also known as topic mode, similar to Routing mode, but routing keys can use wildcards. The sender sends a message with a topic (Topic) to the exchange, and the exchange will route the message to the corresponding queue according to the topic.

6. RPC: Also known as the Remote Procedure Call pattern, it is used to implement communication between distributed applications. The client sends a request message with a callback queue and a unique identifier to RabbitMQ, the server receives and processes the request, and sends the response back to the callback queue specified by the client.

1.7 Install RabbitMQ

It is relatively simple Linux系统to use it dockerto install it RabbitMQ. Of course, the premise of your virtual machine is to install it docker. For installation, dockerplease refer to: [ docker installation ] .

To use Docker installation in Linux system RabbitMQ, you can follow the steps below:

1. Search for RabbitMQmirrors: Execute the following command in the terminal to search for available RabbitMQmirrors:

   docker search rabbitmq
   

2. Download the RabbitMQ image: Select a suitable RabbitMQ image and download it from Docker Hub using the following command:

   docker pull rabbitmq:3-management
   

   Among them, 3-managementis the image of RabbitMQ 3.x version with management plug-in.

3. Running a RabbitMQ container: Start a RabbitMQ container with the following command:

   docker run -d --name my-rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3-management
   

   Among them, --namespecify the container name, and -pspecify the port mapped between the container and the host. In this example, both the default port 5672 of RabbitMQ and the port 15672 of the web management interface are mapped to the corresponding ports of the host machine.

4. Access the web management interface: access in a browser http://localhost:15672, and use the default guest/guestaccount and password to log in to the RabbitMQ web management interface. If you visit in the local browser, localhostyou need to change the IP address of LInux here.

   After you are done using it, you can stop and delete the container with the following commands :

   # 停止容器
   docker stop my-rabbitmq
   # 删除容器
   docker rm my-rabbitmq
   

After successful access, the following page will appear:

2. Hello World

2.1 Import dependencies

Now use RabbitMQ's simplest entry example to demonstrate how to send and receive simple "Hello World" messages in RabbitMQ.

As shown in the figure above, Pit is the producer Cand the consumer, and the middle box is rabbitMQthe representative message buffer area.

Import dependencies in maven:

 <!-- rabbitmq依赖的客户端 -->
 <dependency>
     <groupId>com.rabbitmq</groupId>
     <artifactId>amqp-client</artifactId>
     <version>5.16.0</version>
 </dependency>

 <!-- 操作文件流的依赖 -->
 <dependency>
     <groupId>commons-io</groupId>
     <artifactId>commons-io</artifactId>
     <version>2.11.0</version>
 </dependency>

2.2 Create a producer

package com.javadouluo.abbitmq;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

/**
 * @author jektong
 * @date 2023年05月03日 17:05
 */
public class Producer {
    
    

    // 队列名称
    private static final  String QUEUE_NAME = "hello world";

    public static void main(String[] args) throws Exception{
    
    
        // 创建一个工厂
        ConnectionFactory connectionFactory = new ConnectionFactory();
        // 工厂IP 连接RabbitMQ队列
        connectionFactory.setHost("192.168.10.100");
        // 用户名
        connectionFactory.setUsername("guest");
        // 密码
        connectionFactory.setPassword("guest");
        // 创建连接
        Connection connection = connectionFactory.newConnection();
        // 获取信道
        Channel channel = connection.createChannel();
        /**
         * 生成一个队列
         * 1.队列名称
         * 2.队列中消息是否持久化，默认消息在内存中
         * 3.是否只给一个消费者消费,true消息可以共享，false消息不可共享
         * 4.是否自动删除，最后一个消费者断开连接之后是否自动删除该队列 true是自动删除
         * 5.其他参数
         */
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        // 发消息
        String message = "hello world";
        /**
         * 发送一个消费
         * 1.发送到哪个交换机
         * 2.路由的key值哪个，这次是队列名称
         * 3.其他参数信息
         * 4.发送消息的消息内容
         */
        channel.basicPublish("",QUEUE_NAME,null,message.getBytes());
        System.out.println("消息发送完毕！！！");
    }
}

basicPublish()The method is one of the methods provided by the AMQP protocol in RabbitMQ, which is used to publish the message to the specified exchange.

The parameters of this method are as follows:

• exchange: Indicates which exchange the message is sent to, it can be empty, indicating that the default exchange is used.
• routingKey: Represents the routing key, which is used to specify which queues to route the message to. If the default switch is used, then the routing key needs to be specified as the queue name.
• props: Indicates the attribute information of the message, which is generally empty, and the default attribute can be used.
• body: Indicates the content of the message to be sent, which needs to be converted into a byte array.

Use basicPublish()the method to send the message to the exchange, and then the exchange will route the message to the corresponding queue according to the routing key, waiting for the consumer to consume.

After the execution is complete, open the MQ management interface, and you will find the hello worldname of the queue named .

The main page will also display detailed information:

2.3 Create a consumer

package com.javadouluo.abbitmq;

import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * @author jektong
 * @date 2023年05月03日 19:26
 */
public class Consumer {
    
    

    public static final String QUEUE_NAME = "hello world";

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        // 创建一个工厂
        ConnectionFactory connectionFactory = new ConnectionFactory();
        // 工厂IP 连接RabbitMQ队列
        connectionFactory.setHost("192.168.10.100");
        // 用户名
        connectionFactory.setUsername("guest");
        // 密码
        connectionFactory.setPassword("guest");
        // 创建连接
        Connection connection = connectionFactory.newConnection();
        // 创建信道
        Channel channel = connection.createChannel();
        // 声明 接收消息
        DeliverCallback deliverCallback = (consumerTag,message)->{
    
    
            System.out.println(new String(message.getBody()));
        };
        // 取消消息时的回调
        CancelCallback cancelCallback = consumerTag->{
    
    
            System.out.println("消息消费被中断");
        };
        /**
         * 消费者消费消息
         * 1.消费哪个队列
         * 2.消费成功之后是否自动应答true自动应答，false代表手动应答
         * 3.消费者未成功消费的回调
         * 4.消费者取消消费的回调
         */
        channel.basicConsume(QUEUE_NAME,true,deliverCallback,cancelCallback);
    }
}

basicConsumemethod is used to start consuming messages from a queue. Specifically, it sends an instruction to the RabbitMQ server, telling it that we want to get messages from the specified queue and start consuming them.

basicConsumeThe common parameters of the method are as follows:

• queue: the name of the queue to be consumed;
• autoAck: If it is true, the consumer will automatically confirm it immediately after getting the message; if it is false, you need to call basicAckthe method to manually confirm the message;
• consumerTag: Consumer ID, used to identify the current consumer. If this parameter is not set, RabbitMQ will generate a unique identifier for each consumer ;
• deliverCallback: Message processing callback function, used to process the messages obtained in the queue.
• cancelCallback: The callback function for canceling consumption, which is used to execute when the consumer is canceled.

When a consumer calls basicConsumethe method, the RabbitMQ server will immediately push the message in the queue to it, and confirm immediately after the message is sent, so that RabbitMQ knows that the message has been consumed.

If autoAckthe parameter is false, the consumer needs to call basicAckthe method to manually acknowledge the message.

If an exception occurs in the process of processing the message, the consumer can also call basicNackthe method to re-enqueue the message for re-consumption.

Run the code to see that the message has been consumed:

3. Work Queues (work queue mode)

3.1 Create a worker thread

In the work queue mode, the producer sends a message to the queue, and multiple consumers get the message from the queue and process it. Each message will only be processed by one consumer, ensuring that each message will only be processed once, and there is a competitive relationship between them .

This pattern is characterized by distributing time-consuming tasks among consumers, and once a message is received by a consumer, it is removed from the queue.

RabbitMQMessages are sent to each consumer in turn. When the consumer processing is slow or a consumer is down, RabbitMQthe message will be resent to other consumers for processing.

Multiple consumers, also known as multiple worker threads, use the following code to complete the work queue mode.

First of all, we first transform the above consumer, copy the code of the two worker threads (consumer), and add a line of comments for convenience, Work01.javathe code is as follows:

/**
 * 这是一个工作线程
 * @author jektong
 * @date 2023年05月08日 21:49
 */
public class Work01 {
    
    

    public static final String QUEUE_NAME = "hello world";

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
		// 中间省略，与上面的消费者代码一致
        // 接收消息
        /**
         * 消费者消费消息
         * 1.消费哪个队列
         * 2.消费成功之后是否自动应答true自动应答，false代表手动应答
         * 3.消费者未成功消费的回调
         * 4.消费者取消消费的回调
         */
        System.out.println("work01等待接收消息");
        channel.basicConsume(QUEUE_NAME,true,deliverCallback,cancelCallback);
    }
}

Work02.javacode show as below:

/**
 * 这是一个工作线程
 * @author jektong
 * @date 2023年05月08日 21:49
 */
public class Work01 {
    
    

    public static final String QUEUE_NAME = "hello world";

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
		// 中间省略，与上面的消费者代码一致
        // 接收消息
        /**
         * 消费者消费消息
         * 1.消费哪个队列
         * 2.消费成功之后是否自动应答true自动应答，false代表手动应答
         * 3.消费者未成功消费的回调
         * 4.消费者取消消费的回调
         */
        System.out.println("work02等待接收消息");
        channel.basicConsume(QUEUE_NAME,true,deliverCallback,cancelCallback);
    }
}

3.2 Create a producer

This part is still based on the above producer code, mainly simulating sending multiple messages:

package com.javadouluo.abbitmq.two;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

import java.util.Scanner;

/**
 * @author jektong
 * @date 2023年05月08日 22:10
 * 生产者发送消息
 */
public class Task01 {
    
    

    // 队列名称
    private static final  String QUEUE_NAME = "hello world";

    public static void main(String[] args) throws Exception{
    
    
         
		// 中间代码省略，与上面一致
		
        /**
         * 发送一个消费
         * 1.发送到哪个交换机
         * 2.路由的key值哪个，这次是队列名称
         * 3.其他参数信息
         * 4.发送消息的消息内容
         */
        // 发送5条消息
        for (int i = 0; i < 5; i++) {
    
    
            channel.basicPublish("",QUEUE_NAME,null,("消息编号" + i).getBytes());
        }
        System.out.println("消息发送完毕！！！");
    }
}

3.3 Analysis of results

Start two worker threads, work01waiting for messages:

work02Also waiting for news:

Ok, now we start the producer to send messages to the consumer, and then look at work01this worker thread:

Look at this worker thread again work02:

Obviously, when there are multiple messages, the worker thread consumes the messages by polling.

4. Message response mechanism

4.1 Concept of Message Response

The message acknowledgment mechanism ( Message Acknowledgment) is a mechanism to confirm whether a message has been successfully processed by the consumer. In the message queue, when a message is acquired and processed by the consumer, a confirmation message needs to be sent to the message queue to tell the message queue that the message has been successfully processed by the consumer.

In the message queue, if a message is not successfully processed by the consumer, it will remain in the message queue until it is processed correctly . Without a message acknowledgment mechanism, the message queue has no way of knowing which messages were successfully processed.

Normally, the message response mechanism is divided into two modes: automatic response mode and manual response mode.

auto answer mode

Automatic response means that when a consumer receives a message from the queue, it immediately deletes the message from the queue without waiting for the consumer to explicitly confirm to RabbitMQ whether the processing has been completed.

The advantage of automatic response is that consumers can quickly remove messages from the queue, which improves the message processing efficiency and throughput of consumers. Plus, it makes message processing easy, since the consumer doesn't need to deal with acknowledgment logic.

Autoresponders also have some disadvantages. If an exception occurs while the consumer is processing messages, those messages will be lost and cannot be re-delivered.

If the consumer takes a long time to process the message without a clear confirmation mechanism, the message queue cannot know whether the message has been processed, which will cause the message to be processed multiple times, and may even cause the message to be lost.

Manual Answer Mode

In an actual production environment, manual responses are generally used to ensure reliable processing of messages.

Manual response means that after the consumer finishes processing a message, it needs to RabbitMQ send an acknowledgment signal to the display.

This method needs to call channel.basicAck()a method to notify that the current message has been consumed and can be deleted from the queue.

If an exception occurs during message processing, channel.basicNack()methods can be called to reject the current message and put it back on the queue . In addition, there are channel.basicReject()methods to reject the message and discard it .

The above three methods need to be remembered in detail later.

The advantage of manual acknowledgment is that it can ensure reliable processing of messages and avoid the problem of message loss due to consumer processing failures.

At the same time, the manual response can control the processing method of the message according to the actual situation.

Another advantage of manual acknowledgment is that batch acknowledgment can be used. In batch acknowledgment, consumers can confirm the processing results of multiple messages at one time to improve the efficiency of message confirmation.

Consumers can use basicAckmethod multipleparameters to batch responses, for example:

channel.basicAck(deliveryTag, true)

It deliveryTagrepresents the unique identifier of the message, and the second parameter determines whether to confirm multiple messages in batches. trueRepresents batch processing of messages.

In this way, consumers can confirm the processing results of multiple messages at one time.

For the difference between the second parameter trueand false:

For example, when is called , all messages from 1 to 10 channel.basicAck(10, true)are acknowledged .Delivery Tag

And when is called , only the message with 10 channel.basicAck(10, false)will be acknowledged .Delivery Tag

The disadvantage of manual response is that it increases the complexity of the code and the difficulty of implementation, and requires developers to handle message confirmation and rejection operations by themselves.

Manual acknowledgments can also cause delays in message processing, as messages need to be acknowledged by consumers before they can be removed from the queue.

4.2 Message manual response

4.2.1 Message re-enqueue

If the message has not been processed correctly as mentioned above, the message needs to be put back into the message queue for other consumers to consume, so as to ensure the accuracy of the message, as shown in the following figure:

4.2.2 Message manual response code implementation

Now write the code, use one producer and two consumers to realize the manual reply of the message without loss, and then re-enqueue to be consumed.

producer code

/**
 * @author jektong
 * @date 2023年05月13日 20:15
 */
public class Task2 {
    
    

    // 队列名称
    private static final  String QUEUE_NAME = "ack_queue";

    public static void main(String[] args) throws Exception{
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 声明队列
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        // 发消息
        Scanner sc = new Scanner(System.in);
        while (sc.hasNext()){
    
    
            String msg = sc.next();
            channel.basicPublish("",QUEUE_NAME,null,msg.getBytes());
            System.out.println("生产者发送消息：" + msg);
        }
    }
}

Consumer Work03 code

/**
 * @author jektong
 * @date 2023年05月13日 20:23
 */
public class Work03 {
    
    

    public static final String QUEUE_NAME = "ack_queue";

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        // 接收消息
        Channel channel = RabbitMqUtils.getChannel();
        System.out.println("消费者1处理消息时间较短");
        // 接收消息后处理
        DeliverCallback deliverCallback = (consumerTag, message)->{
    
    
            try {
    
    
                // 等待1s处理消息
                TimeUnit.SECONDS.sleep(1);
            } catch (InterruptedException e) {
    
    
                throw new RuntimeException(e);
            }
            System.out.println(new String(message.getBody()));
            // 手动应答
            /**
             * arg1:表示消息标识
             * arg2:是否批量应答（之前详细说了此方法）
             */
            channel.basicAck(message.getEnvelope().getDeliveryTag(),false);
        };
        // 取消消息时的回调
        CancelCallback cancelCallback = consumerTag->{
    
    
            System.out.println("消息消费被中断");
        };
        // 手动应答为fasle
        boolean autoAck = false;
        channel.basicConsume(QUEUE_NAME,autoAck,deliverCallback,cancelCallback);
    }
}

For consumer Work04code, just modify the waiting time.

// 等待30s后处理消息
TimeUnit.SECONDS.sleep(30);

Let Work04it be disconnected within 30 seconds, so that the messages for it to consume will be re-enqueued for consumers Work03to consume (please test by yourself).

Five. RabbiMQ message persistence

5.1 Concept of message persistence

The above only deals with the situation that the message will not be lost, but if you want to ensure that RabbiMQthe message will not be lost after the service is stopped, because by default, RabbiMQthe queue and the message will be ignored.

If the message is marked as persistent, then when RabbitMQ is shut down or restarted, the message will still exist. The persistence flag of the message needs to set the flag of both the queue and the message .

5.2 How to persist

5.2.1 Queue Persistence

The producer created above is not persisted, it needs to be persisted, and it needs to be marked asdurable=true

// 声明队列
boolean durable = true
channel.queueDeclare(QUEUE_NAME,durable ,false,false,null);

It should be noted that if the previous queue has not been persisted, the previous queue needs to be deleted, otherwise an error will occur.

Open the message management interface to prove that the queue has been persisted:

5.2.2 Message Persistence

MessageProperties.PERSISTENT_TEXT_PLAINTo make the published message persistent, you need to set the property in the message property and modify the code of the above producer:

// 将消息保存到磁盘上
channel.basicPublish("",QUEUE_NAME,MessageProperties.PERSISTENT_TEXT_PLAIN,msg.getBytes());

Note that marking messages as persistent does not guarantee that they will be stored permanently, because RabbitMQ can still lose messages that were just about to be written to disk, but have not yet been completely written to disk. Therefore, to ensure that messages are not lost, a backup and replication strategy (described later) is also required .

5.2.3 Unfair distribution

In some cases, a consumer is slower than others, and unfair distribution is needed , sending messages to some consumers even if they are busy.

In unfair distribution, RabbitMQeach message is still delivered to all consumers, but the message is delivered to the first idle consumer.

Thus, consumers that process messages quickly will get more messages more quickly, while consumers that process slowly will gradually receive fewer messages.

The implementation method of unfair distribution is the same as that of fair distribution, just do not use basicQosthe method setting prefetchCount, Work03and Work04add the following code:

int prefetchCount = 1;
// 使用不公平分发
channel.basicQos(prefetchCount);

5.2.4 Prefetch value

When a consumer connects to the queue and starts receiving messages, RabbitMQ will determine the number of messages sent to the consumer at one time according to the prefetch value setting.

The setting of the prefetch value takes effect on the consumer side, not on the queue side. Each consumer can independently set its own prefetch value.

Therefore, different consumers can set appropriate prefetch values ​​according to their own processing capabilities and needs.

For example, there are 7 messages at the beginning, and 2 and 5 messages are sent to consumers 1 and 2 respectively by setting the prefetch value. Use by channel.basicQos(prefetchCount)setting the prefetch value.

The following code will be Work03modified:

// 设置预取值
int prefetchCount = 2;
channel.basicQos(prefetchCount);

The following code will be Work04modified:

// 设置预取值
int prefetchCount = 5;
channel.basicQos(prefetchCount);

channel.basicQos(prefetchCount)If the parameter value of this method is 0, it is round-robin distribution, 1 is unfair distribution, and other values ​​are set prefetch values.

6. Release Confirmation

6.1 Overview of Release Confirmation

The principle of release confirmation is based on the channel (Channel) level confirmation mechanism in the AMQP protocol.

When a producer sends a message to RabbitMQ, publish acknowledgment mode is enabled on the channel. Once the publish confirmation mode is enabled, each time a message is sent, the producer will assign a unique delivery tag (Delivery Tag) to the message.

After RabbitMQ receives the message, it will send an acknowledgment message (ACK) to the producer to notify the producer that the message has been successfully received. The delivery tag of the corresponding message is included in the acknowledgment message.

Producers can process release confirmation in three ways: single confirmation release, batch confirmation release and asynchronous confirmation release.

6.2 Single confirmation release

A simple confirmation mode, using synchronous confirmation release , the basic flow of a single message confirmation is as follows:

1. Producers send messages to RabbitMQ.
2. The acknowledgment message the producer is waiting for RabbitMQ.
3. If an acknowledgment message is received within the specified timeout, the message has been successfully received and the producer can proceed to send the next message.
4. If the confirmation message is not received within the timeout period, the producer can perform corresponding processing according to the demand, such as resending the message, recording logs, and executing compensation logic, etc.

The disadvantage is that the publishing speed is very slow. The code below implements this method and checks the time for sending messages in this way.

public static void publishMessageSingle() throws Exception {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 队列声明
        String queueName = UUID.randomUUID().toString();
        channel.queueDeclare(queueName,true,false,false,null);
        // 开启发布确认
        channel.confirmSelect();
        // 开始时间
        long startTime = System.currentTimeMillis();
        // 批量发送消息
        for (int i = 0; i < 1000; i++) {
    
    
            String msg = i + "";
            channel.basicPublish("",queueName,null,msg.getBytes());
            // 单个消息发布确认
            boolean flag = channel.waitForConfirms();
            if(flag){
    
    
                System.out.println("消息发送成功");
            }
        }
        // 结束时间
        long endTime = System.currentTimeMillis();
        System.out.println("发布"+1000+"个单独确认消息耗时"+(endTime-startTime)+"ms");
    }

Run the code and find that it takes time 410ms:

6.3 Batch confirmation release

In batch message acknowledgment mode, the producer can send multiple messages at one time and confirm after all messages are successfully received.

The producer sets an acknowledgment window ( Confirm Window), the size of which determines the number of messages that can be unacknowledged.

When all messages in the window have been acknowledged, the producer receives a batch acknowledgment message ( Batch Ack).

The basic process of batch message confirmation is as follows:

1. Producer sends multiple messages to RabbitMQ.
2. The producer sets an acknowledgment window size.
3. When the number of messages sent reaches the acknowledgment window size, the producer waits for RabbitMQ's batch acknowledgment messages.
4. If a batch confirmation message is received, it means that all messages in the window have been successfully received, and the producer can continue to send the next batch of messages.
5. If the batch confirmation message is not received within the timeout period, the producer can perform corresponding processing according to the demand, such as resending the message, recording logs, executing compensation logic, etc.

By confirming messages in batches, producers can ensure the integrity of a batch of messages, which is suitable for scenarios where the requirements for message integrity are not so strict.

But if there is a problem, it is not known which message has a problem.

The following is the implementation of batch release message confirmation:

/**
     * 批量消息确认
     * @throws Exception
     */
    public static void publishMessageBatch() throws Exception {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 队列声明
        String queueName = UUID.randomUUID().toString();
        channel.queueDeclare(queueName,true,false,false,null);
        // 开启发布确认
        channel.confirmSelect();
        // 开始时间
        long startTime = System.currentTimeMillis();
        // 批量确认消息的大小
        int batchSize = 100;
        // 批量发送消息
        for (int i = 0; i < 1000; i++) {
    
    
            String msg = i + "";
            channel.basicPublish("",queueName,null,msg.getBytes());

            if(i % batchSize == 0){
    
    
                // 消息发布确认
                channel.waitForConfirms();
                System.out.println("消息发送成功");
            }
        }
        // 结束时间
        long endTime = System.currentTimeMillis();
        System.out.println("发布"+1000+"个单独确认消息耗时"+(endTime-startTime)+"ms");
    }

Run the code and find that it takes time 34ms:

6.4 Asynchronous confirmation release

Asynchronous confirmation is more cost-effective than the above two methods, because the producer does not wait for the confirmation message immediately after sending the message, but continues to send the next message.

At the same time, the producer will use an asynchronous callback (Callback) function to process the callback operation of the confirmation message to confirm whether the message is sent successfully.

The basic process of asynchronous message confirmation is as follows:

1. Producers send messages to RabbitMQ.
2. The producer does not immediately wait for an acknowledgment message, but proceeds to send the next message.
3. The producer registers an asynchronous callback function for processing acknowledgment messages.
4. When RabbitMQ receives the message and completes the processing, it will asynchronously send a confirmation message to the producer.
5. Once the producer receives the confirmation message, it will trigger the callback function to execute the corresponding logic, such as logging, updating status, etc.

The following is the implementation of message asynchronous confirmation:

/**
     * 异步消息确认
     * @throws Exception
     */
    public static void publishMessageAsync() throws Exception {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 队列声明
        String queueName = UUID.randomUUID().toString();
        channel.queueDeclare(queueName,true,false,false,null);
        // 开启发布确认
        channel.confirmSelect();
        // 开始时间
        long startTime = System.currentTimeMillis();
        // 消息处理成功回调
        ConfirmCallback ackCallback = (var1,var2)->{
    
    
            System.out.println("未确认的消息" + var1);
        };
        // 消息未处理成功回调
        ConfirmCallback nackCallback = (var1,var2)->{
    
    
            System.out.println("消息发送成功了" + var1);
        };
        // 消息监听器
        channel.addConfirmListener(ackCallback,nackCallback);
        // 批量发送消息
        for (int i = 0; i < 1000; i++) {
    
    
            String msg = i + "";
            channel.basicPublish("",queueName,null,msg.getBytes());
        }
        // 结束时间
        long endTime = System.currentTimeMillis();
        System.out.println("发布"+1000+"个异步确认消息耗时"+(endTime-startTime)+"ms");
    }

Run the code and find that it takes time 18ms:

6.5 Asynchronous unacknowledged message processing

For the processing of unacknowledged messages in asynchronous acknowledgments, one solution is to put unacknowledged messages in a memory-based queue that can be accessed by the publishing thread.

For example, use ConcurrentLinkedQequemessage passing between multiple threads. Multiple threads can send and receive messages at the same time, realizing concurrent delivery of messages.

When sending a message, record the sent message, and delete the confirmed successful message in the callback function. The code is implemented as follows:

	/**
     * 异步消息确认
     * @throws Exception
     */
    public static void publishMessageAsync() throws Exception {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 队列声明
        String queueName = UUID.randomUUID().toString();
        channel.queueDeclare(queueName,true,false,false,null);
        // 开启发布确认
        channel.confirmSelect();
        /**
         * 安全的线程有序的哈希表，就是一个容器，适用于高并发
         * 1.将序号与消息关联
         * 2.轻松批量删除
         * 3.支持高并发
         */
        ConcurrentSkipListMap<Long,String> concurrentSkipListMap  = new ConcurrentSkipListMap<>();
        // 开始时间
        long startTime = System.currentTimeMillis();
        // 消息处理成功回调
        // var1: 消息序列号
        // var2: 是否批量
        ConfirmCallback ackCallback = (var1,var2)->{
    
    
            if(var2){
    
    
                // 删除已经确认的消息，剩下的就是未确认的消息
                ConcurrentNavigableMap<Long, String> confirmed =
                        concurrentSkipListMap.headMap(var1);
                confirmed.clear();
            }else{
    
    
                concurrentSkipListMap.remove(var1);
            }
            System.out.println("确认的消息" + var1);
        };
        // 消息未处理成功回调
        ConfirmCallback nackCallback = (var1,var2)->{
    
    
            String unConfirm = concurrentSkipListMap.get(var1);
            System.out.println("未确认的消息是："+unConfirm+"，消息发送失败了失败标记：" + var1);
        };
        // 消息监听器
        channel.addConfirmListener(ackCallback,nackCallback);
        // 批量发送消息
        for (int i = 0; i < 1000; i++) {
    
    
            String msg = i + "";
            channel.basicPublish("",queueName,null,msg.getBytes());
            concurrentSkipListMap.put(channel.getNextPublishSeqNo(),msg);
        }
        // 结束时间
        long endTime = System.currentTimeMillis();
        System.out.println("发布"+1000+"个异步确认消息耗时"+(endTime-startTime)+"ms");
    }

ConcurrentSkipListMapIt is a thread-safe ordered hash table, suitable for high-concurrency environments. It can associate the serial number of the message with the content of the message, and supports highly concurrent read and write operations.

Use it to implement the confirmation and unacknowledgment of the message through the callback function.

7. Switch

7.1 What is a switch

The switch ( Exchange) is the routing center of the message, responsible for receiving the message sent by the producer , and routing the message to one or more queues according to certain routing rules , and determining the path from the producer to the queue.

RabbitMQThere are several common routing rules in : Direct Mode, Topic Mode, Header Mode and Fanout Mode, etc., which will be detailed later. In RabbitMQthe provided management interface you can see:

The switch mainly has the following functions:

1. Receiving messages : The exchange receives messages from producers and is responsible for sending the messages to the appropriate queue.
2. Routing messages : The exchange routes messages to one or more queues according to predefined routing rules.
3. Distributing messages : If an exchange routes messages to multiple queues, the exchange will copy the messages to all queues that meet the routing rules to implement broadcast or multicast of messages.
4. Support different routing modes : The switch can decide how to route messages according to different routing modes, such as direct exchange, fan exchange, topic exchange, etc.

There are also some concepts related to the switch, such as binding ( bindings), which is well understood, that is, the switch and the queue can be bound through one RoutingKey, so that the desired message can be sent to the specified queue.

#pic_center

7.2 fanout switch

In fanoutmode, the exchange will broadcast the message to all queues it is bound to, regardless of the message's routing key.

fanoutThe characteristics of the mode are as follows:

1. Broadcast message : The Fanout switch will copy the message to all queues bound to it to realize the broadcast of the message. Every consumer receives the same copy of the message.
2. Ignore Routing Keys : Fanout exchanges ignore routing keys for messages, it only cares about the queues it is bound to.
3. Applicable to publish/subscribe mode : Fanout mode is often used in publish/subscribe mode , in which one producer sends a message, and multiple consumers receive and process the message.

Here is the code to test fanoutthe pattern:

Consumer code :

package com.javadouluo.abbitmq.five;

import com.javadouluo.abbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.CancelCallback;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * @author jektong
 * @date 2023年05月24日 22:23
 */
public class ReceiveLogs01 {
    
    

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 声明一个交换机logs,类型是fanout
        channel.exchangeDeclare("logs","fanout");
        // 声明一个临时队列，名称是随机的
        // 当消费者断开与队列的连接时，队列自动删除
        String queueName = channel.queueDeclare().getQueue();
        // 绑定交换机与对列
        channel.queueBind(queueName,"logs","");
        System.out.println("将消息打印到控制台上......");
        // 接收消息后处理
        DeliverCallback deliverCallback = (consumerTag, message)->{
    
    
            System.out.println("01接收的消息是："+ new String(message.getBody()));
        };
        // 取消消息时的回调
        CancelCallback cancelCallback = consumerTag->{
    
    
            System.out.println("消息消费被中断");
        };
        channel.basicConsume(queueName,true,deliverCallback,cancelCallback);
    }
}

Producer code :

package com.javadouluo.abbitmq.five;

import com.javadouluo.abbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;

import java.io.IOException;
import java.util.Scanner;
import java.util.concurrent.TimeoutException;

/**
 * @author jektong
 * @date 2023年05月27日 10:12
 */
public class EmitLog {
    
    

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 声明一个交换机logs,类型是fanout
        channel.exchangeDeclare("logs","fanout");
        Scanner scanner = new Scanner(System.in);
        while (scanner.hasNext()){
    
    
            String msg = scanner.next();
            // 发送消息
            channel.basicPublish("logs","",null,msg.getBytes());
            System.out.println("生产者发出消息：" + msg);
        }
    }
}

Copy the above consumer twice, then start the producer and the consumer, send a message through the producer, and find that both consumers have received the message. This is the fanoutcharacteristic of the broadcast message in the mode:

At the same time, you can also see the created switch on the management platform:

7.3 direct switch

The main feature of the direct switch (direct) is that the bound routing keys are different, and it also has a function to realize multiple bindings .

Multiple binding means that a direct switch can have multiple routing keys to bind a switch, as shown in the following figure:

The following code implements the above functions:

Consumer DirectReceiveLogs01 code :

package com.javadouluo.abbitmq.six;

import com.javadouluo.abbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.CancelCallback;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * @author jektong
 * @date 2023年06月28日 0:58
 */
public class DirectReceiveLogs01 {
    
    

    // 队列名称
    public static  final String EXCHANGE_NAME = "direct_logs";

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 声明一个交换机logs,类型是direct
        channel.exchangeDeclare(EXCHANGE_NAME,"direct");
        // 声明一个队列
        channel.queueDeclare("console",false,false,false,null);
        // 绑定交换机与对列
        channel.queueBind("console",EXCHANGE_NAME,"info");
        channel.queueBind("console",EXCHANGE_NAME,"warn");
        // 接收消息后处理
        DeliverCallback deliverCallback = (consumerTag, message)->{
    
    
            System.out.println("DirectReceiveLogs01接收的消息是："+ new String(message.getBody()));
        };
        // 取消消息时的回调
        CancelCallback cancelCallback = consumerTag->{
    
    
            System.out.println("消息消费被中断");
        };
        channel.basicConsume("console",true,deliverCallback,cancelCallback);
    }
}

Consumer DirectReceiveLogs02 code :

package com.javadouluo.abbitmq.six;

import com.javadouluo.abbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.CancelCallback;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * @author jektong
 * @date 2023年06月28日 0:58
 */
public class DirectReceiveLogs02 {
    
    

    // 队列名称
    public static  final String EXCHANGE_NAME = "direct_logs";

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 声明一个交换机logs,类型是direct
        channel.exchangeDeclare(EXCHANGE_NAME,"direct");
        // 声明一个队列
        channel.queueDeclare("disk",false,false,false,null);
        // 绑定交换机与对列
        channel.queueBind("disk",EXCHANGE_NAME,"error");
        // 接收消息后处理
        DeliverCallback deliverCallback = (consumerTag, message)->{
    
    
            System.out.println("DirectReceiveLogs02接收的消息是："+ new String(message.getBody()));
        };
        // 取消消息时的回调
        CancelCallback cancelCallback = consumerTag->{
    
    
            System.out.println("消息消费被中断");
        };
        channel.basicConsume("disk",true,deliverCallback,cancelCallback);
    }
}

Producer DirectLog code :

package com.javadouluo.abbitmq.six;

import com.javadouluo.abbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;

import java.io.IOException;
import java.util.Scanner;
import java.util.concurrent.TimeoutException;

/**
 * @author jektong
 * @date 2023年06月28日 1:15
 */
public class DirectLog {
    
    

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        Scanner scanner = new Scanner(System.in);
        while (scanner.hasNext()){
    
    
            String msg = scanner.next();
            // 发送消息
            channel.basicPublish("direct_logs","info",null,msg.getBytes());
            System.out.println("生产者发出消息：" + msg);
        }
    }
}

The test method is the same as that of the Fanout switch, please test it yourself. The final binding method is as follows:

7.4 topic switch

topicerrorA switch is also called a topic switch. Looking at the picture of the direct switch before, if you want to route to the queue through a certain rule at the same time through the routing key disk, and route warnto the queue through routing console, then you need to use topicthe switch to route the message.

The benefit of topic exchanges is that they make the way messages are routed to queues more flexible.

For topicthe routing key of the switch, the following rules need to be followed: the routing key (RoutingKey) must be a word and separated by a dot . For example aa.route.bb, it is also noted that an asterisk *represents a word, #representing one or two words.

So the three words in the above picture *.orange.*represent the middle word , the three words that represent the last word , and the multiple words that represent the first word .orange*.*.rabbitrabbitlazy.#lazy

Now, according to the queue binding relationship in the above figure, an example is given to illustrate their data reception:

routing key	illustrate
quick.orange.rabbit	Received by Q1,Q2
lazy.orange.elephant	Received by Q1,Q2
quick.orange.fox	Received by Q1
lazy.brown.fox	Received by Q2
lazy.pink.rabbit	Satisfies both bindings but will be received once by Q2
quick.brown.fox	Mismatches will not be accepted by any queue and will be discarded
quick.orange.male.rabbit	Mismatches will not be accepted by any queue and will be discarded
lazy.orange.male.rabbit	four words, but would match Q2

topicNext, implement the routing message mode of the switch through the code .

Consumer ReceiveLogsTopic01 code

package com.javadouluo.abbitmq.seven;

import com.javadouluo.abbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.CancelCallback;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

import java.io.IOException;
import java.sql.SQLOutput;
import java.util.concurrent.TimeoutException;

/**
 * 主题交换机
 * @author jektong
 * @date 2023年06月29日 0:35
 */
public class ReceiveLogsTopic01 {
    
    

    // 交换机名称
    public static  final String EXCHANGE_NAME = "topic_logs";

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 声明一个交换机logs,类型是topic
        channel.exchangeDeclare(EXCHANGE_NAME,"topic");
        // 声明一个队列
        channel.queueDeclare("Q1",false,false,false,null);
        // 路由键绑定队列
        channel.queueBind("Q1",EXCHANGE_NAME,"*.orange.*");
        System.out.println("等待接收消息");
        // 接收消息后处理
        DeliverCallback deliverCallback = (consumerTag, message)->{
    
    
            System.out.println(new String(message.getBody(),"UTF-8"));
            System.out.println("接收队列：" + "Q1"+" 绑定键：" + message.getEnvelope().getRoutingKey());
        };
        // 取消消息时的回调
        CancelCallback cancelCallback = consumerTag->{
    
    
            System.out.println("消息消费被中断");
        };
        channel.basicConsume("Q1",true,deliverCallback,cancelCallback);
    }
}

Consumer ReceiveLogsTopic02 code

package com.javadouluo.abbitmq.seven;

import com.javadouluo.abbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.CancelCallback;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 主题交换机
 * @author jektong
 * @date 2023年06月29日 0:35
 */
public class ReceiveLogsTopic02 {
    
    

    // 交换机名称
    public static  final String EXCHANGE_NAME = "topic_logs";

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 声明一个交换机logs,类型是topic
        channel.exchangeDeclare(EXCHANGE_NAME,"topic");
        // 声明一个队列
        channel.queueDeclare("Q2",false,false,false,null);
        // 路由键绑定队列
        channel.queueBind("Q2",EXCHANGE_NAME,"*.*.rabbit");
        channel.queueBind("Q2",EXCHANGE_NAME,"lazy.#");
        System.out.println("等待接收消息");
        // 接收消息后处理
        DeliverCallback deliverCallback = (consumerTag, message)->{
    
    
            System.out.println(new String(message.getBody(),"UTF-8"));
            System.out.println("接收队列：" + "Q2"+" 绑定键：" + message.getEnvelope().getRoutingKey());
        };
        // 取消消息时的回调
        CancelCallback cancelCallback = consumerTag->{
    
    
            System.out.println("消息消费被中断");
        };
        channel.basicConsume("Q2",true,deliverCallback,cancelCallback);
    }
}

Producer TopicLog code

package com.javadouluo.abbitmq.seven;

import com.javadouluo.abbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeoutException;

/**
 * @author jektong
 * @date 2023年06月29日 0:46
 */
public class TopicLog {
    
    

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        Channel channel = RabbitMqUtils.getChannel();

        Map<String,String> map = new HashMap<>();
        map.put("quick.orange.rabbit","被Q1,Q2接收");
        map.put("lazy.orange.elephant","被Q1,Q2接收");
        map.put("quick.orange.fox","被Q1接收");
        map.put("lazy.brown.fox","被Q2接收");
        map.put("lazy.pink.rabbit","满足两个绑定但是会被Q2接收一次");
        map.put("quick.brown.fox","不匹配不会被任何队列接收，会被丢弃");
        map.put("quick.orange.male.rabbi","不匹配不会被任何队列接收，会被丢弃");
        map.put("lazy.orange.male.rabbit","不匹配不会被任何队列接收，会被丢弃");

        for(Map.Entry<String,String> maps:map.entrySet()){
    
    
            // 路由匹配键
            String key = maps.getKey();
            // 路由匹配值
            String value = maps.getValue();
            channel.basicPublish("topic_logs",key,null,value.getBytes("UTF-8"));
            System.out.println("生产者发送消息：" + value);
        }
    }
}

Please test it yourself. Finally, you can find in the management platform that the binding method is as follows:

8. Dead letter queue

8.1 The concept and source of dead letters

In some special cases, some messages cannot be consumed normally, and there is no follow-up processing afterwards. The message queue usually provides a special queue, that is, the dead letter queue .

The advantage of using a dead letter queue is that it provides a fault tolerance mechanism to ensure that messages that cannot be consumed normally will not be lost. At the same time, it also provides opportunities for troubleshooting and problem handling.

Sources of dead letter messages may have some of the following reasons:

1. Queue reached maximum length : The queue is full and messages cannot be added.

2. Abnormal error in the consumer : An error may occur when the consumer processes the message, resulting in message processing failure.

3. Message TTL expiration : Some messages may have a processing time limit. If the consumer fails to process the message within the specified time, the message may be considered as unable to be consumed normally. We call this message TTL expiration.

8.2 Dead letter combat

Before writing code, look at a picture to understand how the dead letter queue interacts in RabbitMQ:

Briefly explain the above figure, the producer sends a message to the MQ through the direct exchange and binds the queue normal_exchangethrough the routing key . When the message in it cannot be consumed by the C1 consumer, this message is a dead letter message and will be passed through the direct exchange. The routing key binding is put into the queue and handed over to C2 for consumption.zhangsannormal_queuedead_exchangelisidead_queue

The most important thing in this process is how the dead letter message can interact normal_queuewith the direct switch dead_exchangeand then put it into the dead letter queue dead_queue. The code implementation in the above figure is implemented below.

Consumer Consumer01 code

package com.javadouluo.abbitmq.eight;

import com.javadouluo.abbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.BuiltinExchangeType;
import com.rabbitmq.client.CancelCallback;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeoutException;

/**
 * @author jektong
 * @date 2023年06月30日 0:02
 */
public class Consumer01 {
    
    

    // 直接交换机normal_exchange
    public static  final String NORMAL_EXCHANGE = "normal_exchange";
    // 直接交换机dead_exchange
    public static  final String DEAD_EXCHANGE = "dead_exchange";
    // 队列normal_queue
    public static  final String NORMAL_QUEUE = "normal_queue";
    // 队列dead_queue
    public static  final String DEAD_QUEUE = "dead_queue";

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 声明一个普通交换机
        channel.exchangeDeclare(NORMAL_EXCHANGE, BuiltinExchangeType.DIRECT);
        // 声明一个死信交换机
        channel.exchangeDeclare(DEAD_EXCHANGE, BuiltinExchangeType.DIRECT);
        // 声明普通与死信队列
        Map<String,Object> arguments = new HashMap<>();
        arguments.put("x-dead-letter-exchange",DEAD_EXCHANGE);
        arguments.put("x-dead-letter-routing-key","lisi");
        // arguments就是用来绑定死信交换机的
        channel.queueDeclare(NORMAL_QUEUE,false,false,false,arguments);
        channel.queueDeclare(DEAD_QUEUE,false,false,false,null);
        // 普通交换机与普通队列绑定
        channel.queueBind(NORMAL_QUEUE,NORMAL_EXCHANGE,"zhangsan");
        // 死信交换机与死信队列绑定
        channel.queueBind(DEAD_QUEUE,DEAD_EXCHANGE,"lisi");
        // 接收消息后处理
        DeliverCallback deliverCallback = (consumerTag, message)->{
    
    
            System.out.println("Consumer01接收的消息是: " + new String(message.getBody(),"UTF-8"));
        };
        // 取消消息时的回调
        CancelCallback cancelCallback = consumerTag->{
    
    
            System.out.println("消息消费被中断");
        };
        channel.basicConsume(NORMAL_QUEUE,true,deliverCallback,cancelCallback);
    }
}

Producer Product Code

package com.javadouluo.abbitmq.eight;

import com.javadouluo.abbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * @author jektong
 * @date 2023年06月30日 0:27
 */
public class Product {
    
    

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        // 死信消息设置过期时间10s
        AMQP.BasicProperties properties = new AMQP.BasicProperties()
                .builder().expiration("10000").build();
        // 向普通队列发送消息
        for (int i = 0; i < 11; i++) {
    
    
            String msg = "info" + i;
            channel.basicPublish("normal_exchange","zhangsan",properties,msg.getBytes());
        }
    }
}

Start the consumer first Consumer01and then stop it, start the producer Productto send messages, and the messages will be timed out and sent to the dead letter queue:

Consumer Consumer02 code

package com.javadouluo.abbitmq.eight;

import com.javadouluo.abbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.BuiltinExchangeType;
import com.rabbitmq.client.CancelCallback;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeoutException;

/**
 * @author jektong
 * @date 2023年06月30日 0:02
 */
public class Consumer02 {
    
    

    // 队列dead_queue
    public static  final String DEAD_QUEUE = "dead_queue";

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        Channel channel = RabbitMqUtils.getChannel();
        channel.queueDeclare(DEAD_QUEUE,false,false,false,null);
        // 接收消息后处理
        DeliverCallback deliverCallback = (consumerTag, message)->{
    
    
            System.out.println("Consumer02接收的消息是: " + new String(message.getBody(),"UTF-8"));
        };
        // 取消消息时的回调
        CancelCallback cancelCallback = consumerTag->{
    
    
            System.out.println("消息消费被中断");
        };
        channel.basicConsume(DEAD_QUEUE,true,deliverCallback,cancelCallback);
    }
}

Start the consumer first Consumer02, and the previous messages will be consumed by it: