RabbitMQ: concepts and installation, simple mode, working, release acknowledgments, switches, dead letter queues, delay queues, release acknowledgments advanced, other knowledge, clustering

1. Message queue

1.0 Course Introduction

1.1.Related concepts of MQ

1.1.1.What is MQ

MQ (message queue: message queue) , literally, it is a queue , FIFO first in, first out , but the content stored in the queue is message , it is also a cross-process communication mechanism , used to transmit messages upstream and downstream . In the Internet architecture, MQ is a very common upstream and downstream “逻辑解耦+物理解耦”message communication service. After using MQ, the upstream message sending only needs to rely on MQ and does not need to rely on other services.

• Passing messages upstream and downstream : For example, on QQ account, classmate A sends a message to classmate B, then classmate A is the upstream and classmate B is the downstream. This process of transmitting messages is to transmit messages upstream and downstream.

1.1.2. Why use MQ

1. Traffic peak reduction
For example, if the order system can handle up to 10,000 orders, this processing capacity is more than enough to handle orders during normal periods. During normal periods, we can return results in one second after placing orders. However, during the peak period, if there are 20,000 orders placed, the operating system cannot handle it. It can only limit the number of orders exceeding 10,000 and not allow users to place orders. Using the message queue as buffer, we can cancel this restriction and spread the orders placed within one second into a period of time for processing. At this time, some users may not receive the successful order operation more than ten seconds after placing the order, but it is better than not being able to place the order. The single experience is better.

Summary : Excess order data will be queued.

• Advantages: The system will not go down
• Disadvantages: Slower speed
  

2. Application decoupling.
Take e-commerce applications as an example. The applications include order systems, inventory systems, logistics systems, and payment systems. After the user creates an order, if the inventory system, logistics system, and payment system are coupled and called, if any subsystem fails, the order operation will be abnormal. When converted to a message queue-based approach, the problems of inter-system calls will be greatly reduced. For example, a logistics system will take a few minutes to repair due to a failure. During these few minutes, the memory to be processed by the logistics system is cached in the message queue, and the user's order operation can be completed normally. When the logistics system is restored, order information can continue to be processed. The mid-order user will not feel the failure of the logistics system, improving the availability of the system.

Summary :

• Before use: The order system directly calls the subsystem. Once the subsystem is abnormal, the entire order system will also fail.
• After use: The order system will send messages to the queue only after the execution is completed. Subsequent tasks will be distributed by the queue to the payment system, inventory system, and logistics system in turn, until the three major subsystems are completed. Once an exception occurs in any of the subsystems during the execution process, the queue will supervise it to continue until the delivery is completed.
  

3. Asynchronous processing
Some calls between services are asynchronous. For example, A calls B, and B takes a long time to execute, but A needs to know when B can be completed. In the past, there were generally two ways. A calls B after a while. Query API query. Or A provides a callback API, and after B completes the execution, it calls the API to notify A of the service. Both of these methods are not very elegant. Using the message bus can easily solve this problem. After A calls the B service, it only needs to monitor the message that B has completed. When B completes the processing, it will send a message to MQ, and MQ will Forward this message to service A. In this way, service A does not need to call B's query API in a loop, nor does it need to provide a callback API. Similarly, service B does not need to perform these operations. Service A can also get the message of successful asynchronous processing in a timely manner.

Scenario : A sends a message to B, and B takes a long time to execute, but A needs to know when B can complete the execution.

• Synchronous processing: A calls B's query api query after a period of time. Or A provides a callback API, and after B completes the execution, it calls the API to notify A of the service. This process A needs to wait forever and cannot do other things.
• Asynchronous processing: After A calls service B, it only needs to listen to the message that B has completed processing. When B completes processing, it will send a message to MQ, and MQ will forward the message to service A. A does not need to wait all the time in this process and can do other things.

1.1.3. Classification of MQ

1. ActiveMQ ( 最先出现的MQ，比较老)
Advantages : Single machine throughput of 10,000 levels, timeliness of ms level, high availability, high availability based on master-slave architecture, low probability of data loss with low message reliability

Disadvantages : The official community now maintains ActiveMQ 5.x less and less, and it is less used in high-throughput scenarios.

Shang Silicon Valley official website video: http://www.gulixueyuan.com/course/322

2. The trump card of Kafka
big data. When it comes to message transmission in the field of big data, Kafka cannot be avoided. This message middleware for big data has become famous for its million-level TPS throughput and is rapidly becoming popular. It has become the darling of the big data field and plays a decisive role in the process of data collection, transmission and storage. It has been adopted by major companies such as LinkedIn, Uber, Twitter, and Netflix.

Advantages : Excellent performance, single-machine write TPS is about one million entries/second, the biggest advantage is high throughput . Timeliness and MS-level availability are very high. Kafka is distributed. There are multiple copies of one data. If a few machines are down, there will be no loss of data or unavailability. Consumers use the Pull method to obtain messages, and the messages are in order. Through control It can ensure that all messages are consumed and only once; there is an excellent third-party Kafka Web management interface Kafka-Manager; it is relatively mature in the field of logs and is used by many companies and multiple open source projects; function support: The function is relatively simple, mainly Supports simple MQ functions, and is used on a large scale for real-time computing and log collection in the field of big data

Disadvantages : If Kafka has more than 64 queues/partitions on a single machine, the load will obviously skyrocket. The more queues, the higher the load, and the response time for sending messages becomes longer. If short polling is used, the real-time performance depends on the polling interval. Retry is not supported for consumption failures; message order is supported, but when an agent goes down, messages will be out of order and community updates will be slow ;

3. RocketMQ
RocketMQ is an open source product from Alibaba. It is implemented in Java language. It referred to Kafka during design and made some improvements of its own. It is widely used by Alibaba in order, transaction, recharge, stream computing, message push, log streaming processing, binglog distribution and other scenarios.

Advantages : Single-machine throughput of 100,000 levels , very high availability, distributed architecture, zero message loss , MQ function is relatively complete, distributed, good scalability, supports 1 billion level message accumulation , and will not be affected by accumulation Causes performance degradation. The source code is java. We can read the source code ourselves and customize our own company's MQ.

Disadvantages : There are not many supported client languages , currently Java and C++, of which C++ is immature; the community activity is average, and interfaces such as JMS are not implemented in the MQ core. Migration of some systems requires a lot of code modifications.

4. RabbitMQ
was released in 2007. It is a reusable enterprise messaging system based on AMQP (Advanced Message Queuing Protocol). It is one of the most mainstream message middleware currently .

Advantages : Due to the high concurrency characteristics of the erlang language , the performance is better; the throughput reaches 10,000 levels , the MQ function is relatively complete, robust, stable, easy to use, cross-platform, and supports multiple languages ​​​​such as: Python, Ruby, .NET, Java, JMS, C, PHP, ActionScript, XMPP, STOMP, etc., support AJAX and have complete documentation; the management interface provided by the open source is very good, easy to use, and the community is highly active ; the update frequency is quite high

Official website : https://www.rabbitmq.com/news.html

Disadvantages : The commercial version requires a fee and the learning cost is high

1.1.4.MQ selection

1.
Kafka The main feature of Kafka is to handle message consumption based on the Pull mode and pursue high throughput. Its initial purpose is to use log collection and transmission, which is suitable for the data collection business of Internet services that generate large amounts of data . Large companies are recommended to use it. If there is a log collection function, kafka is definitely the first choice. Shang Silicon Valley official website kafka video link http://www.gulixueyuan.com/course/330/tasks

2. RocketMQ
was born for the financial Internet field . For scenarios with high reliability requirements, especially order deductions in e-commerce and business peak shaving, when a large number of transactions pour in, the backend may not be able to process it in time. . RoketMQ may be more trustworthy in terms of stability. These business scenarios have been tested many times in Alibaba Double 11. If your business has the above concurrency scenarios, it is recommended to choose RocketMQ.
3. RabbitMQ
combines the concurrency advantages of the erlang language itself, with good performance and microsecond timeliness, and relatively high community activity . The management interface is very convenient to use. If your data volume is not that large , small and medium-sized companies will give priority to it with relatively complete functions. RabbitMQ.

1.2.RabbitMQ

1.2.1. Concept of RabbitMQ

RabbitMQ is a messaging middleware : it accepts and forwards messages. You can think of it as a courier site . When you want to send a package, you put your package at the courier station, and the courier will eventually deliver your package to the recipient. According to this logic, RabbitMQ is a courier. Station, a courier will help you deliver the express. The main difference between RabbitMQ and express delivery stations is that it does not process express mail but receives, stores and forwards message data.

1.2.2. Four core concepts

• MQ consists of two parts: exchange and queue
• Switch----》Queue: one-to-many
• Queue----》Consumer: one-to-one (one queue cannot correspond to multiple consumers, because one express delivery cannot have 2 recipients)

Producer
The program that generates data and sends messages is the producer

Switch
The switch is a very important component of RabbitMQ . On the one hand, it receives messages from producers, and on the other hand, it pushes messages to the queue. The switch must know exactly how to process the messages it receives, whether to push these messages to a specific queue or to multiple queues, or to discard the messages. This depends on the type of switch.

queue

A queue is a data structure used internally by RabbitMQ , and although messages flow through RabbitMQ and the application, they can only be stored in the queue. A queue is limited only by the memory and disk limitations of the host and is essentially a large message buffer. Many producers can send messages to a queue, and many consumers can try to receive data from a queue. This is how we use queues

Consumer
Consumption has a similar meaning to receiving. A consumer is mostly a program waiting to receive messages . Please note that the producer, consumer and message middleware are often not on the same machine. The same application can be both a producer and a consumer.

1.2.3. RabbitMQ core part (six major modes)

1. Simple mode
2. Operating mode
3. publish/subscribe model
4. routing mode
5. theme mode
6. Release confirmation mode

1.2.4. Introduction to each term

• Broker : An application that receives and distributes messages. RabbitMQ Server is the Message Broker.
• Virtual host : Designed for multi-tenancy and security reasons, the basic components of AMQP are divided into a virtual group, similar to the namespace concept in the network. When multiple different users use the services provided by the same RabbitMQ server, multiple vhosts can be divided, and each user creates exchange/queue, etc. in their own vhost.
• Connection : TCP connection between publisher/consumer and broker
• Channel (channel: channel for sending messages) : If a Connection is established every time RabbitMQ is accessed, the overhead of establishing a TCP Connection will be huge and the efficiency will be low when the amount of messages is large. Channel is a logical connection established inside the connection. If the application supports multi-threading, each thread usually creates a separate channel for communication. The AMQP method includes the channel id to help the client and message broker identify the channel, so the channels are completely isolated. of. Channel, as a lightweight Connection, greatly reduces the operating system's overhead in establishing TCP connections.
• Exchange : The message reaches the first stop of the broker. According to the distribution rules, it matches the routing key in the query table and distributes the message to the queue. Commonly used types are: direct (point-to-point), topic (publish-subscribe) and fanout (multicast)
• Queue : The message is finally sent here to wait for the consumer to pick it up
• Binding : The virtual connection between exchange and queue. The binding can contain the routing key. The Binding information is saved to the query table in the exchange and is used as the basis for message distribution.

1.2.5.Installation

1) Official website address

https://www.rabbitmq.com/download.html

Note: Enterprises generally use Linux systems for work.

2) File upload

illustrate:

• Method 1: Create a new virtual machine, view details...

• Method 2: Clone a virtual machine. View details...
  Modify the host name node1and IP address to192.168.10.120
  

• Upload to /usr/local/softwarethe directory (if there is no software, you need to create it yourself)
  

  • Go to the local directory and create a new software directory
    
  • Then use the xftp tool to upload the file to the corresponding directory.
    

3) Installation files (install in the following order)

# 安装erlang环境
rpm -ivh erlang-21.3-1.el7.x86_64.rpm 

# 安装依赖包
yum install socat -y

# 安装rabbitmq
rpm -ivh rabbitmq-server-3.8.8-1.el7.noarch.rpm 

step:

• Enter the software directory and find that the file has been uploaded. The suffix el7 indicates that the Linux7 version is supported.
  
• Install in sequence
  

4) Commonly used commands (execute in the following order)

Add the RabbitMQ service to start at boot:

 chkconfig rabbitmq-server on 

Start the service:

/sbin/service rabbitmq-server start   

Check service status:

/sbin/service rabbitmq-server status

Stop the service (select execution):

# 停止服务
/sbin/service rabbitmq-server stop

Enable the web management plug-in, and then enter the start command to restart the service.
Note : For the convenience of later use, you can install a RabbitMQ background management interface (web management plug-in), which can be accessed through a browser, but the service needs to be turned off.

# 先开启 web 管理插件
rabbitmq-plugins enable rabbitmq_management

# 之后再重启服务
/sbin/service rabbitmq-server start

Go to the Windows desktop and open the browser. Use the default account password (guest) to access the address http://192.168.10.120:15672/(host IP + port number). There is a permission problem and you cannot access it normally. It is possible that the firewall is not closed.

Solution: Turn off the firewall or enable the port number

• First check the firewall status and find that the firewall is on.
  systemctl status firewalld
  
• Turn off the firewall and automatically start the service after turning off the firewall:
  systemctl stop firewalld
systemctl disable firewalld.service
  

Problem: When I open the Windows page again and enter the address for access, and enter the initialized account and passwordguest , it prompts that the guest does not have permission to log in.

Solution : Create an account and grant super administrator rights.

5) Add a new user

Create an account: username, password

rabbitmqctl add_user admin 123456

Set user role (administrator: administrator)

rabbitmqctl set_user_tags admin administrator

Set user permissions
set_permissions [-p <vhostpath>] <user> <conf> <write> <read> . User user_admin has configuration, write, and read permissions for all resources in the virtual host /vhost1.

rabbitmqctl set_permissions -p "/" admin ".*" ".*" ".*"

View current users and roles

rabbitmqctl list_users 

6) Log in using the admin user again

Username, password: admin, 123456

7) Reset command

The command to close the application is

rabbitmqctl stop_app

The clear command is

rabbitmqctl reset

The restart command is

rabbitmqctl start_app

2.Hello World (simple mode)

In this part of the tutorial, we will write two programs in Java. A producer that sends a single message and a consumer that receives the message and prints it out . We'll cover some details in the Java API.

In the diagram below, "P" is our producer and "C" is our consumer. The middle box is a queue - a buffer of messages retained by RabbitMQ on behalf of the consumer

Note : Now that the middleware RabbitMQ has been installed in the Linux system, you only need to write java programs (producer and consumer) in idea to test whether messages can be sent.

2.1. Dependence

Create a maven project (here replaced by module) and introduce dependencies.

 <!--指定 jdk 编译版本-->
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>8</source>
                    <target>8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <!--rabbitmq 依赖客户端-->
        <dependency>
            <groupId>com.rabbitmq</groupId>
            <artifactId>amqp-client</artifactId>
            <version>5.8.0</version>
        </dependency>
        <!--操作文件流的一个依赖-->
        <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.6</version>
        </dependency>
    </dependencies>

2.2.Message producer

package com.atguigu.rabbitmq.one;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

/**
 * 生产者：发消息
 */
public class Producer {
    
    

    //队列名称
    private final static String QUEUE_NAME = "hello";

    //发消息
    public static void main(String[] args) throws Exception{
    
    
        //创建一个连接工厂
        ConnectionFactory factory = new ConnectionFactory();
        //工厂ip 连接RabbitMQ的队列
        factory.setHost("192.168.10.120");
        //用户名
        factory.setUsername("admin");
        //密码
        factory.setPassword("123456");

        //创建连接
        Connection connection = factory.newConnection();
        //获取信道
        Channel channel = connection.createChannel();
        //入门级测试，这里直接连接的是队列，没有连接交换机，用的是默认的交换机。
        /**
         * 生成一个队列,参数解释：
         *  1.queue：队列名称
         *  2.durable：是否持久化，当MQ重启后，还在 （true：持久化，保存在磁盘    false：不持久化，保存在内存）
         *  3.exclusive： false不独占，可以多个消费者消费。true只能供一个消费者进行消费
         *      功能1：是否独占。只能有一个消费者监听这个队列
         *      功能2：当Connection关闭时，是否删除队列
         *  4.autoDelete：是否自动删除 最后一个消费者端开连接以后 该队列是否自动删除 true自动删除 false不自动删除
         *  5.arguments：其它参数，在高级特性中讲，暂时设置为null
         */
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        //发消息
        String message = "hello world"; //初次使用

        /**
         *发送一个消息
         *1.发送到那个交换机       本次是入门级程序，没有考虑交换机，可以写空串
         *2.路由的 key 是哪个     本次是队列的名称
         *3.其他的参数信息        本次没有
         *4.发送消息的消息体      不能直接发消息，需要调用它的二进制。
         */
        channel.basicPublish("",QUEUE_NAME,null,message.getBytes());
        System.out.println("消息发送完毕");
    }
}

Run the main method test: a prompt message appears, indicating that the execution is successful.

View the background management interface:

2.3.Message consumer

package com.atguigu.rabbitmq.one;

import com.rabbitmq.client.*;

/**
 * 消费者：接收消息的
 */
public class Consumer {
    
    
    //队列名称
    private final static String QUEUE_NAME = "hello";

    //接收消息
    public static void main(String[] args) throws Exception {
    
    
        //创建连接工厂
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("192.168.10.120");//设置ip
        factory.setUsername("admin");//设置用户名
        factory.setPassword("123456");//设置密码
        Connection connection = factory.newConnection();//创建连接
        Channel channel = connection.createChannel();//通过连接创建信道
        System.out.println("等待接收消息	");


        //推送的消息如何进行消费的接口回调 使用lambda表达式代替匿名内部类的写法
        DeliverCallback deliverCallback = (consumerTag, message) -> {
    
    
            //直接输出message参数是对象的地址值，通过方法获取message对象的消息体并转化为字符串输出。
            String mes = new String(message.getBody());
            System.out.println(mes);
        };
        //取消消费的一个回调接口 如在消费的时候队列被删除掉了
        CancelCallback cancelCallback = (consumerTag) -> {
    
    
            System.out.println("消息消费被中断");
        };
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(QUEUE_NAME, true, deliverCallback, cancelCallback);
    }
}

3.Work Queues (work mode)

The main idea of ​​a work queue (also called a task queue) is to avoid executing a resource-intensive task immediately and having to wait for it to complete. Instead we schedule the task to be executed later. We encapsulate the task as a message and send it to the queue. A worker process running in the background will pop out the tasks and eventually execute the job. When there are multiple worker threads, these worker threads will process these tasks together.

• Summarize:
  • When a producer sends a large number of messages to the queue, it would be too slow to receive and process them one by one with only one worker thread (consumer), so multiple worker threads are used to process them at the same time.
• Principles to follow:
  • The message sent by the producer can only be processed once, so the working mode is characterized by following the polling distribution of messages, that is, the worker thread processes the message in turn.
  • There is a competitive relationship between working threads. After working thread 1 grabs the same message, other working threads cannot grab it.

3.1. Rotation training distribution message

In this case we will start two worker threads and one message sending thread. Let's take a look at how these two worker threads work.

3.1.1. Extraction tool class

package com.atguigu.rabbitmq.utils;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

/**
 * 此类为连接工厂创建信道的工具类
 */
public class RabbitMqUtils {
    
    
    //得到一个连接的 channel
    public static Channel getChannel() throws Exception{
    
    
        //创建一个连接工厂
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("192.168.10.120");
        factory.setUsername("admin");
        factory.setPassword("123456");
        Connection connection = factory.newConnection();
        Channel channel = connection.createChannel();
        return channel;
    }
}

3.1.2. Start two worker threads (consumers)

• Worker thread code:
  

package com.atguigu.rabbitmq.two;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.CancelCallback;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

/**
 * 这是一个工作线程（相当于之前消费者)
 */
public class Worker01 {
    
    
    //队列的名称
    private static final String QUEUE_NAME="hello";

    //接收消息
    public static void main(String[] args) throws Exception {
    
    
        //通过工具类获取信道
        Channel channel = RabbitMqUtils.getChannel();

        //消息的接收
        DeliverCallback deliverCallback=(consumerTag,message)->{
    
    
            String receivedMessage = new String(message.getBody());
            System.out.println("接收到消息:"+receivedMessage);
        };

        //消息接收被取消时 执行下面的内容
        CancelCallback cancelCallback=(consumerTag)->{
    
    
            System.out.println(consumerTag+"消费者取消消费接口回调逻辑");
        };


        System.out.println("C2等待接收消息...");
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(QUEUE_NAME,true,deliverCallback,cancelCallback);
    }
}

• Note: The code of the two worker threads is exactly the same, so there is no need to copy it. It can be configured in the idea tool to allow multiple threads to run.
• step:

  • Run a consumer first, and then make modifications. You can locate the current program that is running.
    

  • Worker thread 1 running window
    

  • Configure run to run multiple runs.
    

  • Then change the prompt message to the prompt message of work queue 2
    

  • Click Run. At this time, the code of worker thread 2 is running.
    

3.1.3. Start a sending thread (producer)

package com.atguigu.rabbitmq.two;


import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;

import java.util.Scanner;

/**
 * 生产者 发送大量的消息
 */
public class Task01 {
    
    
    //队列的名称
    private static final String QUEUE_NAME="hello";

    //发送大量的消息
    public static void main(String[] args) throws Exception {
    
    

        //通过工具类创建信道
        try(Channel channel= RabbitMqUtils.getChannel();) {
    
    
            /**
             * 生成一个队列,参数解释：
             *  1.queue：队列名称
             *  2.durable：是否持久化，当MQ重启后，还在 （true：持久化，保存在磁盘    false：不持久化，保存在内存）
             *  3.exclusive： false不独占，可以多个消费者消费。true只能供一个消费者进行消费
             *      功能1：是否独占。只能有一个消费者监听这个队列
             *      功能2：当Connection关闭时，是否删除队列
             *  4.autoDelete：是否自动删除 最后一个消费者端开连接以后 该队列是否自动删除 true自动删除 false不自动删除
             *  5.arguments：其它参数，在高级特性中讲，暂时设置为null
             */
            channel.queueDeclare(QUEUE_NAME,false,false,false,null);

            //从控制台当中接受信息进行发送，之前是直接定义一个参数写死了
            Scanner scanner = new Scanner(System.in);

            while (scanner.hasNext()){
    
     //判断是否还有输入的数据，有才进行循环获取
                String message = scanner.next();
                /**
                 *发送一个消息
                 *1.发送到那个交换机       本次是入门级程序，没有考虑交换机，可以写空串，表示使用默认的交换机
                 *2.路由的 key 是哪个     本次是队列的名称
                 *3.其他的参数信息        本次没有
                 *4.发送消息的消息体      不能直接发消息，需要调用它的二进制。
                 */
                channel.basicPublish("",QUEUE_NAME,null,message.getBytes());
                System.out.println("发送消息完成:"+message);
            }
        }
    }
}

3.1.4.Result display

Through program execution, it was found that the producer sent a total of 4 messages, and consumer 1 and consumer 2 received two messages each, and received the messages one at a time in order.

3.2.Message response

3.2.1. Concept

It may take a while for consumers to complete a task, what happens if one of the consumers is working on a long task and only partially completes it and suddenly it hangs . As soon as RabbitMQ delivers a message to a consumer, it immediately marks the message for deletion . In this case, if a consumer suddenly hangs up, we will lose the message being processed . and subsequent messages sent to the consumer because it cannot receive them.

In order to ensure that the message is not lost during the sending process, rabbitmq introduces a message response mechanism. The message response is: after the consumer receives the message and processes the message, it tells rabbitmq that it has processed it, and rabbitmq can delete the message.

Classification of message responses:

• Auto answer
• Manual answer

3.2.2. Automatic response (not recommended)

The message is considered to have been successfully delivered immediately after it is sent. This mode requires a trade-off between high throughput and data transmission security , because in this mode, if a connection occurs on the consumer side or the channel is closed before the message is received, then the message It is lost. Of course, on the other hand, the consumer side of this model can deliver overloaded messages without limiting the number of messages delivered . Of course, this may cause the consumer side to receive too many messages that are too late to process, resulting in The backlog of these messages eventually exhausts the memory, and eventually these consumer threads are killed by the operating system, so this mode is only suitable for use when the consumer can process these messages efficiently and at a certain rate.

Summarize:

• Automatic answering has shortcomings, and a good environment must be ensured before it can be used.
• As long as the automatic response receives the message, it will immediately tell the queue that it has been completed. In fact, it has not been completed. Once there is a problem with the subsequent code, the message will also be lost.
  

3.2.3. Manual answer

1) Method of manual message response

• A.Channel.basicAck (for positive confirmation)
  • RabbitMQ already knows the message and successfully processed it, so it can discard it.
• B.Channel.basicNack (for negative confirmation)
• C.Channel.basicReject (for negative confirmation)
  • Compared with Channel.basicNack, there is one less batch processing parameter (Multiple)
  • If you don’t want to process the message, just reject it and discard it.

2) Explanation of Multiple

The advantage of manual answering is that it can answer in batches and reduce network congestion

The true and false of multiple represent different meanings

• true represents batch response to unanswered messages on the channel
  • For example, there are messages with tags 5,6,7,8 on the channel and the current tag is 8. Then at this time
  • These unanswered messages from 5 to 8 will be acknowledged as receiving message responses.
• false compared to above (recommended)
  • Only messages 5, 6, and 7 with tag=8 will be responded to. These three messages will still not be acknowledged.
  • It is recommended not to use batch response. Processing 5 6 7 after processing 8 may also cause message loss, so batch response is generally not recommended.

3) Messages are automatically re-queued

If the consumer loses connection for some reason (its channel is closed, the connection is closed, or the TCP connection is lost) and the message is not sent an ACK, RabbitMQ will understand that the message was not fully processed and will requeue it. If another consumer can handle it at this time, it will quickly redistribute it to another consumer. This way, even if a consumer dies occasionally, you can be sure that no messages are lost.

• Summarize:
  • If a message is lost at one point, how to ensure the integrity of the message? ?
  • Answer: Messages are automatically re-queued
  • When the queue detects which message is disconnected, it will immediately save the message and put it back in the queue for processing by the next consumer to ensure that the message will not be lost.

4) Message manual response code

Note : It has nothing to do with the producer. The producer is only responsible for sending messages to the queue, and the worker thread is responsible for responding. Because the problem occurs in the worker thread, you should not respond until the code of the worker thread is executed. You should wait for the code execution to complete and respond manually. , so code modifications should be made in the worker thread.

The default message uses automatic response, so if we want to ensure that the message is not lost during the consumption process, we need to change the automatic response to manual response. On the basis of the above code, the consumer adds the code drawn in red below.

Write a new producer and consumer code to test:

• message producer
  

package com.atguigu.rabbitmq.three;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;

import java.util.Scanner;

/**
 * 消息在手动应答时是不丢失、一旦丢失会自动放回队列中重新消费
 */
public class Task02 {
    
    

    //队列名称
    private static final String TASK_QUEUE_NAME = "ack_queue";

    public static void main(String[] argv) throws Exception {
    
    

            //通过工具类获取信道
            Channel channel = RabbitMqUtils.getChannel();

            //声明队列
            channel.queueDeclare(TASK_QUEUE_NAME, false, false, false, null);
            Scanner sc = new Scanner(System.in);
            System.out.println("请输入信息");

            while (sc.hasNext()) {
    
    
                String message = sc.nextLine();
                /**
                 *发送一个消息
                 *1.发送到那个交换机       本次是入门级程序，没有考虑交换机，可以写空串，表示使用默认的交换机
                 *2.路由的 key 是哪个     本次是队列的名称
                 *3.其他的参数信息        本次没有
                 *4.发送消息的消息体      不能直接发消息，需要调用它的二进制。中文汉字有可能是乱码，所以
                 *                     如果发送的是中文，一般需要指定编码格式。
                 */
                channel.basicPublish("", TASK_QUEUE_NAME, null, message.getBytes("UTF-8"));
                System.out.println("生产者发出消息" + message);
            }

    }
}

Consumer 01

package com.atguigu.rabbitmq.three;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.atguigu.rabbitmq.utils.SleepUtils;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

/**
 * 消息在手动应答时是不丢失、一旦丢失会自动放回队列中重新消费
 */
public class Work03 {
    
    
    //队列名称
    private static final String ACK_QUEUE_NAME="ack_queue";

    //接收消息
    public static void main(String[] args) throws Exception {
    
    
        //通过工具类获取信道
        Channel channel = RabbitMqUtils.getChannel();
        System.out.println("C1 等待接收消息处理时间较短");

        //接收消息
        DeliverCallback deliverCallback=(consumerTag, delivery)->{
    
    
            String message= new String(delivery.getBody(),"utf-8");
            //接收到消息后，沉睡1s
            SleepUtils.sleep(1);
            System.out.println("接收到消息:"+message);
            /**
             * 确认手动应答：
             *  1.消息标记 tag：每一个消息都有一个唯一的标识，表示应答的是哪一个消息。
             *  2.是否批量应答未应答消息 false:不批量应答信道中的消息 true:批量
             */
            channel.basicAck(delivery.getEnvelope().getDeliveryTag(),false);
        };

        //采用手动应答
        boolean autoAck=false;
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(ACK_QUEUE_NAME,autoAck,deliverCallback,(consumerTag)->{
    
    
            System.out.println(consumerTag+"消费者取消消费接口回调逻辑");
        });
    }
}

Consumer 02

package com.atguigu.rabbitmq.three;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.atguigu.rabbitmq.utils.SleepUtils;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

/**
 * 消息在手动应答时是不丢失、一旦丢失会自动放回队列中重新消费
 */
public class Work04 {
    
    
    //队列名称
    private static final String ACK_QUEUE_NAME="ack_queue";

    //接收消息
    public static void main(String[] args) throws Exception {
    
    
        //通过工具类获取信道
        Channel channel = RabbitMqUtils.getChannel();
        System.out.println("C2 等待接收消息处理时间较长");

        //接收消息
        DeliverCallback deliverCallback=(consumerTag, delivery)->{
    
    
            String message= new String(delivery.getBody(),"utf-8");
            //接收到消息后，沉睡30s
            SleepUtils.sleep(30);
            System.out.println("接收到消息:"+message);
            /**
             * 确认手动应答：
             *  1.消息标记 tag：每一个消息都有一个唯一的标识，表示应答的是哪一个消息。
             *  2.是否批量应答未应答消息 false:不批量应答信道中的消息 true:批量
             */
            channel.basicAck(delivery.getEnvelope().getDeliveryTag(),false);
        };

        //采用手动应答
        boolean autoAck=false;
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(ACK_QUEUE_NAME,autoAck,deliverCallback,(consumerTag)->{
    
    
            System.out.println(consumerTag+"消费者取消消费接口回调逻辑");
        });
    }
}

Sleep tools

public class SleepUtils {
    
    
	 public static void sleep(int second){
    
    
		 try {
    
    
		     Thread.sleep(1000*second);
		 } catch (InterruptedException _ignored) {
    
    
		     Thread.currentThread().interrupt();
		 }
	 }
}

5) Demonstration of manual response effect

Under normal circumstances, the message sender sends two messages, C1 and C2, and receives and processes the messages respectively.

After the sender sends message dd, the C2 consumer is stopped after sending the message. Logically speaking, C2 should process the message, but due to its long processing time, it has not been processed yet, which means that C2 has not executed the ack code. At this time, C2 is stopped. At this time, you will see that the message is received by C1, indicating that the message dd is re-enqueued and then assigned to C1 that can process the message.

3.3.RabbitMQ persistence

3.3.1. Concept

We have just seen how to deal with the situation of not losing tasks, but how to ensure that the messages sent by the message producer are not lost when the RabbitMQ service is stopped. By default when RabbitMQ exits or crashes for some reason, it ignores queues and messages unless it is told not to do so. Ensuring that messages are not lost requires two things: We need to mark both the queue and the message as persistent.

3.3.2. How to achieve persistence in queues

The queues we created before are all non-persistent. If rabbitmq is restarted, the queue will be deleted. If you want the queue to be durable, you need to set the durable parameter to persistent when declaring the queue.

Make modifications in the producer:

However, it should be noted that if the previously declared queue is not persistent, the original queue needs to be deleted first, or a persistent queue must be re-created, otherwise an error will occur.

Delete the original queue:

Run the producer consumer again: The following is the UI display area of ​​the persistent and non-persistent queues in the console,

At this time, even if rabbitmq is restarted, the queue still exists

3.3.3. Message persistence

To make the message persistent, you need to modify the code in the message producer and MessageProperties.PERSISTENT_TEXT_PLAINadd this attribute.

Make modifications in the producer:

Marking a message as persistent does not completely guarantee that the message will not be lost. Although it tells RabbitMQ to save the message to disk, there is still a gap where the message is still cached when it is just about to be stored on disk but not yet. Nothing is actually written to disk at this time. The durability guarantees are not strong, but for our simple task queue it's more than enough. If you need a more powerful persistence strategy, refer to the courseware release confirmation chapter below.

3.3.4. Unfair distribution

At the beginning, we learned that RabbitMQ uses rotation distribution by default to distribute messages , but this strategy is not very good in certain scenarios. For example, there are two consumers processing tasks, and one of them, consumer 1, is processing the task. The speed of consumer 2 is very fast, while the processing speed of another consumer 2 is very slow . At this time, we still use rotation training distribution. The consumer with the fast processing speed will be idle for a large part of the time , while the consumer with the slow processing speed will be idle for a large part of the time. The consumer has been working, and this distribution method is actually not very good in this case, but RabbitMQ does not know this situation and it still distributes it fairly.

To avoid this situation, we can set the parameter channel.basicQos(1);

Modifications in the producer-consumer:


Test: It is found that the one with less time does more, and the one with longer time does less. (Those who can do more work)
Benefits: Compared with the previous polling mechanism, unfair distribution allows idle consumers to process more messages, fully utilizing the capabilities of consumers and improving efficiency.

This means that if I haven't finished processing this task or I haven't responded to you yet, don't assign it to me yet. I can only handle one task at the moment, and then rabbitmq will assign the task to the idle consumer who is not so busy. Of course, if all consumers have not completed the tasks at hand and the queue continues to add new tasks, the queue may be full. At this time, you can only add new workers or change other storage Mission strategy.

3.3.5. Prefetch value

The message itself is sent asynchronously, so at any time, there must be more than one message on the channel, and the manual confirmation from the consumer is also asynchronous in nature. So there is an unacknowledged message buffer here, so I hope developers can limit the size of this buffer to avoid the problem of unlimited unacknowledged messages in the buffer . This can be accomplished by setting the "prefetch count" value using the basic.qos method. This value defines the maximum number of unacknowledged messages allowed on the channel . Once the number reaches the configured number, RabbitMQ will stop delivering more messages on the channel unless at least one unprocessed message is acknowledged, for example, suppose there are unacknowledged messages 5, 6, 7, 8 on the channel, and the channel The prefetch count is set to 4. At this time RabbitMQ will not deliver any more messages on this channel unless at least one unanswered message is acked. For example, if the message with tag=6 has just been ACKed, RabbitMQ will sense this situation and send another message. Message acknowledgment and QoS prefetch values ​​have a significant impact on user throughput. Generally, increasing prefetching will increase the speed of message delivery to consumers. Although the auto-answer transmission message rate is optimal, the number of delivered but not yet processed messages will also increase in this case, thereby increasing the consumer's RAM consumption (Random Access Memory) and should be used with care with unlimited In the automatic confirmation mode or manual confirmation mode of preprocessing, if the consumer consumes a large number of messages without confirmation, the memory consumption of the consumer's connection node will increase, so finding the appropriate prefetch value is a process of trial and error. Different This value also varies across loads. Values ​​in the range of 100 to 300 generally provide the best throughput without posing too much risk to consumers. A prefetch value of 1 is the most conservative. Of course, this will make the throughput very low, especially if the consumer connection delay is serious, especially in an environment where the consumer connection wait time is long. For most applications, a slightly higher value will be optimal.

Summary :

• RabbitMQ's prefetch value refers to the number of messages that a consumer obtains at one time when obtaining messages from the queue. By setting appropriate prefetch values, message distribution and consumer load balancing can be optimized.

• In RabbitMQ, the prefetch value refers to the number of messages the consumer gets from the queue. When a consumer processes a message, it can get multiple messages at once instead of just one message at a time. By setting appropriate prefetch values, the efficiency of message processing can be improved and network latency and communication overhead between consumers can be reduced.
  

• Modify code test: make modifications in consumer
  
  

• Effect: Send 7 messages. It is expected that consumer 1 will receive 2 messages and consumer 2 will receive 5 messages. However, in fact, consumer 1 will receive 3 messages and consumer 2 will receive 4 messages.

• reason:

  • First of all, there is a competitive relationship between consumers, so the order of messages is not consistent. For example, the first message is sent to consumer 1, and it may also be sent to consumer 2.
  • The prefetch value actually sets the maximum accumulation of messages in the channel (the pipe between the queue and the consumer). It does not mean that the consumer will accept as many messages as you set.
  • Because Consumer 1 only takes 1 second to process the message, and Consumer 2’s processing is set to 30 seconds, it is possible that when the third message is sent, Consumer 1’s first message has already been processed, so there will be another message in the future. Send message to consumer 1.
  • When the set prefetch value of 7 is exceeded, the 8th piece of data will be processed by the consumer that consumes it faster.

4. Release confirmation

Note : To ensure that the messages sent by the producer are not lost, the following three steps are required.

1. The setting requires that the queue must be persistent (this ensures that once the RabbitMq server goes down, the queue will not be lost)
2. The settings require that messages in the queue must be persisted (messages in the queue can be guaranteed not to be lost)
3. Release confirmation (even if the first two items have been set up, the machine will be down before the message is transferred to the queue and stored on the disk. At this time, the message will still be lost, so you need to set up release confirmation)

Release confirmation overview :

• After the producer sends the message to the queue and saves it to the disk, MQ will return a prompt message to the producer, telling the producer that the message has been saved on the disk. Only at this time can it be guaranteed that the message will not be lost.

4.1. Principle of release confirmation

The producer sets the channel to confirm mode. Once the channel enters confirm mode, all messages published on the channel will be assigned a unique ID (starting from 1). Once the message is delivered to all matching queues, the broker A confirmation will be sent to the producer (containing the unique ID of the message), which allows the producer to know that the message has correctly arrived at the destination queue. If the message and queue are durable, the confirmation message will write the message to disk. Afterwards, the delivery-tag field in the confirmation message sent back to the producer by the broker contains the sequence number of the confirmation message. In addition, the broker can also set the multiple field of basic.ack to indicate that all messages up to this sequence number have been received. deal with.

The biggest advantage of confirm mode is that it is asynchronous. Once a message is published, the producer application can continue to send the next message while waiting for the channel to return confirmation. When the message is finally confirmed, the producer application can use the callback method To process the confirmation message, if RabbitMQ causes the message to be lost due to its own internal error, a nack message will be sent. The producer application can also handle the nack message in the callback method.

4.2. Strategy for release confirmation

4.2.1. Method to enable release confirmation

Release confirmation is not enabled by default . If you want to enable it, you need to call the confirmSelect method. Whenever you want to use release confirmation, you need to call this method on the channel.

• Call this method after the producer's channel creation:
  
• test:

   //开启发布确认
   channel.confirmSelect();

4.2.2.Single confirmed release

This is a simple confirmation method. It is a way of synchronously confirming the release . That is, after a message is released, only if it is confirmed to be released, subsequent messages can continue to be released. The waitForConfirmsOrDie(long) method can only be released after the message is confirmed. It will return only when the time is up. If the message is not confirmed within the specified time range, it will throw an exception.

The biggest disadvantage of this confirmation method is that the publishing speed is extremely slow , because if the published message is not confirmed, it will block the publishing of all subsequent messages. This method can provide a throughput of no more than hundreds of published messages per second. . Of course for some applications this may be enough.

• Send one to confirm one (synchronous confirmation). The second piece of data will not be sent until the previous piece of data is published and confirmed, which is very slow.

• Test: Write a new producer code to test

package com.atguigu.rabbitmq.four;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;

import java.util.UUID;

/**
 * 发布确认模式
 * 使用的时间比较哪种确认方式是最好的
 * 1、单个确认
 * 2、批量确认
 * 3、异步批量确认
 */
public class ConfirmMessage {
    
    
    //批量发消息的个数
    public static final int MESSAGE_COUNT = 1000;

    public static void main(String[] args) throws Exception {
    
    
        //1、单个确认
        ConfirmMessage.publishMessageIndividually();//发布1000个单独确认消息,耗时989ms
        //2、批量确认
        //3、异步批量确认

    }

    //1、单个确认
    public static void publishMessageIndividually() throws Exception {
    
    
        //获取信道
        Channel channel = RabbitMqUtils.getChannel();
        //队列的声明
        String queueName = UUID.randomUUID().toString();
        channel.queueDeclare(queueName, false, false, false, null);

        //开启发布确认
        channel.confirmSelect();
        //开始时间
        long begin = System.currentTimeMillis();
        for (int i = 0; i < MESSAGE_COUNT; i++) {
    
    
            String message = i + "";
            //发送消息
            channel.basicPublish("", queueName, null, message.getBytes());
            //单个消息就马上发布确认，如果是true代表发送成功，使用if判断编写提示信息。
            //服务端返回 false 或超时时间内未返回，生产者可以消息重发
            boolean flag = channel.waitForConfirms();
            if (flag) {
    
    
                System.out.println("消息发送成功");
            }
        }
        //结束时间
        long end = System.currentTimeMillis();
        System.out.println("发布" + MESSAGE_COUNT + "个单独确认消息,耗时" + (end - begin) + "ms");
    }

}

• Effect: Publish 1000 separate confirmation messages, taking 989ms
  

4.2.3. Batch confirmation release

The above method is very slow. Compared with waiting for a single confirmation message, publishing a batch of messages and then confirming them together can greatly improve the throughput. Of course, the disadvantage of this method is: when a failure occurs and causes a publishing problem, it is not known what the problem is. Which message has a problem , we must save the entire batch in memory to record important information and then republish the message. Of course, this solution is still synchronous and blocks the release of messages.

• test:
  

package com.atguigu.rabbitmq.four;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;

import java.util.UUID;

/**
 * 发布确认模式
 * 使用的时间比较哪种确认方式是最好的
 * 1、单个确认
 * 2、批量确认
 * 3、异步批量确认
 */
public class ConfirmMessage {
    
    
    //批量发消息的个数
    public static final int MESSAGE_COUNT = 1000;

    public static void main(String[] args) throws Exception {
    
    
        //1、单个确认
        //ConfirmMessage.publishMessageIndividually();//发布1000个单独确认消息,耗时989ms
        //2、批量确认
        ConfirmMessage.publishMessageBatch(); //发布1000个批量确认消息,耗时78ms
        //3、异步批量确认

    }

    //2、批量确认
    public static void publishMessageBatch() throws Exception {
    
    
        //获取信道
        Channel channel = RabbitMqUtils.getChannel();
        //队列的声明
        String queueName = UUID.randomUUID().toString();
        channel.queueDeclare(queueName, false, false, false, null);

        //开启发布确认
        channel.confirmSelect();
        //开始时间
        long begin = System.currentTimeMillis();
        //批量确认消息大小
        int batchSize = 100;
        //批量发送消息 批量发布确认
        for (int i = 0; i < MESSAGE_COUNT; i++) {
    
    
            String message = i + "";
            //发送消息
            channel.basicPublish("", queueName, null, message.getBytes());
            //判断达到100条消息的时候，批量确认一次
            if(i%batchSize==0){
    
    
                //发布确认
                channel.waitForConfirms();
            }
        }
        //结束时间
        long end = System.currentTimeMillis();
        System.out.println("发布" + MESSAGE_COUNT + "个批量确认消息,耗时" + (end - begin) + "ms");

    }


}

• Effect:
  

4.2.4. Asynchronous confirmation release

Although the programming logic of asynchronous confirmation is more complicated than the previous two, it is the most cost-effective, both in terms of reliability and efficiency. It uses callback functions to achieve reliable message delivery. This middleware also uses function callbacks to ensure whether The delivery is successful. Let us explain in detail how asynchronous confirmation is implemented.

• test:
  
  

package com.atguigu.rabbitmq.four;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.ConfirmCallback;

import java.util.UUID;
import java.util.concurrent.ConcurrentNavigableMap;
import java.util.concurrent.ConcurrentSkipListMap;

/**
 * 发布确认模式
 * 使用的时间比较哪种确认方式是最好的
 * 1、单个确认
 * 2、批量确认
 * 3、异步批量确认
 */
public class ConfirmMessage {
    
    
    //批量发消息的个数
    public static final int MESSAGE_COUNT = 1000;

    public static void main(String[] args) throws Exception {
    
    
        //1、单个确认
        //ConfirmMessage.publishMessageIndividually();//发布1000个单独确认消息,耗时989ms
        //2、批量确认
        //ConfirmMessage.publishMessageBatch(); //发布1000个批量确认消息,耗时78ms
        //3、异步批量确认
        ConfirmMessage.publishMessageAsync(); //发布1000个异步发布确认消息,耗时52ms


    }


    //3、异步批量确认
    public static void publishMessageAsync() throws Exception {
    
    
        //获取信道
        Channel channel = RabbitMqUtils.getChannel();
        //队列的声明
        String queueName = UUID.randomUUID().toString();
        channel.queueDeclare(queueName, false, false, false, null);

        //开启发布确认
        channel.confirmSelect();
        //开始时间
        long begin = System.currentTimeMillis();

        //消息确认成功 回调函数
        ConfirmCallback ackCallback = (deliveryTag,multiple)->{
    
    
            System.out.println("确认的消息："+deliveryTag);
        };
        //消息确认失败 回调函数
        /**
         * 1.消息的标记
         * 2.是否为批量确认
         */
        ConfirmCallback nackCallback = (deliveryTag,multiple)->{
    
    
            System.out.println("未确认的消息："+deliveryTag);
        };
        /**
         * 准备消息的监听器 监听那些消息成功了 那些消息失败了
         *  位置：监听器要写在发送消息之前，如果写在发消息之后，有可能在发消息的
         *       过程中通知你的时候就接收不到消息了。因为你必须发完1000条消息
         *       才会触发监听器，所以要把监听器放到前面，在你没有发的时候就准备出来，
         *       在发的过程中就有可能随时监听到哪些消息成功哪些消息失败。
         * 参数1：监听哪些消息成功了
         * 参数2：监听哪些消息失败了
         * 参数类型：函数式接口，使用lambda代替匿名内部类的写法
         *
         */
        channel.addConfirmListener(ackCallback,nackCallback);//异步通知

        for(int i =0;i<MESSAGE_COUNT;i++){
    
    
            String message ="消息"+i;
            //发送消息
            channel.basicPublish("", queueName, null, message.getBytes());
        }

        //结束时间
        long end = System.currentTimeMillis();
        System.out.println("发布" + MESSAGE_COUNT + "个异步发布确认消息,耗时" + (end - begin) + "ms");
    }

}

4.2.5. How to handle asynchronous unconfirmed messages

The best solution is to put unconfirmed messages in a memory-based queue that can be accessed by the publishing thread . For example, use ConcurrentLinkedQueue( 并发链路队列) to transfer messages between confirm callbacks and the publishing thread.

• Question 1: In the previous asynchronous release confirmation, the listener has been used to asynchronously monitor which messages were confirmed successfully and which messages failed. We don’t need to process messages that are successfully confirmed, but how do we deal with messages that are failed to be confirmed? ? ?
  • Unconfirmed messages can be resent or unconfirmed messages can be saved for later re-publishing.
• Question 2: There are now two threads, the listener and the thread that sends messages, and they are asynchronous. After the message is sent, the listener is still executing, so how to find those unconfirmed messages at this time? ? ?
  • Using ConcurrentLinkedQueue( 并发链路队列)
  • The following test solves problem 2, but problem 1 is not solved.

Modify the code for asynchronous confirmation release:

package com.atguigu.rabbitmq.four;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.ConfirmCallback;

import java.util.UUID;
import java.util.concurrent.ConcurrentNavigableMap;
import java.util.concurrent.ConcurrentSkipListMap;

/**
 * 发布确认模式
 * 使用的时间比较哪种确认方式是最好的
 * 1、单个确认
 * 2、批量确认
 * 3、异步批量确认
 */
public class ConfirmMessage {
    
    
    //批量发消息的个数
    public static final int MESSAGE_COUNT = 1000;

    public static void main(String[] args) throws Exception {
    
    
        //1、单个确认
        //ConfirmMessage.publishMessageIndividually();//发布1000个单独确认消息,耗时989ms
        //2、批量确认
        //ConfirmMessage.publishMessageBatch(); //发布1000个批量确认消息,耗时78ms
        //3、异步批量确认
        ConfirmMessage.publishMessageAsync(); //发布1000个异步发布确认消息,耗时52ms


    }


    //3、异步批量确认
    public static void publishMessageAsync() throws Exception {
    
    
        //获取信道
        Channel channel = RabbitMqUtils.getChannel();
        //队列的声明
        String queueName = UUID.randomUUID().toString();
        channel.queueDeclare(queueName, false, false, false, null);

        //开启发布确认
        channel.confirmSelect();

        /**
         * 线程安全有序的一个哈希表，适用于高并发的情况(Map集合)
         * 此哈希表可以做以下3件事：
         *   1.轻松的将序号与消息进行关联
         *   2.轻松批量删除条目 只要给到序列号
         *   3.支持并发访问
         */
        ConcurrentSkipListMap<Long, String> outstandingConfirms = new
                ConcurrentSkipListMap<>();

        /**
         * 消息确认成功 回调函数
         * 1.消息序列号
         * 2.true 可以确认小于等于当前序列号的消息
         *   false 确认当前序列号消息
         */
        ConfirmCallback ackCallback = (deliveryTag,multiple)->{
    
    
            if (multiple) {
    
     //判断是否为批量确认，如果是则调用批量删除的方法
                //2）:删除掉已经确认的消息  剩下的就是未确认的消息
                ConcurrentNavigableMap<Long, String> confirmed =
                        outstandingConfirms.headMap(deliveryTag);//返回的是小于等于当前序列号的被确认消息 是一个 map
                //清除该部分被确认消息
                confirmed.clear();
            }else{
    
     //如果不是则调用单个删除的方法
                //删除当前消息
                outstandingConfirms.remove(deliveryTag);
            }
            System.out.println("确认的消息："+deliveryTag);
        };
        //消息确认失败 回调函数
        /**
         * 1.消息的标记
         * 2.是否为批量确认
         */
        ConfirmCallback nackCallback = (deliveryTag,multiple)->{
    
    
            //3）:打印一下未确认的消息都有哪些   此处测试并没有未被确认的消息
            String message = outstandingConfirms.get(deliveryTag);
            System.out.println("未确认的消息是:"+message+"::::未被确认的消息是tag:"+deliveryTag);
        };
        /**
         * 准备消息的监听器 监听那些消息成功了 那些消息失败了
         *  位置：监听器要写在发送消息之前，如果写在发消息之后，有可能在发消息的
         *       过程中通知你的时候就接收不到消息了。因为你必须发完1000条消息
         *       才会触发监听器，所以要把监听器放到前面，在你没有发的时候就准备出来，
         *       在发的过程中就有可能随时监听到哪些消息成功哪些消息失败。
         * 参数1：监听哪些消息成功了
         * 参数2：监听哪些消息失败了
         * 参数类型：函数式接口，使用lambda代替匿名内部类的写法
         *
         */
        channel.addConfirmListener(ackCallback,nackCallback);//异步通知

        //开始时间
        long begin = System.currentTimeMillis();

        for(int i =0;i<MESSAGE_COUNT;i++){
    
    
            String message ="消息"+i;
            //1）:此处记录下所有要发送的消息  消息的总和
            //   通过put方法向此map集合中添加k,v    （通过信道调取发布的序号，发送的信息）
            outstandingConfirms.put(channel.getNextPublishSeqNo(), message);
            
            //发送消息
            channel.basicPublish("", queueName, null, message.getBytes());
        }

        //结束时间
        long end = System.currentTimeMillis();
        System.out.println("发布" + MESSAGE_COUNT + "个异步发布确认消息,耗时" + (end - begin) + "ms");
    }

}

4.2.6. Comparison of the above three release confirmation speeds

• Post messages individually
  • Synchronous waiting for confirmation, simple, but very limited throughput.
• Publish messages in batches
  • Batch synchronization waits for confirmation, simple, reasonable throughput. Once a problem occurs, it is difficult to infer which message has the problem.
• Asynchronous processing: ( 推荐)
  • Optimum performance and resource usage, well controlled in case of errors, but slightly harder to achieve

public static void main(String[] args) throws Exception {
    
    
	 //这个消息数量设置为 1000 好些 不然花费时间太长
	 publishMessagesIndividually();
	 publishMessagesInBatch();
	 handlePublishConfirmsAsynchronously();
}

//运行结果
发布 1,000 个单独确认消息耗时 50,278 ms
发布 1,000 个批量确认消息耗时 635 ms
发布 1,000 个异步确认消息耗时 92 ms

5.Switch

In the previous section, we created a work queue. What we assume is that behind the work queue, each task is delivered to exactly one consumer (worker process). In this part, we're going to do something completely different - we're communicating a message to multiple consumers. This pattern is called "publish/subscribe".

Summarize:

• Previous queues used the default switch, and a message could only be consumed by one consumer . ( 简单模式、工作模式)
  

• Now if you want to consume a message twice , you can assign the message to two queues through the switch. The message in one queue can only be consumed once. ( 发布订阅模式)
  

To illustrate this pattern, we will build a simple logging system. It will consist of two programs: the first program will emit log messages and the second program will be the consumer. Among them, we will start two consumers. One consumer will store the log on the disk after receiving the message. The other consumer will print the message on the screen after receiving the message. In fact, the log message sent by the first program will be broadcast. to all consumers.

5.1.Exchanges

5.1.1.Exchanges concept

The core idea of ​​RabbitMQ's messaging model is that messages produced by producers are never sent directly to queues . In fact, often the producer doesn't even know which queues these messages are being delivered to.

Instead, producers can only send messages to an exchange . The job of an exchange is very simple. On the one hand, it receives messages from producers, and on the other hand, it pushes them into a queue. The switch must know exactly what to do with the received message. Should these messages be put into a specific queue, put them into many queues, or should they be discarded. This depends on the type of switch .

5.1.2.Types of Exchanges

In total there are the following types:

• direct (direct / routing type),
• topic,
• Headers (headers/header types),
• Fanout (fanout / publish-subscribe type)

5.1.3. Nameless exchange (default type)

Earlier in this tutorial we knew nothing about exchange but were still able to send messages to the queue. The reason it was possible before was because we were using the default exchange , which we passed 空字符串(“”)进行标识.

The first parameter is the name of the switch . An empty string represents a default or unnamed switch : the message can be routed to the queue specified by the routingKey (bindingkey) binding key, if it exists.

• That is : use the default switch, write an empty string as the first parameter, and write the queue name as the second parameter. If you specify a switch, the second parameter is routingKey.

5.2. Temporary queue

In previous chapters we used queues with specific names (remember hello and ack_queue?). The name of the queue is very important to us - we need to specify which queue our consumer should consume messages from.

Whenever we connect to Rabbit, we need a fresh, empty queue, for which we can create a queue with a random name , or even better, let the server choose a random queue name for us. Secondly, once we disconnect the consumer, the queue will be automatically deleted.

Summarize:

• A temporary queue is a queue without persistence. Once we disconnect the consumer, the queue will be automatically deleted.
  

• Here's how to create a temporary queue:
  String queueName = channel.queueDeclare().getQueue();

• After creation, it looks like this:
  

5.3.Bindings

What is binding? Binding is actually the bridge between exchange and queue. It tells us that exchange is bound to that queue. For example, what the picture below tells us is that X is bound to Q1 and Q2.

5.4.Fanout (publish-subscribe mode)

5.4.1.Fanout Introduction

Fanout This type is very simple. As you might guess from the name, it broadcasts all messages it receives to all queues it knows about. There are some exchange types in the system by default

• amq.fanout: The publish-subscribe switch provided by the system by default.
  

5.4.2.Fanout actual combat

• Scenario : A producer sends a message to the exchange, and then the exchange binds two queues (the binding conditions are the same, and the messages sent to the queues are the same). Message 1 is received by consumer 1, and message 2 is received by consumer 2.
  

The binding relationship between Logs and temporary queue is as shown below

• Switch name: logs
• 2 queue names: Randomly generated queue names.
• Binding relationship: empty string

ReceiveLogs01 prints the received messages to the console (consumer 1)

package com.atguigu.rabbitmq.five;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

/**
 * 消息的接收 01
 */
public class ReceiveLogs01 {
    
    
    //定义交换机的名称
    private static final String EXCHANGE_NAME = "logs";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        Channel channel = RabbitMqUtils.getChannel();
        //声明一个交换机：名称，类型(fanout:发布订阅类型)
        channel.exchangeDeclare(EXCHANGE_NAME, "fanout");
        /**
         * 生成一个临时的队列 队列的名称是随机的
         * 当消费者断开和该队列的连接时 队列自动删除
         */
        String queueName = channel.queueDeclare().getQueue();
        /**
         * 绑定交换机与队列：
         *  参数1：队列
         *  参数2：交换机
         *  参数3：关键词routingKey为空串
         */
        channel.queueBind(queueName, EXCHANGE_NAME, "");
        System.out.println("等待接收消息,把接收到的消息打印在屏幕.....");


        //消费者接收消息的回调
        DeliverCallback deliverCallback = (consumerTag, delivery) -> {
    
    
            String message = new String(delivery.getBody(), "UTF-8");
            System.out.println("消费者1：控制台打印接收到的消息"+message);
        };
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(queueName, true, deliverCallback, consumerTag -> {
    
     });
    }
}

ReceiveLogs02 stores received messages on disk (Consumer 2)

package com.atguigu.rabbitmq.five;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;
import org.apache.commons.io.FileUtils;

import java.io.File;

/**
 * 消息的接收 02
 */
public class ReceiveLogs02 {
    
    
    //定义交换机的名称
    private static final String EXCHANGE_NAME = "logs";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        Channel channel = RabbitMqUtils.getChannel();
        //声明一个交换机：名称，类型(fanout:发布订阅类型)
        channel.exchangeDeclare(EXCHANGE_NAME, "fanout");
        /**
         * 生成一个临时的队列 队列的名称是随机的
         * 当消费者断开和该队列的连接时 队列自动删除
         */
        String queueName = channel.queueDeclare().getQueue();
        /**
         * 绑定交换机与队列：
         *  参数1：队列
         *  参数2：交换机
         *  参数3：关键词routingKey为空串
         */
        channel.queueBind(queueName, EXCHANGE_NAME, "");
        System.out.println("等待接收消息,把接收到的消息打印在屏幕.....");


        //消费者接收消息的回调
        DeliverCallback deliverCallback = (consumerTag, delivery) -> {
    
    
            String message = new String(delivery.getBody(), "UTF-8");
            File file = new File("D:\\work\\rabbitmq_info.txt");
            /*
            * FileUtils：文件操作工具类
            *  存在于Common工具库中。包含一些工具类,它们基于File对象工作，包括读，写，拷贝和比较文件。
            * writeStringToFile（）：把字符串写入文件
            *  参数：file文件名  写入文件的内容  编码   append为true表示在文本内容后面添加内容而不会覆盖文件中的内容重写。
            * windows系统换行：\r\n
            * */
            FileUtils.writeStringToFile(file,message + "\r\n","UTF-8",true);
            System.out.println("消费者2：数据写入文件成功");
        };
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调 
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(queueName, true, deliverCallback, consumerTag -> {
    
     });
    }
}

EmitLog sends a message to two consumers (producers)

package com.atguigu.rabbitmq.five;


import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;

import java.util.Scanner;

/**
 * 生成者：发消息给交换机
 */
public class EmitLog {
    
    
    //交换机的名称
    private static final String EXCHANGE_NAME = "logs";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        try (Channel channel = RabbitMqUtils.getChannel()) {
    
    
            /**
             * 声明一个 exchange（交换机）
             * 1.exchange 的名称
             * 2.exchange 的类型
             */
            channel.exchangeDeclare(EXCHANGE_NAME, "fanout");

            //接收键盘输入的消息
            Scanner sc = new Scanner(System.in);
            System.out.println("请输入信息");
            while (sc.hasNext()) {
    
    
                String message = sc.nextLine();
                /**
                 *发送一个消息
                 *1.发送到那个交换机       如果是入门级程序，没有考虑交换机，可以写空串，表示使用默认的交换机
                 *2.路由的 key 是哪个     使用默认的交换机写的是：队列的名称   指定了交换机写的是：routingKey关键词
                 *3.其他的参数信息        本次没有
                 *4.发送消息的消息体      不能直接发消息，需要调用它的二进制。
                 */
                channel.basicPublish(EXCHANGE_NAME, "", null, message.getBytes("UTF-8"));
                System.out.println("生产者发出消息" + message);
            }
        }
    }
}

run:

• Producer:
  
• Consumer 1:
  
• Consumer 2:
  

5.5.Direct exchange (routing mode)

5.5.1. Review

In the previous section, we built a simple logging system. We are able to broadcast log messages to many receivers. In this section we will add some special functionality to it - let's say we only allow a certain consumer to subscribe to some of the messages published. For example, we only redirect critical error messages to the log file (to save disk space), while still being able to print all log messages on the console.

Let's review what bindings are again. Binding is the bridge relationship between switches and queues. It can also be understood this way: the queue is only interested in messages from the switch it is bound to . Binding is represented by the parameter: routingKey, which can also be called a binding key. To create a binding, we use the code: channel.queueBind(queueName, EXCHANGE_NAME, “routingKey”); the meaning after binding is determined by its exchange type.

5.5.2.Direct exchange introduction

Our logging system from the previous section broadcasts all messages to all consumers, we want to make some changes to this, for example we want the program that writes log messages to disk to only receive critical errors (errros) and not store warnings (warning) or information (info) log messages to avoid wasting disk space. The Fanout exchange type does not give us a lot of flexibility - it only does unconscious broadcasts. Here we will use the direct type to replace it. The way this type works is that the message only goes to The routingKey it is bound to goes to the queue.

In the picture above, we can see that X is bound to two queues, and the binding type is direct. The binding key of queue Q1 is orange, and the binding key of queue Q2 is two: one binding key is black, and the other binding key is green.

In this binding case, the producer publishes messages to exchange, and messages with the binding key orange will be published to queue Q1. Messages with binding keys blackgreen and will be published to queue Q2, and messages of other message types will be discarded.

Summary :

• Publish-subscribe mode: The binding type is Fanult, and the binding condition routingKey is the same (one person sends a message, and multiple people can receive it)
• Routing mode: The binding type is direct, and the binding conditions routingKey are different (one person sends a message and is designated to receive it, and others cannot receive it)

5.5.3.Multiple binding

Of course, if the binding type of exchange is direct, but the keys of the multiple queues it is bound to are all the same , in this case, although the binding type is direct , it behaves a bit like fanout , almost like broadcast. ,As shown in FIG.

• The binding type is direct, and the binding condition routingKey is the same: the effect is consistent with the publish-subscribe mode.

5.5.4. Actual combat

• switch:direct_logs
• Queue 1: console
• Queue 2:disk
  

producer

package com.atguigu.rabbitmq.six;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.BuiltinExchangeType;
import com.rabbitmq.client.Channel;

import java.util.HashMap;
import java.util.Map;

/**
 * 生产者
 */
public class EmitLogDirect {
    
    
    //交换机的名称
    private static final String EXCHANGE_NAME = "direct_logs";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        try (Channel channel = RabbitMqUtils.getChannel()) {
    
    
            /**
             * 声明一个 exchange（交换机）
             * 1.exchange 的名称
             * 2.exchange 的类型
             */
            channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT);

            //创建多个 bindingKey
            Map<String, String> bindingKeyMap = new HashMap<>();
            bindingKeyMap.put("info","普通 info 信息");
            bindingKeyMap.put("warning","警告 warning 信息");
            bindingKeyMap.put("error","错误 error 信息");
            //debug 没有消费这接收这个消息 所有就丢失了
            bindingKeyMap.put("debug","调试 debug 信息");
            for (Map.Entry<String, String> bindingKeyEntry: bindingKeyMap.entrySet()){
    
    
                String bindingKey = bindingKeyEntry.getKey();
                String message = bindingKeyEntry.getValue();
                /**
                 *发送一个消息
                 *1.发送到那个交换机       如果是入门级程序，没有考虑交换机，可以写空串，表示使用默认的交换机
                 *2.路由的 key 是哪个     使用默认的交换机写的是：队列的名称   指定了交换机写的是：routingKey关键词
                 *3.其他的参数信息        本次没有
                 *4.发送消息的消息体      不能直接发消息，需要调用它的二进制。
                 */
                channel.basicPublish(EXCHANGE_NAME,bindingKey, null, message.getBytes("UTF-8"));
                System.out.println("生产者发出消息:" + message);
            }


        }
    }
}

Consumer 1:

package com.atguigu.rabbitmq.six;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.BuiltinExchangeType;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

/**
 * 消费者1：
 */
public class ReceiveLogsDirect01 {
    
    
    //定义交换机的名称
    private static final String EXCHANGE_NAME = "direct_logs";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        Channel channel = RabbitMqUtils.getChannel();
        //声明一个交换机：名称，类型(direct:路由类型)
        channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT);
        /**
         * 生成一个队列,参数解释：
         *  1.queue：队列名称
         *  2.durable：是否持久化，当MQ重启后，还在 （true：持久化，保存在磁盘    false：不持久化，保存在内存）
         *  3.exclusive： false不独占，可以多个消费者消费。true只能供一个消费者进行消费
         *      功能1：是否独占。只能有一个消费者监听这个队列
         *      功能2：当Connection关闭时，是否删除队列
         *  4.autoDelete：是否自动删除 最后一个消费者端开连接以后 该队列是否自动删除 true自动删除 false不自动删除
         *  5.arguments：其它参数，在高级特性中讲，暂时设置为null
         */
        String queueName = "console";
        channel.queueDeclare(queueName, false, false, false, null);
        /**
         * 绑定交换机与队列：
         *  参数1：队列
         *  参数2：交换机
         *  参数3：关键词routingKey为info,warning
         */
        channel.queueBind(queueName, EXCHANGE_NAME, "info");
        channel.queueBind(queueName, EXCHANGE_NAME, "warning");
        System.out.println("等待接收消息.....");

        //消费者接收消息的回调
        DeliverCallback deliverCallback = (consumerTag, delivery) -> {
    
    
            String message = new String(delivery.getBody(), "UTF-8");
            System.out.println(" 消费者01 :"+delivery.getEnvelope().getRoutingKey()+",消息:"+message);
        };
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(queueName, true, deliverCallback, consumerTag -> {
    
    
        });
    }
}

Consumer 2:

package com.atguigu.rabbitmq.six;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.BuiltinExchangeType;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

/**
 * 消费者1：
 */
public class ReceiveLogsDirect02 {
    
    
    //定义交换机的名称
    private static final String EXCHANGE_NAME = "direct_logs";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        Channel channel = RabbitMqUtils.getChannel();
        //声明一个交换机：名称，类型(direct:路由类型)
        channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT);
        /**
         * 生成一个队列,参数解释：
         *  1.queue：队列名称
         *  2.durable：是否持久化，当MQ重启后，还在 （true：持久化，保存在磁盘    false：不持久化，保存在内存）
         *  3.exclusive： false不独占，可以多个消费者消费。true只能供一个消费者进行消费
         *      功能1：是否独占。只能有一个消费者监听这个队列
         *      功能2：当Connection关闭时，是否删除队列
         *  4.autoDelete：是否自动删除 最后一个消费者端开连接以后 该队列是否自动删除 true自动删除 false不自动删除
         *  5.arguments：其它参数，在高级特性中讲，暂时设置为null
         */
        String queueName = "disk";
        channel.queueDeclare(queueName, false, false, false, null);
        /**
         * 绑定交换机与队列：
         *  参数1：队列
         *  参数2：交换机
         *  参数3：关键词routingKey为error
         */
        channel.queueBind(queueName, EXCHANGE_NAME, "error");
        System.out.println("等待接收消息.....");

        //消费者接收消息的回调
        DeliverCallback deliverCallback = (consumerTag, delivery) -> {
    
    
            String message = new String(delivery.getBody(), "UTF-8");
            System.out.println(" 消费者02 :"+delivery.getEnvelope().getRoutingKey()+",消息:"+message);
        };
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(queueName, true, deliverCallback, consumerTag -> {
    
    
        });
    }
}

Test : The results show that the switch type is routing mode (direct). When binding multiple types, as long as one of the binding relationships is satisfied, messages can be sent to the queue. Binding a relationship must be satisfied before it can be sent. It cannot be sent if the binding relationship is not satisfied.

• producer
  
• Consumer 1:
  
• Consumer 2:
  

5.6.Topics (theme mode)

5.6.1. Previous types of questions

In the previous section, we improved the logging system. Instead of using a fanout switch that can only perform random broadcasts, we use a direct switch to selectively receive logs.

Although using the direct switch has improved our system, it still has limitations - for example, the log types we want to receive are info.base and info.advantage, and a certain queue only wants messages from info.base, then direct will It can't be done. At this time, only the topic type can be used

5.6.2.Topic requirements

The routing_key of the message sent to the topic switch cannot be written arbitrarily and must meet certain requirements. It must be a list of words separated by dots . These words can be any words, such as: "stock.usd.nyse", "nyse.vmw", "quick.orange.rabbit". This type. Of course, this word list cannot exceed 255 bytes at most.

In this rule list, there are two replacement characters that everyone needs to pay attention to

• *(星号)可以代替一个单词
• #(pound sign) can replace zero or more words

Summary : Compared with the publish-subscribe mode and the routing mode, it is more flexible. It can be sent conditionally or uniformly. The principle is to use wildcards to describe the binding relationship.

5.6.3.Topic matching case

The binding relationship in the figure below is as follows

• Q1–>Bound is
  • A string of 3 words with orange in the middle ( *.orange.*)
• Q2–>Bound is
  • The last word is 3 words for rabbit ( *.*.rabbit)
  • The first word is multiple words of lazy (lazy.#)

The picture above is a queue binding relationship diagram. Let’s take a look at the data reception situation between them.

quick.orange.rabbit	        被队列 Q1Q2 接收到
lazy.orange.elephant	    被队列 Q1Q2 接收到
quick.orange.fox	        被队列 Q1 接收到
lazy.brown.fox	            被队列 Q2 接收到
lazy.pink.rabbit	        虽然满足两个绑定但只被队列 Q2 接收一次
quick.brown.fox	            不匹配任何绑定不会被任何队列接收到会被丢弃
quick.orange.male.rabbit	是四个单词不匹配任何绑定会被丢弃
lazy.orange.male.rabbit	    是四个单词但匹配 Q2

Attention needs to be paid when the queue binding relationship is as follows:

• 当一个队列绑定键是#,那么这个队列将接收所有数据，就有点像 fanout 了
• 如果队列绑定键当中没有#和*出现，那么该队列绑定类型就是 direct 了

5.6.4. Actual combat

• switch: topic_logs
• Queue 1: Q1
• Queue 2: Q2

Producer:

package com.atguigu.rabbitmq.seven;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;

import java.util.HashMap;
import java.util.Map;

/**
 * 生产者：
 */
public class EmitLogTopic {
    
    
    //交换机的名称
    private static final String EXCHANGE_NAME = "topic_logs";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        try (Channel channel = RabbitMqUtils.getChannel()) {
    
    
            /**
             * 声明一个 exchange（交换机）
             * 1.exchange 的名称
             * 2.exchange 的类型
             */
            channel.exchangeDeclare(EXCHANGE_NAME, "topic");

            /**
             * Q1-->绑定的是
             * 中间带 orange 带 3 个单词的字符串(*.orange.*)
             * Q2-->绑定的是
             * 最后一个单词是 rabbit 的 3 个单词(*.*.rabbit)
             * 第一个单词是 lazy 的多个单词(lazy.#)
             *
             */
            Map<String, String> bindingKeyMap = new HashMap<>();
            bindingKeyMap.put("quick.orange.rabbit","被队列 Q1Q2 接收到");
            bindingKeyMap.put("lazy.orange.elephant","被队列 Q1Q2 接收到");
            bindingKeyMap.put("quick.orange.fox","被队列 Q1 接收到");
            bindingKeyMap.put("lazy.brown.fox","被队列 Q2 接收到");
            bindingKeyMap.put("lazy.pink.rabbit","虽然满足两个绑定但只被队列 Q2 接收一次");
            bindingKeyMap.put("quick.brown.fox","不匹配任何绑定不会被任何队列接收到会被丢弃");
            bindingKeyMap.put("quick.orange.male.rabbit","是四个单词不匹配任何绑定会被丢弃");
            bindingKeyMap.put("lazy.orange.male.rabbit","是四个单词但匹配 Q2");

            for (Map.Entry<String, String> bindingKeyEntry: bindingKeyMap.entrySet()){
    
    
                String bindingKey = bindingKeyEntry.getKey();
                String message = bindingKeyEntry.getValue();
                /**
                 *发送一个消息
                 *1.发送到那个交换机       如果是入门级程序，没有考虑交换机，可以写空串，表示使用默认的交换机
                 *2.路由的 key 是哪个     使用默认的交换机写的是：队列的名称   指定了交换机写的是：routingKey关键词
                 *3.其他的参数信息        本次没有
                 *4.发送消息的消息体      不能直接发消息，需要调用它的二进制。
                 */
                channel.basicPublish(EXCHANGE_NAME,bindingKey, null, message.getBytes("UTF-8"));
                System.out.println("生产者发出消息" + message);
            }
        }
    }
}

Consumer 1:

package com.atguigu.rabbitmq.seven;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

/**
 * 声明主题交换机 以及相关队列
 *
 * 消费者C1
 */
public class ReceiveLogsTopic01 {
    
    
    //定义交换机的名称
    private static final String EXCHANGE_NAME = "topic_logs";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        Channel channel = RabbitMqUtils.getChannel();
        //声明一个交换机：名称，类型(topic:主题类型)
        channel.exchangeDeclare(EXCHANGE_NAME, "topic");
        /**
         * 生成一个队列Q1,参数解释：
         *  1.queue：队列名称
         *  2.durable：是否持久化，当MQ重启后，还在 （true：持久化，保存在磁盘    false：不持久化，保存在内存）
         *  3.exclusive： false不独占，可以多个消费者消费。true只能供一个消费者进行消费
         *      功能1：是否独占。只能有一个消费者监听这个队列
         *      功能2：当Connection关闭时，是否删除队列
         *  4.autoDelete：是否自动删除 最后一个消费者端开连接以后 该队列是否自动删除 true自动删除 false不自动删除
         *  5.arguments：其它参数，在高级特性中讲，暂时设置为null
         */
        String queueName="Q1";
        channel.queueDeclare(queueName, false, false, false, null);
        /**
         * 绑定交换机与队列：
         *  参数1：队列
         *  参数2：交换机
         *  参数3：关键词routingKey为
         */
        channel.queueBind(queueName, EXCHANGE_NAME, "*.orange.*");
        System.out.println("等待接收消息.....");

        //消费者接收消息的回调
        DeliverCallback deliverCallback = (consumerTag, delivery) -> {
    
    
            String message = new String(delivery.getBody(), "UTF-8");
            System.out.println(" 接收队列 :"+queueName+" 绑定键:"+delivery.getEnvelope().getRoutingKey()+",消息:"+message);
        };
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(queueName, true, deliverCallback, consumerTag -> {
    
    
        });
    }
}

Consumer 2:

package com.atguigu.rabbitmq.seven;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

/**
 * 声明主题交换机 以及相关队列
 *
 * 消费者C2
 */
public class ReceiveLogsTopic02 {
    
    
    //定义交换机的名称
    private static final String EXCHANGE_NAME = "topic_logs";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        Channel channel = RabbitMqUtils.getChannel();
        //声明一个交换机：名称，类型(topic:主题类型)
        channel.exchangeDeclare(EXCHANGE_NAME, "topic");
        /**
         * 生成一个队列Q2,参数解释：
         *  1.queue：队列名称
         *  2.durable：是否持久化，当MQ重启后，还在 （true：持久化，保存在磁盘    false：不持久化，保存在内存）
         *  3.exclusive： false不独占，可以多个消费者消费。true只能供一个消费者进行消费
         *      功能1：是否独占。只能有一个消费者监听这个队列
         *      功能2：当Connection关闭时，是否删除队列
         *  4.autoDelete：是否自动删除 最后一个消费者端开连接以后 该队列是否自动删除 true自动删除 false不自动删除
         *  5.arguments：其它参数，在高级特性中讲，暂时设置为null
         */
        String queueName="Q1";
        channel.queueDeclare(queueName, false, false, false, null);
        /**
         * 绑定交换机与队列：
         *  参数1：队列
         *  参数2：交换机
         *  参数3：关键词routingKey为
         */
        channel.queueBind(queueName, EXCHANGE_NAME, "*.*.rabbit");
        channel.queueBind(queueName, EXCHANGE_NAME, "lazy.#");
        System.out.println("等待接收消息.....");

        //消费者接收消息的回调
        DeliverCallback deliverCallback = (consumerTag, delivery) -> {
    
    
            String message = new String(delivery.getBody(), "UTF-8");
            System.out.println(" 接收队列 :"+queueName+" 绑定键:"+delivery.getEnvelope().getRoutingKey()+",消息:"+message);
        };
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(queueName, true, deliverCallback, consumerTag -> {
    
    
        });
    }
}

test:

• producer
  

• Consumer 01
  

• Consumer 02
  

6.Dead letter queue

6.1. Concept of dead letter

Let’s first clarify this definition from a conceptual explanation. Dead letters , as the name suggests, are messages that cannot be consumed . The literal meaning can be understood like this. Generally speaking, the producer delivers the message to the broker or directly to the queue. , the consumer (consumer) takes out the message from the queue for consumption, but sometimes due to specific reasons, some messages in the queue cannot be consumed . If such a message is not processed subsequently, it becomes a dead letter , and there is a possibility of death. Letters naturally have a dead letter queue .

Application scenarios :

• In order to ensure that the message data of the order business is not lost, the dead letter queue mechanism of RabbitMQ needs to be used. When an exception occurs in message consumption, the message will be put into the dead letter queue.
  • When the message cannot be consumed, the message is placed in the bad-letter queue. The purpose is to wait until the environment improves to consume the messages in the bad-letter queue.
• For example, if the user successfully places an order in the mall and clicks to pay, it will automatically expire if the payment is not made within the specified time.
  • It reflects that there is a certain delay in the dead letter queue, and the dead letter can be consumed by consumers within a certain period of time.

6.2. Sources of dead letters

• Message TTL (time to live) expires
• The queue reaches the maximum length (the queue is full and no more data can be added to mq)
• The message was rejected (basic.reject or basic.nack) and requeue=false.
  • Explanation: When the message party responded, it refused to respond or responded negatively, and it has not been put back into the queue.

6.3. Dead letter practice

6.3.1. Code architecture diagram

• The producer sends the message to the normal exchange, and the exchange sends it to the queue and is consumed by the consumer. Once the message sent is a dead letter message, the dead letter message will be sent from the normal queue to the dead letter exchange, and then sent to the dead letter queue to be consumed by consumer 2.
  

6.3.2. Message TTL expiration

• Test process: Every time a message occurs in the producer, there will be a 10s timeout. If it is not consumed by the consumer within these 10 seconds, the dead letter message will be forwarded to the dead letter switch and eventually consumed by consumer 2.
• The purpose of starting consumer 1 after shutting down: after starting, it will create a normal switch, a normal queue, a dead letter queue, a dead letter queue, and the relationship between the normal switch and the dead letter switch, and the dead letter switch and rouTingKey. After closing, it can be guaranteed that the message will time out within 10 seconds after the death of consumer 1. At this time, the dead letter message will be forwarded to the dead letter switch.

producer code

package com.atguigu.rabbitmq.eight;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.BuiltinExchangeType;
import com.rabbitmq.client.Channel;

/**
 * 死信队列之 生产者代码
 */
public class Producer {
    
    

    //交换机的名称
    private static final String NORMAL_EXCHANGE = "normal_exchange";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        try (Channel channel = RabbitMqUtils.getChannel()) {
    
    
            /**
             * 声明一个 exchange（交换机）
             * 1.exchange 的名称
             * 2.exchange 的类型
             */
            channel.exchangeDeclare(NORMAL_EXCHANGE, BuiltinExchangeType.DIRECT);

            //设置消息的 TTL（超时）时间  单位是ms毫秒  10000ms=10s
            AMQP.BasicProperties properties = new AMQP.BasicProperties().builder().expiration("10000").build();
            //该信息是用作演示队列个数限制
            for (int i = 1; i <11 ; i++) {
    
    
                /**
                 *发送一个消息
                 *1.发送到那个交换机       如果是入门级程序，没有考虑交换机，可以写空串，表示使用默认的交换机
                 *2.路由的 key 是哪个     使用默认的交换机写的是：队列的名称   指定了交换机写的是：routingKey关键词
                 *3.其他的参数信息        本次没有
                 *4.发送消息的消息体      不能直接发消息，需要调用它的二进制。
                 */
                String message="info"+i;
                channel.basicPublish(NORMAL_EXCHANGE, "zhangsan", properties, message.getBytes());
                System.out.println("生产者发送消息:"+message);
            }
        }
    }
}

Consumer C1 code ( 启动之后关闭该消费者 模拟其接收不到消息)

package com.atguigu.rabbitmq.eight;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.BuiltinExchangeType;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

import java.util.HashMap;
import java.util.Map;

/**
 * 死信队列  实战
 *
 * 消费者01
 */
public class Consumer01 {
    
    
    //普通交换机名称
    private static final String NORMAL_EXCHANGE = "normal_exchange";
    //死信交换机名称
    private static final String DEAD_EXCHANGE = "dead_exchange";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        Channel channel = RabbitMqUtils.getChannel();
        //声明普通和死信交换机： 名称，类型(direct:路由模式)
        channel.exchangeDeclare(NORMAL_EXCHANGE, BuiltinExchangeType.DIRECT);
        channel.exchangeDeclare(DEAD_EXCHANGE, BuiltinExchangeType.DIRECT);

        /**
         * 声明普通队列,参数解释：
         *  1.queue：队列名称
         *  2.durable：是否持久化，当MQ重启后，还在 （true：持久化，保存在磁盘    false：不持久化，保存在内存）
         *  3.exclusive： false不独占，可以多个消费者消费。true只能供一个消费者进行消费
         *      功能1：是否独占。只能有一个消费者监听这个队列
         *      功能2：当Connection关闭时，是否删除队列
         *  4.autoDelete：是否自动删除 最后一个消费者端开连接以后 该队列是否自动删除 true自动删除 false不自动删除
         *  5.arguments：其它参数，在高级特性中讲，暂时设置为null
         *              设置消息为死信后转发到死信交换机，参数类型为Map集合
         */
        String normalQueue = "normal-queue";
        //设置消息为死信后转发到死信交换机，参数类型为Map集合
        Map<String, Object> arguments = new HashMap<>();
        //过期时间 10s=10000ms  在这个地方设置过期时间直接写死了，可以在生产者发送消息时设置过期时间，这样更灵活。
        //arguments.put("x-message-ttl",10000);
        //正常队列设置死信交换机是谁：参数key 是固定值，参数v是死信交换机的名字
        arguments.put("x-dead-letter-exchange", DEAD_EXCHANGE);
        //设置死信 routing-key: 参数 key 是固定值,参数v关键词的名字跟图中保持一致
        arguments.put("x-dead-letter-routing-key", "lisi");
        channel.queueDeclare(normalQueue, false, false, false, arguments);
        /**
         * 绑定普通交换机与普通队列：
         *  参数1：队列
         *  参数2：交换机
         *  参数3：关键词routingKey为
         */
        channel.queueBind(normalQueue, NORMAL_EXCHANGE, "zhangsan");

        //声明死信队列
        String deadQueue = "dead-queue";
        channel.queueDeclare(deadQueue, false, false, false, null);
        //绑定死信交换机与死信队列
        channel.queueBind(deadQueue, DEAD_EXCHANGE, "lisi");
        System.out.println("等待接收消息.....");

        //消费者接收消息的回调
        DeliverCallback deliverCallback = (consumerTag, delivery) -> {
    
    
            String message = new String(delivery.getBody(), "UTF-8");
            System.out.println("Consumer01 接收到消息"+message);
        };
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(normalQueue, true, deliverCallback, consumerTag -> {
    
    
        });
    }
}

Consumer C2 code ( 以上步骤完成后 启动 C2 消费者 它消费死信队列里面的消息)

package com.atguigu.rabbitmq.eight;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.BuiltinExchangeType;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

import java.util.HashMap;
import java.util.Map;

/**
 * 死信队列  实战
 *
 * 消费者02
 */
public class Consumer02 {
    
    
    //死信交换机名称
    private static final String deadQueue = "dead-queue";

    public static void main(String[] argv) throws Exception {
    
    
        //获取信道
        Channel channel = RabbitMqUtils.getChannel();
        System.out.println("等待接收死信队列消息.....");

        //消费者接收消息的回调
        DeliverCallback deliverCallback = (consumerTag, delivery) -> {
    
    
            String message = new String(delivery.getBody(), "UTF-8");
            System.out.println("Consumer02 接收死信队列的消息" + message);
        };
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(deadQueue, true, deliverCallback, consumerTag -> {
    
    
        });
    }
}

Start the test: Start consumer 01 first and then shut it down, then start consumer 02, and then start the producer.

6.3.3. Queue reaches maximum length

1. Remove the TTL attribute from the message producer code

package com.atguigu.rabbitmq.eight;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.BuiltinExchangeType;
import com.rabbitmq.client.Channel;

/**
 * 死信队列之 生产者代码
 */
public class Producer {
    
    

    //交换机的名称
    private static final String NORMAL_EXCHANGE = "normal_exchange";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        try (Channel channel = RabbitMqUtils.getChannel()) {
    
    
            /**
             * 声明一个 exchange（交换机）
             * 1.exchange 的名称
             * 2.exchange 的类型
             */
            channel.exchangeDeclare(NORMAL_EXCHANGE, BuiltinExchangeType.DIRECT);

            //设置消息的 TTL（超时）时间  单位是ms毫秒  10000ms=10s
            //AMQP.BasicProperties properties = new AMQP.BasicProperties().builder().expiration("10000").build();

            //该信息是用作演示队列个数限制
            for (int i = 1; i <11 ; i++) {
    
    
                /**
                 *发送一个消息
                 *1.发送到那个交换机       如果是入门级程序，没有考虑交换机，可以写空串，表示使用默认的交换机
                 *2.路由的 key 是哪个     使用默认的交换机写的是：队列的名称   指定了交换机写的是：routingKey关键词
                 *3.其他的参数信息        本次没有
                 *4.发送消息的消息体      不能直接发消息，需要调用它的二进制。
                 */
                String message="info"+i;
                channel.basicPublish(NORMAL_EXCHANGE, "zhangsan", null, message.getBytes());
                System.out.println("生产者发送消息:"+message);
            }
        }
    }
}

2.C1 consumer modifies the following code ( 启动之后关闭该消费者 模拟其接收不到消息)

        //设置正常队列的长度限制
        arguments.put("x-max-length", 6);

• Note : During the expiration time TTL test, consumer 1 has been started to generate a queue. At this time, if the parameters are modified and started again, an error will be reported.
• Solution : Delete the original queue and start again to regenerate the queue.

3.C2 consumer code remains unchanged (start C2 consumer)

6.3.4. Message rejected

1. The code of the message producer is the same as that of the producer above.

2.C1 consumer code ( 启动之后关闭该消费者 模拟其接收不到消息)

package com.atguigu.rabbitmq.eight;

import com.atguigu.rabbitmq.utils.RabbitMqUtils;
import com.rabbitmq.client.BuiltinExchangeType;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;

import java.util.HashMap;
import java.util.Map;

/**
 * 死信队列  实战
 *
 * 消费者01
 */
public class Consumer01 {
    
    
    //普通交换机名称
    private static final String NORMAL_EXCHANGE = "normal_exchange";
    //死信交换机名称
    private static final String DEAD_EXCHANGE = "dead_exchange";

    public static void main(String[] argv) throws Exception {
    
    
        //创建信道
        Channel channel = RabbitMqUtils.getChannel();
        //声明普通和死信交换机： 名称，类型(direct:路由模式)
        channel.exchangeDeclare(NORMAL_EXCHANGE, BuiltinExchangeType.DIRECT);
        channel.exchangeDeclare(DEAD_EXCHANGE, BuiltinExchangeType.DIRECT);

        /**
         * 声明普通队列,参数解释：
         *  1.queue：队列名称
         *  2.durable：是否持久化，当MQ重启后，还在 （true：持久化，保存在磁盘    false：不持久化，保存在内存）
         *  3.exclusive： false不独占，可以多个消费者消费。true只能供一个消费者进行消费
         *      功能1：是否独占。只能有一个消费者监听这个队列
         *      功能2：当Connection关闭时，是否删除队列
         *  4.autoDelete：是否自动删除 最后一个消费者端开连接以后 该队列是否自动删除 true自动删除 false不自动删除
         *  5.arguments：其它参数，在高级特性中讲，暂时设置为null
         *              设置消息为死信后转发到死信交换机，参数类型为Map集合
         */
        String normalQueue = "normal-queue";
        //设置消息为死信后转发到死信交换机，参数类型为Map集合
        Map<String, Object> arguments = new HashMap<>();
        //过期时间 10s=10000ms  在这个地方设置过期时间直接写死了，可以在生产者发送消息时设置过期时间，这样更灵活。
        //arguments.put("x-message-ttl",10000);
        //正常队列设置死信交换机是谁：参数key 是固定值，参数v是死信交换机的名字
        arguments.put("x-dead-letter-exchange", DEAD_EXCHANGE);
        //设置死信 routing-key: 参数 key 是固定值,参数v关键词的名字跟图中保持一致
        arguments.put("x-dead-letter-routing-key", "lisi");
        //设置正常队列的长度限制
        //arguments.put("x-max-length", 6);
        channel.queueDeclare(normalQueue, false, false, false, arguments);
        /**
         * 绑定普通交换机与普通队列：
         *  参数1：队列
         *  参数2：交换机
         *  参数3：关键词routingKey为
         */
        channel.queueBind(normalQueue, NORMAL_EXCHANGE, "zhangsan");

        //声明死信队列
        String deadQueue = "dead-queue";
        channel.queueDeclare(deadQueue, false, false, false, null);
        //绑定死信交换机与死信队列
        channel.queueBind(deadQueue, DEAD_EXCHANGE, "lisi");
        System.out.println("等待接收消息.....");

        //消费者接收消息的回调
        DeliverCallback deliverCallback = (consumerTag, delivery) -> {
    
    
            String message = new String(delivery.getBody(), "UTF-8");
            if(message.equals("info5")){
    
    
                System.out.println("Consumer01 接收到消息" + message + "并拒绝签收该消息");
                /*
                * 拒绝应答：
                * 1.消息标记 tag：每一个消息都有一个唯一的标识，表示应答的是哪一个消息。
                * 2.是否重新放入队列， false:不放回，消息变为死信，该队列如果配置了死信交换机将发送到死信队列中
                *                  true:消息放回队列
                * */
                channel.basicReject(delivery.getEnvelope().getDeliveryTag(), false);
            }else {
    
    
                /**
                 * 确认手动应答：
                 *  1.消息标记 tag：每一个消息都有一个唯一的标识，表示应答的是哪一个消息。
                 *  2.是否批量应答未应答消息 false:不批量应答信道中的消息 true:批量
                 */
                System.out.println("Consumer01 接收到消息"+message);
                channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
            }
        };
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(normalQueue, false, deliverCallback, consumerTag -> {
    
    
        });
    }
}

3.C2 consumer code remains unchanged
启动消费者 1 然后再启动消费者 2

7. Delay queue

7.1.Delay queue concept

• The delay queue is the TTL expiration of the message in the dead letter queue.

Delay queue, the queue is internally ordered , and its most important feature is reflected in its delay attribute . The elements in the delay queue are expected to be taken out and processed after or before the specified time. Simply put, delay A queue is a queue used to store elements that need to be processed at a specified time .

7.2. Delay queue usage scenarios

1. Orders not paid within ten minutes will be automatically canceled
   • After placing an order, a message will be sent to the delay queue. The expiration time of this message is 10 minutes. After ten minutes, the message will come out of the delay queue and cancel the unpaid order.
2. If a newly created store has not uploaded products within ten days, a message reminder will be automatically sent.
3. After the user successfully registers, an SMS reminder will be sent if the user does not log in within three days.
4. The user initiates a refund, and if it is not processed within three days, the relevant operations personnel will be notified.
5. After booking a meeting, each participant needs to be notified ten minutes before the scheduled time to attend the meeting.

These scenarios all have a characteristic that requires completing a certain task at a specified time point after an event occurs or before , such as: When an order generation event occurs, check the payment status of the order after ten minutes, and then process the unpaid order. Close; it seems that a scheduled task is used to poll the data all the time, check it once every second, take out the data that needs to be processed, and then process it, isn't it done? If the amount of data is relatively small , this can indeed be done. For example, for a requirement such as "automatic settlement if the bill is not paid within a week", if the time is not strictly limited, but a week in a loose sense , then run a A scheduled task to check all unpaid bills is indeed a feasible solution. However, for scenarios with a relatively large amount of data and strong timeliness , such as: "The order will be closed if it is not paid within ten minutes." There may be a lot of unpaid order data in the short term, and even reach millions or even tens of millions during the event. Level, it is obviously not advisable to still use the polling method for such a huge amount of data. It is likely that all orders cannot be completed within one second. At the same time, it will put a lot of pressure on the database, which cannot meet business requirements and has low performance. .

• Summarize:
  • For small amounts of data and broad timeliness - you can use a timer
  • For large amounts of data and strong timeliness, delay queues can be used.

7.3. TTL in RabbitMQ

What is TTL? TTL is an attribute of a message or queue in RabbitMQ, indicating the maximum survival time of a message or all messages in the queue.

The unit is milliseconds. In other words, if a message has a TTL attribute set or enters a queue with a TTL attribute set, then if the message is not consumed within the time set by the TTL, it will become a "dead letter". If both the queue's TTL and the message's TTL are configured, the smaller value will be used. There are two ways to set the TTL.

7.3.1. Queue setting TTL

The first is to set the "x-message-ttl" attribute of the queue when creating the queue.

7.3.2. Message setting TTL

Another way is to set the TTL for each message

7.3.3. The difference between the two

If the TTL attribute of the queue is set, then once the message expires, it will be discarded by the queue (if the dead letter queue is configured, it will be thrown into the dead letter queue). In the second method, even if the message expires, it may not be immediately discarded. Discard , because whether the message has expired is determined before it is delivered to the consumer . If the current queue has a serious message backlog, the expired message may still survive for a long time; in addition, one thing to note is that if Not setting TTL means that the message will never expire. If TTL is set to 0, it means that the message will be discarded unless it can be directly delivered to the consumer at this time.

We introduced the dead letter queue in the previous section, and just introduced TTL. At this point, the two major elements of using RabbitMQ to implement delay queues have been gathered. Next, we only need to fuse them, add a little bit of seasoning, and delay queues. It will be freshly baked. Think about it, delay queue, doesn't it mean how long the message is delayed to be processed? TTL is just how long the message will be delayed to become a dead letter. On the other hand, the message that becomes a dead letter will be delivered to the dead letter queue. , so that the consumer only needs to keep consuming the messages in the dead letter queue, because the messages inside are all messages that are expected to be processed immediately.

7.4. Integrate springboot

7.4.1. Create project

7.4.2. Add dependencies

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <!--RabbitMQ 依赖-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-amqp</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!--json转换包-->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.47</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>
        <!--swagger：界面测试包-->
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger2</artifactId>
            <version>2.9.2</version>
        </dependency>
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger-ui</artifactId>
            <version>2.9.2</version>
        </dependency>
        <!--RabbitMQ 测试依赖-->
        <dependency>
            <groupId>org.springframework.amqp</groupId>
            <artifactId>spring-rabbit-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

7.4.3. Modify configuration file

#配置连接rabbitmq的信息
spring.rabbitmq.host=192.168.10.120
#注意15672是访问管理界面的端口号
spring.rabbitmq.port=5672
spring.rabbitmq.username=admin
spring.rabbitmq.password=123456

7.4.4. Add Swagger configuration class

• Used to access the interface.

package com.cn.config;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import springfox.documentation.builders.ApiInfoBuilder;
import springfox.documentation.service.ApiInfo;
import springfox.documentation.service.Contact;
import springfox.documentation.spi.DocumentationType;
import springfox.documentation.spring.web.plugins.Docket;
import springfox.documentation.swagger2.annotations.EnableSwagger2;

@Configuration
@EnableSwagger2
public class SwaggerConfig {
    
    
    @Bean
    public Docket webApiConfig(){
    
    
        return new Docket(DocumentationType.SWAGGER_2)
                .groupName("webApi")
                .apiInfo(webApiInfo())
                .select()
                .build();
    }
    private ApiInfo webApiInfo(){
    
    
        return new ApiInfoBuilder()
                .title("rabbitmq 接口文档")
                .description("本文档描述了 rabbitmq 微服务接口定义")
                .version("1.0")
                .contact(new Contact("enjoy6288", "http://atguigu.com",
                        "[email protected]"))
                .build();
    }
}

7.5. Queue TTL

7.5.1. Code architecture diagram

Create two queues QA and QB, set the TTL of the two queues to 10S and 40S respectively, then create a switch X and a dead letter switch Y, their types are both direct, create a dead letter queue QD, their binding relationship as follows:

7.5.2. Configuration file class code

• illustrate:
  • When the Springboot project was not integrated before, all declarations including ordinary switches, delayed switches, ordinary queues, and delayed queues were declared in the consumer.
  • After integrating the Springboot project, these statements are unified into a separate class, that is, the configuration file class.

package com.cn.config;

import org.springframework.amqp.core.*;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.util.HashMap;
import java.util.Map;

/**
 * TTL队列 配置文件类代码
 */

@Configuration
public class TtlQueueConfig {
    
    

    //普通交换机名称
    public static final String X_EXCHANGE = "X";
    //死信交换机名称
    public static final String Y_DEAD_LETTER_EXCHANGE = "Y";


    //2个普通队列名称
    public static final String QUEUE_A = "QA";
    public static final String QUEUE_B = "QB";
    //1个死信队列名称
    public static final String DEAD_LETTER_QUEUE = "QD";


    // 声明 xExchange（普通交换机）
    @Bean("xExchange")
    public DirectExchange xExchange(){
    
    
        return new DirectExchange(X_EXCHANGE);
    }
    // 声明 yExchange（死信交换机）
    @Bean("yExchange")
    public DirectExchange yExchange(){
    
    
        return new DirectExchange(Y_DEAD_LETTER_EXCHANGE);
    }


    //声明普通队列A  ttl过期时间为10s 并绑定到对应的死信交换机
    @Bean("queueA")
    public Queue queueA(){
    
    
        //指定map的长度，可以节省创建时间。
        Map<String, Object> args = new HashMap<>(3);
        //声明当前队列绑定的死信交换机
        args.put("x-dead-letter-exchange", Y_DEAD_LETTER_EXCHANGE);
        //声明当前队列的死信路由 key
        args.put("x-dead-letter-routing-key", "YD");
        //声明队列的 TTL 单位是毫秒
        args.put("x-message-ttl", 10000);

        //durable:持久化的队列名  withArguments：设置参数
        return QueueBuilder.durable(QUEUE_A).withArguments(args).build();
    }
    //声明普通队列B  ttl过期时间为40s 并绑定到对应的死信交换机
    @Bean("queueB")
    public Queue queueB(){
    
    
        Map<String, Object> args = new HashMap<>(3);
        //声明当前队列绑定的死信交换机
        args.put("x-dead-letter-exchange", Y_DEAD_LETTER_EXCHANGE);
        //声明当前队列的死信路由 key
        args.put("x-dead-letter-routing-key", "YD");
        //声明队列的 TTL
        args.put("x-message-ttl", 40000);

        return QueueBuilder.durable(QUEUE_B).withArguments(args).build();
    }
    //声明死信队列 QD
    @Bean("queueD")
    public Queue queueD(){
    
    
        return new Queue(DEAD_LETTER_QUEUE);
    }


    // 声明队列 A 绑定 X 交换机
    @Bean
    public Binding queueaBindingX(@Qualifier("queueA") Queue queueA,
                                  @Qualifier("xExchange") DirectExchange xExchange){
    
    //根据属性名注入参数
        //  队列  交换机  关键词
        return BindingBuilder.bind(queueA).to(xExchange).with("XA");
    }
    //声明队列 B 绑定 X 交换机
    @Bean
    public Binding queuebBindingX(@Qualifier("queueB") Queue queue1B,
                                  @Qualifier("xExchange") DirectExchange xExchange){
    
    
        return BindingBuilder.bind(queue1B).to(xExchange).with("XB");
    }
    //声明死信队列 QD 绑定关系
    @Bean
    public Binding deadLetterBindingQad(@Qualifier("queueD") Queue queueD,
                                        @Qualifier("yExchange") DirectExchange yExchange){
    
    
        return BindingBuilder.bind(queueD).to(yExchange).with("YD");
    }

}

7.5.3. Message producer code

package com.cn.controller;

import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.Date;

/**
 * 发送延迟消息  生产者
 *
 * http://localhost:8080/ttl/sendMsg/嘻嘻嘻
 */

@Slf4j
@RequestMapping("/ttl")
@RestController
public class SendMsgController {
    
    

    //此对象用来发送消息：Springboot版本过高注入会报错，可以降低版本，在pom文件中修改springboot版本(2.6.0以下)，之后重新加载jar包
    @Autowired
    private RabbitTemplate rabbitTemplate;

    @GetMapping("sendMsg/{message}")
    public void sendMsg(@PathVariable String message){
    
    
        //{}为日志的占位符，后面的参数会替换掉占位符的位置
        log.info("当前时间：{},发送一条信息给两个TTL队列:{}", new Date(), message);
        /*
        * 发送消息：
        *    交换机
        *    routingKey
        *    发送的消息
        * */
        rabbitTemplate.convertAndSend("X", "XA", "消息来自 ttl 为 10S 的队列: "+message);
        rabbitTemplate.convertAndSend("X", "XB", "消息来自 ttl 为 40S 的队列: "+message);
    }
}

Lower the Springboot version:

7.5.4. Message consumer code

package com.cn.consumer;

import com.rabbitmq.client.Channel;
import lombok.extern.slf4j.Slf4j;

import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;

import java.io.IOException;
import java.util.Date;

/**
 * 队列TTL  消费者
 */

@Slf4j
@Component //为了消费者能够实例化
public class DeadLetterQueueConsumer {
    
    

    @RabbitListener(queues = "QD")//接收消息：队列名称
    public void receiveD(Message message, Channel channel) throws IOException {
    
    
        //将消息体转化为字符串
        String msg = new String(message.getBody());
        log.info("当前时间：{},收到死信队列信息{}", new Date().toString(), msg);
    }
}

Make a requesthttp://localhost:8080/ttl/sendMsg/嘻嘻嘻

The first message becomes a dead letter message after 10S and is then consumed by the consumer. The second message becomes a dead letter message after 40S and is then consumed. In this way, a delay queue is completed.

However, if used in this way, wouldn't it mean that every time a new time requirement is added, a new queue must be added . There are only two time options of 10S and 40S. If it needs to be processed after one hour, then the TTL needs to be increased to one hour. Queue, if it is a scenario like booking a conference room and notifying it in advance, wouldn’t it be necessary to add countless queues to meet the demand?

• Summary: Now we write a queue with a delay of 10 seconds, and write a queue with a delay of 40 seconds. In the future, do we have to add a queue every time we increase the delay time? ? ? ?
• Answer: You should optimize and write a queue that meets all requirements.

7.6. Delay queue optimization

7.6.1. Code architecture diagram

A new queue QC is added here. The binding relationship is as follows. The queue does not set a TTL time, and the delay time is determined by the producer when sending a message.

7.6.2. Configuration file class code

Add a new queue in the configuration file class:

• Define a new queue name
  
• Create this new queue
  
• Bind this newly created queue to the x switch.
  

    //普通队列名称
    public static final String QUEUE_C = "QC";


    //声明普通队列C  ttl没有设置过期时间 并绑定到对应的死信交换机
    @Bean("queueC")
    public Queue queueC(){
    
    
        Map<String, Object> args = new HashMap<>(3);
        //声明当前队列绑定的死信交换机
        args.put("x-dead-letter-exchange", Y_DEAD_LETTER_EXCHANGE);
        //声明当前队列的死信路由 key
        args.put("x-dead-letter-routing-key", "YD");
        //没有声明 TTL 属性
        return QueueBuilder.durable(QUEUE_C).withArguments(args).build();
    }


    //声明队列 C 绑定 X 交换机
    @Bean
    public Binding queuecBindingX(@Qualifier("queueC") Queue queueC,
                                  @Qualifier("xExchange") DirectExchange xExchange){
    
    
        return BindingBuilder.bind(queueC).to(xExchange).with("XC");
    }

7.6.3. Message producer code

• Write a new method that carries a timeout when sending a request.

    //开始发消息  消息 TTL
    @GetMapping("sendExpirationMsg/{message}/{ttlTime}")
    public void sendMsg(@PathVariable String message,@PathVariable String ttlTime) {
    
    
        log.info("当前时间：{},发送一条时长{}毫秒 TTL 信息给队列 C:{}", new Date(),ttlTime, message);
        /*
         * 发送消息：
         *    交换机
         *    routingKey
         *    发送的消息
         *    设置过期时间：类型为接口类型
         * */
        rabbitTemplate.convertAndSend("X", "XC", message, correlationData ->{
    
    
            //发送消息的时候 延迟时长
            correlationData.getMessageProperties().setExpiration(ttlTime);
            return correlationData;
        });

    }

Make a request

• http://localhost:8080/ttl/sendExpirationMsg/你好 1/20000
• http://localhost:8080/ttl/sendExpirationMsg/你好 2/2000

There seems to be no problem, but at the beginning, it was introduced that if you use the method of setting TTL on the message properties (setting TTL for each message), the message may not "die" on time, because RabbitMQ will only check the first Whether a message has expired , if it has expired, it will be thrown into the dead letter queue. If the delay time of the first message is very long, and the delay time of the second message is very short, the second message will not be executed first.

• Phenomenon : The first message is set with a 20s delay, and the second message is set with a 2s delay. After executing the request, it was found that the first piece of data was indeed executed with a delay of close to 20 seconds, but the second piece of data was clearly set to be executed with a delay of 2 seconds, but it was actually executed after 14 seconds. It means that the second piece of data waits for the first piece of data to be executed before executing it immediately.

• Defects of delay queue optimization : The optimization of delay queue can already satisfy the requirement that one queue can be used at any time, but when more than two messages are sent, there will be a sequence, and the second message can only be sent after the first message is sent.

• Solution : Use delay plugin

7.7.Rabbitmq plug-in implements delay queue

The problem mentioned above is indeed a problem. If TTL cannot be implemented at message granularity and it dies in time at the set TTL time, it cannot be designed as a general delay queue. So how to solve it? Next we will solve this problem.

7.7.1. Install delay queue plug-in

On the official https://www.rabbitmq.com/community-plugins.htmlwebsite , download the rabbitmq_delayed_message_exchange plug-in , then unzip and place it in the plugins directory of RabbitMQ.

• Enter the official website
  
  
  
• Use MobaXterm to put the downloaded compressed package into the plugins directory under the RabbitMQ installation directory.

  • First: enter the plug-in directory ( cd /usr/lib/rabbitmq/lib/rabbitmq_server-3.8.8/plugins/)
    

  • Secondly: Upload the plug-in to this directory: (The 3.8.0.ez version is used here. Note that the plug-in version is lower than the installed Rabbitmq version)
    

Enter the plugins directory under the RabbitMQ installation directory, execute the following command to make the plug-in take effect, and then restart RabbitMQ

• Enter the plugin directory:cd /usr/lib/rabbitmq/lib/rabbitmq_server-3.8.8/plugins/
• Installation (no version information required):rabbitmq-plugins enable rabbitmq_delayed_message_exchange
  
• Restart RabbitMQ:systemctl restart rabbitmq-server
• Effect
• The original position of the delayed message is at the queue, but after using the plug-in, it is delayed at the switch.
  

7.7.2. Code architecture diagram

A new queue delayed.queue and a custom switch delayed.exchange are added here. The binding relationship is as follows:

7.7.3.Configuration file class code

In our custom switch, this is a new exchange type. This type of message supports a delayed delivery mechanism. After the message is delivered, it will not be delivered to the target queue immediately, but will be stored in the mnesia (a distributed data system) table. When the delivery time is reached, it will be delivered to the target queue.

package com.cn.config;

import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.CustomExchange;
import org.springframework.amqp.core.Queue;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.util.HashMap;
import java.util.Map;

/**
 *
 */
@Configuration
public class DelayedQueueConfig {
    
    
    //交换机
    public static final String DELAYED_EXCHANGE_NAME = "delayed.exchange";
    //队列
    public static final String DELAYED_QUEUE_NAME = "delayed.queue";
    //routingkey
    public static final String DELAYED_ROUTING_KEY = "delayed.routingkey";


    /**
     * 声明交换机：
     * 因为系统没有提供插件使用的交换机类型x-delayed-message，
     * 所以只能使用自定义交换机来定义一个延迟交换机
     */
    @Bean
    public CustomExchange delayedExchange() {
    
    

        Map<String, Object> args = new HashMap<>();
        args.put("x-delayed-type", "direct");//延迟类型

        // 交换机的名称  类型  是否持久化  是否自动删除  其它的参数
        return new CustomExchange(DELAYED_EXCHANGE_NAME, "x-delayed-message", true, false, args);
    }

    //声明队列
    @Bean
    public Queue delayedQueue() {
    
    
        return new Queue(DELAYED_QUEUE_NAME);
    }

    //交换机与队列进行绑定
    @Bean
    public Binding bindingDelayedQueue(@Qualifier("delayedQueue") Queue queue,
                                       @Qualifier("delayedExchange") CustomExchange delayedExchange) {
    
    //根据属性名注入参数

        //队列 交换机 关键词 构建
        return BindingBuilder.bind(queue).to(delayedExchange).with(DELAYED_ROUTING_KEY).noargs();
    }
}

7.7.4. Message producer code

    //开始发消息  基于插件的  消息 及 延迟的时间
    @GetMapping("sendDelayMsg/{message}/{delayTime}")
    public void sendMsg(@PathVariable String message,@PathVariable Integer delayTime) {
    
    
        log.info(" 当前时间：{},发送一条延迟{}毫秒的信息给队列delayed.queue:{}", new Date(),delayTime, message);
        /*
         * 发送消息：
         *    交换机
         *    routingKey
         *    发送的消息
         *    设置过期时间：类型为接口类型
         * */
        rabbitTemplate.convertAndSend(DelayedQueueConfig.DELAYED_EXCHANGE_NAME, DelayedQueueConfig.DELAYED_ROUTING_KEY, message,
                correlationData ->{
    
    
                    //发送消息的时候 延迟时长 单位：ms毫秒
                    correlationData.getMessageProperties().setDelay(delayTime);
                    return correlationData;
                });

    }

7.7.5. Message consumer code

package com.cn.consumer;

import com.cn.config.DelayedQueueConfig;
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;

import java.util.Date;

/**
 * 消费者   基于插件的延迟消息
 */
@Slf4j
@Component
public class DelayQueueConsumer {
    
    

    //监听消息
    @RabbitListener(queues = DelayedQueueConfig.DELAYED_QUEUE_NAME) //队列名称
    public void receiveDelayedQueue(Message message){
    
    
        //将消息体转化为字符串
        String msg = new String(message.getBody());
        log.info("当前时间：{},收到延时队列的消息：{}", new Date().toString(), msg);
    }
}

Make a request:

• http://localhost:8080/ttl/sendDelayMsg/come on baby1/20000
• http://localhost:8080/ttl/sendDelayMsg/come on baby2/2000

The second message was consumed first, in line with expectations.

7.8. Summary

• Delayed messages: 2 ways
  • based on dead letter
  • Plug-in based

Delay queues are very useful in scenarios that require delayed processing. Using RabbitMQ to implement delay queues can make good use of RabbitMQ's features, such as: reliable message sending, reliable message delivery, and dead letter queues to ensure that messages are consumed at least once. And messages that are not processed correctly are not discarded. In addition, through the characteristics of the RabbitMQ cluster, the problem of single point of failure can be well solved, and the delay queue will not be unavailable or messages will be lost due to the failure of a single node.

Of course, there are many other options for delay queues, such as using Java's DelayQueue, using Redis's zset, using Quartz or using kafka's time wheel. Each of these methods has its own characteristics, depending on the applicable scenario.

8. Release Confirmation Advanced

In the production environment, due to some unknown reasons, rabbitmq restarts . During the restart of RabbitMQ, the producer message delivery fails, resulting in message loss, which requires manual processing and recovery. So, we started thinking, how can we achieve reliable message delivery in RabbitMQ? Especially in such extreme situations, when the RabbitMQ cluster is unavailable, how to deal with undeliverable messages:

• Exception examples:

应 用 [xxx] 在 [08-1516:36:04] 发 生 [ 错误日志异常 ] ， alertId=[xxx] 。 由
[org.springframework.amqp.rabbit.listener.BlockingQueueConsumer:start:620] 触发。 
应用 xxx 可能原因如下
服务名为： 
异常为： org.springframework.amqp.rabbit.listener.BlockingQueueConsumer:start:620, 
产 生 原 因 如 下 :1.org.springframework.amqp.rabbit.listener.QueuesNotAvailableException: 
Cannot prepare queue for listener. Either the queue doesn't exist or the broker will not 
allow us to use it.||Consumer received fatal=false exception on startup:

8.1. Release and confirm springboot version

8.1.1. Confirmation mechanism plan

• As long as one of the switches and queues does not exist, the message will be lost.
  

8.1.2. Code architecture diagram

8.1.3.Configuration file

Need to be added to the configuration file

• spring.rabbitmq.publisher-confirm-type=correlated
  • NONE
    disables release confirmation mode, which is the default value
  • CORRELATED (recommended)
    The callback method will be triggered after the message is successfully published to the exchange.
  • SIMPLE (equivalent to the single synchronization confirmation learned before, slower)
    has been tested to have two effects:
    • One of the effects will trigger the callback method just like the CORRELATED value.
    • Second, after successfully publishing the message, use rabbitTemplate to call the waitForConfirms or waitForConfirmsOrDie method to wait for the broker node to return the sending result, and determine the next step logic based on the return result. The point to note is that if the waitForConfirmsOrDie method returns false, the channel will be closed, and the next step will not be possible . Send message to broker

#配置连接rabbitmq的信息
spring.rabbitmq.host=192.168.10.120
#注意15672是访问管理界面的端口号
spring.rabbitmq.port=5672
spring.rabbitmq.username=admin
spring.rabbitmq.password=123456
spring.rabbitmq.publisher-confirm-type=correlated

8.1.4. Add configuration class

package com.cn.config;

import org.springframework.amqp.core.*;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * 配置类   发布确认（高级）
 */

@Configuration
public class ConfirmConfig {
    
    

    //交换机
    public static final String CONFIRM_EXCHANGE_NAME = "confirm.exchange";
    //队列
    public static final String CONFIRM_QUEUE_NAME = "confirm.queue";
    //routingkey
    public static final String CONFIRM_ROUTING_KEY = "key1";


    //声明交换机
    @Bean("confirmExchange")
    public DirectExchange confirmExchange(){
    
    
        return new DirectExchange(CONFIRM_EXCHANGE_NAME);
    }

    // 声明确认队列
    @Bean("confirmQueue")
    public Queue confirmQueue(){
    
    

        //return new Queue(CONFIRM_QUEUE_NAME); //方式一
        return QueueBuilder.durable(CONFIRM_QUEUE_NAME).build(); //方式二
    }

    // 声明确认队列绑定关系
    @Bean
    public Binding queueBinding(@Qualifier("confirmQueue") Queue queue,
                                @Qualifier("confirmExchange") DirectExchange exchange){
    
    //根据属性名注入参数
        //  队列  交换机  关键词
        return BindingBuilder.bind(queue).to(exchange).with("key1");
    }
}

8.1.5. Message producer

package com.cn.controller;

import com.cn.config.ConfirmConfig;
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.rabbit.connection.CorrelationData;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.PostConstruct;

/**
 * 开始发消息  发布确认（高级）
 * http://localhost:8080/confirm/sendMessage/大家好
 */

@RestController
@RequestMapping("/confirm")
@Slf4j
public class ProducerController {
    
    

    //此对象用来发送消息：Springboot版本过高注入会报错，可以降低版本，在pom文件中修改springboot版本(2.6.0以下)，之后重新加载jar包
    @Autowired
    private RabbitTemplate rabbitTemplate;

    //发送消息
    @GetMapping("sendMessage/{message}")
    public void sendMessage(@PathVariable String message){
    
    

        /*
         * 正常情况下发送消息：交换机正常+队列正常
         *    交换机
         *    routingKey
         *    发送的消息
         *    填写回调消息的ID及相关信息（有不同重载的构造方法）
         * */
        //指定消息 id 为 1
        CorrelationData correlationData1 = new CorrelationData("1");
        rabbitTemplate.convertAndSend(ConfirmConfig.CONFIRM_EXCHANGE_NAME,ConfirmConfig.CONFIRM_ROUTING_KEY,
                message, correlationData1);
        log.info("发送消息内容:{}",message);

        //指定消息 id 为 2
        CorrelationData correlationData2 = new CorrelationData("2");
        //异常情况下发送消息  交换机错误（写错交换机名字）+队列正常
        rabbitTemplate.convertAndSend(ConfirmConfig.CONFIRM_EXCHANGE_NAME+"123",ConfirmConfig.CONFIRM_ROUTING_KEY,
                message, correlationData2);
        log.info("发送消息内容:{}",message);

        //指定消息 id 为 3
        CorrelationData correlationData3 = new CorrelationData("3");
        //异常情况下发送消息  交换机正常+队列错误（交换机绑定队列需要正确的关键词，写错routingKey，这样队列就接收不到消息了）
        rabbitTemplate.convertAndSend(ConfirmConfig.CONFIRM_EXCHANGE_NAME,ConfirmConfig.CONFIRM_ROUTING_KEY+"2",
                message, correlationData3);
        log.info("发送消息内容:{}",message);
    }
}

8.1.6.Callback interface

package com.cn.config;

import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.rabbit.connection.CorrelationData;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;


/**
 * 回调接口
 */
@Component
@Slf4j
public class MyCallBack implements RabbitTemplate.ConfirmCallback {
    
    //实现的是RabbitTemplate的一个内部接口


    /**
     * 问题：当前类实现的的是RabbitTemplate的一个内部接口，这样会导致当前
     *    类MyCallBack不在这个RabbitTemplate接口对象中，所以导致RabbitTemplate
     *    在调用自身接口ConfirmCallback时，根本调不到这个实现类MyCallBack。
     *
     * 解决：把这个实现类在注入到RabbitTemplate接口中
     */
    @Autowired
    private RabbitTemplate rabbitTemplate;

    /**
     * 此方法并没有被调用，所以加上@PostConstruct注解，作用是：
     * @PostConstruct是Java自带的注解，在方法上加该注解会在项目启动的时候执行该方法，
     * 也可以理解为在spring容器初始化的时候执行该方法。
     */
    @PostConstruct
    public void init(){
    
    
        //注入
        rabbitTemplate.setConfirmCallback(this);
    }
    /**
        * 交换机确认回调方法
        * 1.发消息  交换机接收到了  回调
        *  1.1 correlationData 保存回调消息的ID及相关信息   它是来自于生成者发送消息时自己填写的
        *  1.2 交换机是否收到消息  ack = true  表示收到消息
        *  1.3 失败的原因：如果发送成功，则此参数为null
        *
        * 2.发消息  交换机接收失败了  回调
        *  2.1 correlationData 保存回调消息的ID及相关信息
        *  2.2 交交换机是否收到消息  ack = false 表示没有收到消息
        *  2.3 cause 失败的原因
        */
    @Override
    public void confirm(CorrelationData correlationData, boolean ack, String cause) {
    
    

        //获取id
        String id=correlationData!=null?correlationData.getId():"";
        if(ack){
    
    
            log.info("交换机已经收到 id 为:{}的消息",id);
        }else{
    
    
            log.info("交换机还未收到 id 为:{}消息,由于原因:{}",id,cause);
        }
    }
}

8.1.7. Message consumer

package com.cn.consumer;

import com.cn.config.ConfirmConfig;
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;

/**
 * 接收消息  发布确认（高级）
 */

@Component
@Slf4j
public class ConfirmConsumer {
    
    

    //监听消息
    @RabbitListener(queues = ConfirmConfig.CONFIRM_QUEUE_NAME) //队列名称
    public void receiveMsg(Message message){
    
    
        //将消息体转化为字符串
        String msg=new String(message.getBody());
        log.info("接受到队列 confirm.queue 消息:{}",msg);
    }
}

8.1.8. Result analysis

• Send request :http://localhost:8080/confirm/sendMessage/大家好
  
  • Case 1: Sending messages under normal circumstances: switch is normal + queue is normal
    
    Effect : both switch and queue receive messages
    
  • Situation 2: Error in the switch when sending the message under abnormal circumstances (wrong switch name) + normal
    
    effect of the queue : the switch queue cannot receive the message
    
  • Scenario 3: When sending a message under abnormal circumstances, the switch is normal + queue error (the switch needs the correct keyword to bind the queue, and the routingKey is written incorrectly, so the queue cannot receive the message). Effect: The switch receives the message, but the queue cannot receive the
    
    message . (Because the RoutingKey of the message is inconsistent with the BindingKey of the queue, and no other queue can receive the message, the message is discarded directly .)
    

Question : An exception occurs at the switch. Through the callback interface, you can know where the exception occurred (the switch confirmed and responded). If an exception occurs at the queue, the message is lost directly, and there is no error message (the queue does not confirm or respond). So how to solve the problem of abnormal prompt information at the queue? ? ?

Solution: Use the Mandatory parameter to set the fallback message.

8.2. Rollback message

8.2.1.Mandatory parameters

When only the producer confirmation mechanism is enabled, after receiving the message, the switch will directly send a confirmation message to the message producer. If the message is found to be unroutable (the switch cannot send the message to the queue), the message will be discarded directly. At this time, the producer does not know that the message was discarded . So how can I help me find a way to handle messages that cannot be routed? At least let me know so I can handle it myself. By setting the mandatory parameter, the message can be returned to the producer when the destination cannot be reached during message delivery.

• Summary: If an exception occurs at the switch, the message can be rolled back through the callback interface. However, if an exception occurs in the queue, the message will be lost directly. If you want to roll back the message if an exception occurs at the queue, you can set the mandatory parameter to achieve this.

8.2.2. Add configuration in the configuration class

#发布退回消息：一旦消息发送不到队列，它会回退消息给生产者
spring.rabbitmq.publisher-returns=true

8.2.3. Message producer code

Same as above 8.1.5

8.2.4. Modify the callback interface

package com.cn.config;

import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.core.ReturnedMessage;
import org.springframework.amqp.rabbit.connection.CorrelationData;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;


/**
 * 回调接口
 */
@Component
@Slf4j
//实现的是RabbitTemplate的两个内部接口
//ConfirmCallback：交换机确认回调接口
//ReturnsCallback：消息发送失败，回退消息接口
public class MyCallBack implements RabbitTemplate.ConfirmCallback,RabbitTemplate.ReturnsCallback {
    
    

    /**
     * 问题：当前类实现的的是RabbitTemplate的一个内部接口，这样会导致当前
     *    类MyCallBack不在这个RabbitTemplate接口对象中，所以导致RabbitTemplate
     *    在调用自身接口ConfirmCallback时，根本调不到这个实现类MyCallBack。
     *
     * 解决：把这个实现类在注入到RabbitTemplate接口中
     */
    @Autowired
    private RabbitTemplate rabbitTemplate;

    /**
     * 此方法并没有被调用，所以加上@PostConstruct注解，作用是：
     * @PostConstruct是Java自带的注解，在方法上加该注解会在项目启动的时候执行该方法，
     * 也可以理解为在spring容器初始化的时候执行该方法。
     */
    @PostConstruct
    public void init(){
    
    
        //注入
        rabbitTemplate.setConfirmCallback(this);
        rabbitTemplate.setReturnsCallback(this);
    }
    /**
        * 交换机确认回调方法
        * 1.发消息  交换机接收到了  回调
        *  1.1 correlationData 保存回调消息的ID及相关信息   它是来自于生成者发送消息时自己填写的
        *  1.2 交换机是否收到消息  ack = true  表示收到消息
        *  1.3 失败的原因：如果发送成功，则此参数为null
        *
        * 2.发消息  交换机接收失败了  回调
        *  2.1 correlationData 保存回调消息的ID及相关信息
        *  2.2 交交换机是否收到消息  ack = false 表示没有收到消息
        *  2.3 cause 失败的原因
        */
    @Override
    public void confirm(CorrelationData correlationData, boolean ack, String cause) {
    
    

        //获取id
        String id=correlationData!=null?correlationData.getId():"";
        if(ack){
    
    
            log.info("交换机已经收到 id 为:{}的消息",id);
        }else{
    
    
            log.info("交换机还未收到 id 为:{}消息,由于原因:{}",id,cause);
        }
    }

    //可以在当消息传递过程中不可达目的地时 将消息返回给生产者
    // 只有不可达目的地的时候  才进行回退
    @Override
    public void returnedMessage(ReturnedMessage returnedMessage) {
    
    
        log.error(" 消息:{},被交换机:{}退回,退回原因:{},路由key:{}",
                new String(returnedMessage.getMessage().getBody()),returnedMessage.getExchange(),
                returnedMessage.getReplyText(),returnedMessage.getRoutingKey());
    }
}

8.2.5. Result analysis

• send request:http://localhost:8080/confirm/sendMessage/大家好
  
• Effect: Even if the message cannot be sent to the queue, rollback processing can still be performed to ensure that the message is not lost.
  

8.3.Backup switch

With the mandatory parameter and fallback message, we gain the ability to perceive undeliverable messages and have the opportunity to discover and handle when the producer's message cannot be delivered. But sometimes, we don't know how to handle these unroutable messages. At most, we can log them, trigger an alarm, and then handle them manually. It is very inelegant to handle these unroutable messages through logs, especially when the service where the producer is located has multiple machines. Manually copying logs will be more troublesome and error-prone. Moreover, setting the mandatory parameter will increase the complexity of the producer, and it is necessary to add logic to handle these returned messages. What should you do if you don’t want to lose messages but don’t want to increase the complexity of the producer? In the previous article on setting up a dead letter queue, we mentioned that a dead letter switch can be set up for the queue to store messages that failed to be processed. However, these non-routable messages have no chance to enter the queue, so the dead letter queue cannot be used to save messages. . In RabbitMQ, there is a backup switch mechanism that can deal with this problem very well. What is a backup switch? The backup switch can be understood as the "spare tire" of the switch in RabbitMQ. When we declare a corresponding backup switch for a certain switch, we create a spare tire for it. When the switch receives an unroutable message, it will Messages are forwarded to the backup switch, which forwards and processes them. Usually the type of backup switch is Fanout, so that all messages can be delivered to the queue bound to it, and then we bind a Queue, so that all messages that cannot be routed by the original switch will enter this queue. Of course, we can also create an alarm queue and use independent consumers to monitor and alarm.

Summarize:

• Before: If there was a problem with the switch and the message could not be received, the switch was required to confirm the release of the message and let the producer resend the message. If there is a problem with the queue, the message must also be rolled back and then resent.
• Now: It is also possible to use a backup switch. Using the backup switch, messages do not need to be rolled back to the producer. Once the message cannot be delivered to the switch, let the switch send it to the backup switch, and let the backup switch send it to the consumer through the routing queue. This can also achieve the purpose of not losing the message.
  • Benefits: Messages can be monitored and alarmed

8.3.1. Code architecture diagram

• The producer, confirmation switch, confirmation queue, binding relationship, and consumer have all been written. Now you only need to add the backup switch, 2 queues (backup queue, alarm queue), and 2 consumers (backup consumer, alarm consumer) , the alarm consumer does not need to write, because as long as the alarm consumer can receive it, the backup consumer can also receive it.)
  

8.3.2. Modify configuration class

package com.cn.config;

import org.springframework.amqp.core.*;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * 配置类   发布确认（高级）
 */

@Configuration
public class ConfirmConfig {
    
    

    //交换机
    public static final String CONFIRM_EXCHANGE_NAME = "confirm.exchange";
    //队列
    public static final String CONFIRM_QUEUE_NAME = "confirm.queue";
    //routingkey
    public static final String CONFIRM_ROUTING_KEY = "key1";
    //备份交换机
    public static final String BACKUP_EXCHANGE_NAME = "backup.exchange";
    //备份队列
    public static final String BACKUP_QUEUE_NAME = "backup.queue";
    //警告队列
    public static final String WARNING_QUEUE_NAME = "warning.queue";


    //声明确认交换机，以及确认交换机无法投递的消息将发送给备份交换机
    @Bean("confirmExchange")
    public DirectExchange confirmExchange(){
    
    


       //确认交换机  持久化  参数(备份交换机)   构建
        return ExchangeBuilder.directExchange(CONFIRM_EXCHANGE_NAME).durable(true)
                .withArgument("alternate-exchange",BACKUP_EXCHANGE_NAME).build();
    }

    // 声明确认队列
    @Bean("confirmQueue")
    public Queue confirmQueue(){
    
    

        //return new Queue(CONFIRM_QUEUE_NAME); //方式一
        return QueueBuilder.durable(CONFIRM_QUEUE_NAME).build(); //方式二
    }

    // 声明确认队列绑定关系
    @Bean
    public Binding queueBinding(@Qualifier("confirmQueue") Queue queue,
                                @Qualifier("confirmExchange") DirectExchange exchange){
    
    //根据属性名注入参数
        //  队列  交换机  关键词
        return BindingBuilder.bind(queue).to(exchange).with("key1");
    }

    //声明备份交换机
    @Bean("backupExchange")
    public FanoutExchange backupExchange(){
    
    
        return new FanoutExchange(BACKUP_EXCHANGE_NAME);
    }
    // 声明备份队列
    @Bean("backQueue")
    public Queue backQueue(){
    
    
        return QueueBuilder.durable(BACKUP_QUEUE_NAME).build();
    }
    // 声明警告队列
    @Bean("warningQueue")
    public Queue warningQueue(){
    
    
        return QueueBuilder.durable(WARNING_QUEUE_NAME).build();
    }


    // 声明备份队列绑定关系
    @Bean
    public Binding backupBinding(@Qualifier("backQueue") Queue queue,
                                 @Qualifier("backupExchange") FanoutExchange backupExchange){
    
    
        //发布订阅（扇出）类型就没必要写RoutingKey，它是以广播形式发送给所有队列。
        return BindingBuilder.bind(queue).to(backupExchange);
    }
    // 声明报警队列绑定关系
    @Bean
    public Binding warningBinding(@Qualifier("warningQueue") Queue queue,
                                  @Qualifier("backupExchange") FanoutExchange backupExchange){
    
    
        return BindingBuilder.bind(queue).to(backupExchange);
    }

}

8.3.3. Alarm consumers

package com.cn.consumer;

import com.cn.config.ConfirmConfig;
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;

/**
 * 报警消费者
 */
@Component
@Slf4j
public class WarningConsumer {
    
    

    //接收报警消息
    @RabbitListener(queues = ConfirmConfig.WARNING_QUEUE_NAME)
    public void receiveWarningMsg(Message message) {
    
    

        String msg = new String(message.getBody());
        log.error("报警发现不可路由消息：{}", msg);
    }
}

8.3.4. Testing precautions

When restarting the project, you need to delete the original confirm.exchange (confirm switch) because we have modified its binding properties, otherwise the following error will be reported:

8.3.5. Result analysis

Producer as above:

Test request:http://localhost:8080/confirm/sendMessage/大家好

Effect:

When the mandatory parameter and the backup switch can be used together, if both are enabled at the same time, where will the message go? Who has the higher priority? The above results show that the answer is that the backup switch has the highest priority.

9.RabbitMQ other knowledge points

9.1. Idempotence

9.1.1. Concept

The results of one request or multiple requests initiated by the user for the same operation are consistent, and there will be no side effects caused by multiple clicks . The simplest example is payment. The user pays after purchasing the goods. The payment is deducted successfully, but the network is abnormal when the result is returned. At this time, the money has been deducted. If the user clicks the button again, a second deduction will be performed. The payment was returned successfully. The user checked the balance and found that more money was deducted, and the transaction record also became two. In the previous single-application system, we only needed to put data operations into transactions and roll back immediately when an error occurred. However, network interruptions or exceptions may also occur when responding to the client.

9.1.2. Repeated consumption of messages

When the consumer consumes the message in MQ, MQ has sent the message to the consumer. The network was interrupted when the consumer returned the ack response to MQ. Therefore, MQ did not receive the confirmation information, and the message will be resent to other consumers. , or sent to the consumer again after the network is reconnected, but in fact the consumer has successfully consumed the message, causing the consumer to consume duplicate messages .

9.1.3.Solution ideas

The solution to the idempotence of MQ consumers is generally to use a global ID or write a unique identifier such as a timestamp or UUID or order consumers to consume messages in MQ. They can also use the ID of MQ to judge, or they can generate one according to their own rules. Globally unique id. Each time a message is consumed, this id is used to first determine whether the message has been consumed.

9.1.4. Idempotence guarantee on the consumer side

During the peak period of business when massive orders are generated, messages may be repeated on the production side. At this time, the consumer side must implement idempotence, which means that our messages will never be consumed multiple times, even if we receive Same news. There are two mainstream idempotent operations in the industry: a. Unique ID + fingerprint code mechanism, using the database primary key to eliminate duplication, and b. Using the atomicity of redis to achieve

9.1.5. Unique ID + fingerprint code mechanism

Fingerprint code : Some of our rules or timestamps plus unique information codes given by other services. It is not necessarily generated by our system. It is basically spliced ​​from our business rules, but the uniqueness must be guaranteed. Then use the query statement to determine whether the id exists in the database. The advantage is that it is simple to implement, just one splice, and then query to determine whether it is repeated; the disadvantage is that in high concurrency, if it is a single database, there will be a writing performance bottleneck. Of course, it can also be used Splitting databases and tables improves performance, but it is not our most recommended method .

9.1.6.Redis atomicity (recommended)

Using redis to execute the setnx command is naturally idempotent. So as to achieve no repeated consumption

9.2.Priority Queue

9.2.1.Usage scenarios

There is an order reminder scenario in our system . When our customers place an order on Tmall, Taobao will push the order to us in time. If payment is not made within the time set by the user, a text message reminder will be pushed to the user. , a very simple function, right, but for us, tmall merchants must be divided into big customers and small customers , right? For example, big merchants like Apple and Xiaomi can create a lot of profits for us at least a year , so of course, their orders must be processed with priority . In the past, our back-end system used redis to store regular polling. Everyone knows that redis can only use List to make a simple message queue, which cannot be implemented. A priority scenario , so when the order volume is large, RabbitMQ is used for transformation and optimization . If it is found that the order is from a large customer, it will be given a relatively high priority, otherwise it will be the default priority.

• The priority of sending message No. 7 is 3, and the priority of sending message No. 9 is 4...
• Under normal circumstances, the messages sent from the queue to the consumer are first in, first out. Once the priority queue is used, the messages in the queue will be queued. The larger the number, the higher the priority and will be consumed first.

9.2.2.How to add

1) Add console page (Method 1: Not recommended)

2) Code method (Method 2: Recommended)

a. Add priority to the code in the queue

Map<String, Object> params = new HashMap();
params.put("x-max-priority", 10);
channel.queueDeclare("hello", true, false, false, params); 

b. Add priority to the code in the message

AMQP.BasicProperties properties = new AMQP.BasicProperties().builder().priority(5).build();

3) Things to note

The things that need to be done in order for the queue to implement priority are the following: the queue needs to be set as a priority queue, the message needs to set the priority of the message, and the consumer needs to wait for the message to be sent to the queue before consuming it because , in this way, there is a chance to Messages are sorted

• In order to ensure that the consumer needs to wait for the message to be sent to the queue before consuming it, the producer must be started first and then the consumer. In this way, after 100 pieces of data are sent to the queue, they will be sorted according to high priority before being consumed. consumed by the person. If you start the consumer first and then the producer, it is equivalent to receiving one message and consuming one, so you will not see the priority effect.

9.2.3. Actual combat

• I am not rewriting the code, I am using the previous code.

a.Message producer

package com.atguigu.rabbitmq.one;

import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

import java.util.HashMap;
import java.util.Map;

/**
 * 生产者：发消息
 */
public class Producer {
    
    

    //队列名称
    private final static String QUEUE_NAME = "hello";

    //发消息
    public static void main(String[] args) throws Exception{
    
    
        //创建一个连接工厂
        ConnectionFactory factory = new ConnectionFactory();
        //工厂ip 连接RabbitMQ的队列
        factory.setHost("192.168.10.120");
        //用户名
        factory.setUsername("admin");
        //密码
        factory.setPassword("123456");

        //创建连接
        Connection connection = factory.newConnection();
        //获取信道
        Channel channel = connection.createChannel();
        //入门级测试，这里直接连接的是队列，没有连接交换机，用的是默认的交换机。
        /**
         * 生成一个队列,参数解释：
         *  1.queue：队列名称
         *  2.durable：是否持久化，当MQ重启后，还在 （true：持久化，保存在磁盘    false：不持久化，保存在内存）
         *  3.exclusive： false不独占，可以多个消费者消费。true只能供一个消费者进行消费
         *      功能1：是否独占。只能有一个消费者监听这个队列
         *      功能2：当Connection关闭时，是否删除队列
         *  4.autoDelete：是否自动删除 最后一个消费者端开连接以后 该队列是否自动删除 true自动删除 false不自动删除
         *  5.arguments：其它参数，在高级特性中讲，暂时设置为null
         */
        Map<String, Object> params = new HashMap();
        //官方允许是0-255之间 此处设置10 允许优化级范围为0-10 不要设置过大 浪费CPU与内存
        params.put("x-max-priority", 10);
        channel.queueDeclare(QUEUE_NAME,true,false,false,params);

/*        //发消息
        String message = "hello world"; //初次使用

        *//**
         *发送一个消息
         *1.发送到那个交换机       本次是入门级程序，没有考虑交换机，可以写空串
         *2.路由的 key 是哪个     本次是队列的名称（前提：使用默认交换机）
         *3.其他的参数信息        本次没有
         *4.发送消息的消息体      不能直接发消息，需要调用它的二进制。
         *//*
        channel.basicPublish("",QUEUE_NAME,null,message.getBytes());
        System.out.println("消息发送完毕");        */

        //改为发送多条消息
        for(int i=1;i<11;i++){
    
    
            String message = "info"+i;
            //设置第5条消息的优先级，其它正常发布
            if(i==5){
    
    
                //构建优先级
                AMQP.BasicProperties properties = new AMQP.BasicProperties().builder().priority(5).build();
                //发送消息
                channel.basicPublish("",QUEUE_NAME,properties,message.getBytes());
            }else {
    
    
                channel.basicPublish("",QUEUE_NAME,null,message.getBytes());
            }
        }
        System.out.println("消息发送完毕");
    }
}

b.Message consumer

• Consumers are the same as before
  

package com.atguigu.rabbitmq.one;

import com.rabbitmq.client.*;

/**
 * 消费者：接收消息的
 */
public class Consumer {
    
    
    //队列名称
    private final static String QUEUE_NAME = "hello";

    //接收消息
    public static void main(String[] args) throws Exception {
    
    
        //创建连接工厂
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("192.168.10.120");//设置ip
        factory.setUsername("admin");//设置用户名
        factory.setPassword("123456");//设置密码
        Connection connection = factory.newConnection();//创建连接
        Channel channel = connection.createChannel();//通过连接创建信道
        System.out.println("等待接收消息	");


        //推送的消息如何进行消费的接口回调 使用lambda表达式代替匿名内部类的写法
        DeliverCallback deliverCallback = (consumerTag, message) -> {
    
    
            //直接输出message参数是对象的地址值，通过方法获取message对象的消息体并转化为字符串输出。
            String mes = new String(message.getBody());
            System.out.println(mes);
        };
        //取消消费的一个回调接口 如在消费的时候队列被删除掉了
        CancelCallback cancelCallback = (consumerTag) -> {
    
    
            System.out.println("消息消费被中断");
        };
        /**
         *消费者接收消息
         *1.消费哪个队列
         *2.消费成功之后是否要自动应答 true 代表自动应答 false 手动应答
         *3.消费者接收消息的回调
         *4.消费者取消消息的回调
         *
         * 执行流程：在basicConsume（）方法中接收到消息后，会把消息放到声明消息的接口里deliverCallback，
         *         在使用此接口的实现类打印。
         */
        channel.basicConsume(QUEUE_NAME, true, deliverCallback, cancelCallback);
    }
}

Test: Start the consumer first, and then start the producer. You can see that messages with high priority are consumed first.

9.3. Lazy queue

9.3.1.Usage scenarios

RabbitMQ introduced the concept of lazy queue starting from version 3.6.0. The lazy queue will store messages on disk as much as possible, and will only be loaded into memory when the consumer consumes the corresponding message. One of its important design goals is to be able to support longer queues, that is, to support more message storage. Lazy queues are necessary when consumers are unable to consume messages for a long time due to various reasons ( such as consumers going offline, downtime, or shutting down due to maintenance, etc. ) .

By default , when a producer sends a message to RabbitMQ, the message in the queue will be stored in memory as much as possible , so that the message can be sent to the consumer faster . Even persistent messages retain a copy in memory while being written to disk . When RabbitMQ needs to release memory, it will page the messages in the memory to the disk. This operation will take a long time and will also block the queue operation, making it impossible to receive new messages. Although the developers of RabbitMQ have been upgrading related algorithms, the effect is still not ideal, especially when the message volume is particularly large.

• Disadvantages : When we send a message to mq, if the queue in mq happens to be a lazy queue, it will immediately store the message on the disk instead of in the memory. The consumer must first read it from the disk into the memory before consuming it. Afterwards, consumption will be slower.
  

9.3.2. Two modes

The queue has two modes : default and lazy. The default mode is default, and no changes are required for versions prior to 3.6.0. The lazy mode is the mode of the lazy queue. It can be set in the parameters when calling the channel.queueDeclare method, or it can be set through the Policy. If a queue is set using both methods at the same time, the Policy method has higher priority . If you want to change the mode of an existing queue through declaration , you can only delete the queue first and then re-declare a new one.

When declaring the queue, you can set the mode of the queue through the "x-queue-mode" parameter. The values ​​are "default" and "lazy". The following example demonstrates the declaration details of a lazy queue:

Map<String, Object> args = new HashMap<String, Object>(); 
args.put("x-queue-mode", "lazy"); 
channel.queueDeclare("myqueue", false, false, false, args);

9.3.3. Memory overhead comparison

When 1 million messages are sent and each message occupies approximately 1KB, the memory occupied by the ordinary queue is 1.2GB, while the lazy queue only occupies 1.5MB.

10.RabbitMQ cluster

10.1.clustering

10.1.1. Reasons for using clusters

At the beginning, we introduced how to install and run the RabbitMQ service, but these are stand-alone versions and cannot meet the requirements of current real applications. What should you do if the RabbitMQ server encounters memory corruption, machine power outage, or motherboard failure? A single RabbitMQ server can meet the throughput of 1,000 messages per second, but what if the application requires the RabbitMQ service to meet the throughput of 100,000 messages per second? It is difficult to purchase expensive servers to enhance the performance of stand-alone RabbitMQ services. Building a RabbitMQ cluster is the key to solving practical problems.

Cluster construction principle:

• The performance of a single Rabbitmq is limited. In order to cope with the large number of messages sent by producers, a cluster needs to be built.
• As shown in the figure, cluster No. 2 and cluster No. 3 have all joined node No. 1. The three of them have formed a whole. Accessing any one of your nodes is equivalent to accessing all nodes. In the future, adding node No. 4 to No. 3 or No. 2 will be equivalent to adding it to the whole.

10.1.2.Building steps

• Clone another 2 Rabbitmqs
  
  
• Modify the ip to:192.168.10.120、192.168.10.121、192.168.10.122

vim /etc/sysconfig/network-scripts/ifcfg-ens33 

1. Modify the host names of 3 machines (node1, node2, node3)

vim /etc/hostname

Note: Restart will take effect ( reboot)

2. Configure the hosts file of each node so that each node can recognize each other.

• Configure the following three pieces of information on all three Rabbitmqs.

# 配置ip对应主机名
vim /etc/hosts

192.168.10.120 node1
192.168.10.121 node2
192.168.10.122 node3 

3. Make sure that the cookie files of each node use the same value. Execute the remote operation command
on node1
. Confirm the connection: yes.
Password for the second connection: 123456 (Set the login username and password: root 123456)

scp /var/lib/rabbitmq/.erlang.cookie root@node2:/var/lib/rabbitmq/.erlang.cookie 

scp /var/lib/rabbitmq/.erlang.cookie root@node3:/var/lib/rabbitmq/.erlang.cookie

4. Start the RabbitMQ service, and at the same time start the Erlang virtual machine and RBbitMQ application service (execute the following commands on the three nodes respectively)

rabbitmq-server -detached

5. Execute on node 2

#先关掉服务(rabbitmqctl stop 会将Erlang 虚拟机关闭，rabbitmqctl stop_app 只关闭 RabbitMQ 服务) 
rabbitmqctl stop_app

#重置服务
rabbitmqctl reset

#把节点2加入到一号节点中
rabbitmqctl join_cluster rabbit@node1 

#再重启(只启动应用服务)
rabbitmqctl start_app

6. Execute on node 3

rabbitmqctl stop_app

rabbitmqctl reset

#把节点3加入到2号节点中（因为2号节点已经加入到1号节点中了相当于是一个整体，那么3号加点加入到1号或者2号都可以）
#前提是2号节点已经启动好了
rabbitmqctl join_cluster rabbit@node2

rabbitmqctl start_app

7. Cluster status
The current three nodes are already a whole, so anyone can access them.

rabbitmqctl cluster_status

8. User needs to be reset.
Note: Because 3 Rabbitmqs form a cluster, if you create an account under any node, other nodes will also have accounts.

• Create an account
  rabbitmqctl add_user admin 123
• Set user role
  rabbitmqctl set_user_tags admin administrator
• Set user permissions
  rabbitmqctl set_permissions -p "/" admin ".*" ".*" ".*"

Log in to any account and you can see that the original one node has changed to the current three nodes.

9. Deactivate the cluster nodes (executed separately on node2 and node3 machines)

# 想要解除2号节点，以下4个命令就在2号节点下执行
rabbitmqctl stop_app

rabbitmqctl reset 

rabbitmqctl start_app 

rabbitmqctl cluster_status

#之后在1号机上执行，让1号机忘记集群2号机
rabbitmqctl forget_cluster_node rabbit@node2(node1 机器上执行)

10.2. Mirror queue (to be determined)

10.2.1. Reasons for using mirroring

If there is only one Broker node in the RabbitMQ cluster, the failure of this node will cause the overall service to be temporarily unavailable, and may also lead to the loss of messages . All messages can be set to be persistent, and the durable attribute of the corresponding queue is also set to true, but this still cannot avoid problems caused by caching: because there is a gap between the time the message is sent and the time it is written to the disk and the flushing action is performed. A short but problematic window of time. The publisherconfirm mechanism ensures that the client knows which messages have been stored on disk. However, you generally do not want to encounter service unavailability due to a single point of failure.

Introducing the Mirror Queue mechanism , the queue can be mirrored to other Broker nodes in the cluster. If a node in the cluster fails, the queue can automatically switch to another node in the mirror to ensure service continuity. Availability.

• Problem: At present, the mq cluster cannot be reused at all. The queue created on the first machine does not exist on the second machine. Once the first machine goes down, the queue will disappear. It will not be used just because there are three machines. There are 3 queues, but only one. The queue is created on that machine, and there is no queue at all on other machines.
  
• Solution: Use a mirror queue, that is, make a backup for each machine. At least one message should not be sent to only one node. For example, if it exists on node No. 1, the message will be lost once node No. 1 goes down. Send a message to node 1, and node 1 will back up the message to node 2.
• Note: You can also back up both Node 2 and Node 3, but it will only waste resources. If there are as many as 1 million messages, backing up several times is a waste of space.

10.2.2.Building steps

1. Start three cluster nodes

2. Find a node and add policy (policy)

3. Create a queue on node1 to send a message. The queue has a mirror queue.

4. Stop node1 and find that node2 becomes a mirror queue.

5. Even if there is only one machine left in the entire cluster, messages in the queue can still be consumed.

It means that the messages in the queue are delivered to the corresponding machine by the mirror queue.

10.3.Haproxy+Keepalive achieves high availability load balancing

• Problem: The current code is hard-coded and can only connect to one MQ. If the current MQ connected is down, the producer is still connected to the current MQ and messages cannot be sent. (A big problem when producers connect to MQ clusters: the IP address connecting to MQ cannot be changed)
  
• Solution: Use third-party tools such as Haproxy.

10.3.1. Overall architecture diagram

10.3.2.Haproxy implements load balancing

HAProxy provides high availability, load balancing and TCPHTTP application-based proxies, supporting virtual hosts. It is a free, fast and reliable solution used by many well-known Internet companies including Twitter, Reddit, StackOverflow, and GitHub. HAProxy implements an event-driven, single-process model that supports very large numbers of direct connections.

Extend the difference between nginx, lvs, and haproxy:http://www.ha97.com/5646.html

10.3.3.Building steps

1. Download haproxy (on node1 and node2)

yum -y install haproxy

2. Modify haproxy.cfg of node1 and node2

vim /etc/haproxy/haproxy.cfg

Need to modify the red IP to the current machine IP

3. Start haproxy on both nodes

haproxy -f /etc/haproxy/haproxy.cfg
ps -ef | grep haproxy

4.Access address

http://10.211.55.71:8888/stats

10.3.4.Keepalived realizes dual-machine (active and backup) hot backup

Imagine that if the HAProxy host configured earlier suddenly crashes or the network card fails, then although there is no fault in the RbbitMQ cluster, all connections will be disconnected for external clients. The result will be catastrophic. To ensure the reliability of the load balancing service It is also very important. Here we need to introduce Keepalived, which can achieve high availability (dual-machine hot backup) through its own health check and resource takeover functions to achieve failover.

10.3.5.Building steps

1. Download keepalived

yum -y install keepalived

2.Node node1 configuration file

vim /etc/keepalived/keepalived.conf

Modify and replace keepalived.conf in the information.

3.Node node2 configuration file

• Need to modify the router_id of global_defs, such as: nodeB
• Secondly, modify the state in vrrp_instance_VI to "BACKUP";
• Finally, set priority to a value less than 100

4. Add haproxy_chk.sh

(In order to prevent Keepalived from still working normally after the HAProxy service hangs up without switching to Backup, a script needs to be written here to detect the status of the HAProxy service. When the HAProxy service hangs up, the script will automatically restart the HAProxy service. If not If successful, close the Keepalived service so that you can switch to Backup and continue working)

vim /etc/keepalived/haproxy_chk.sh(可以直接上传文件)
修改权限 chmod 777 /etc/keepalived/haproxy_chk.sh

5. Start the keepalive command (node1 and node2 start)

systemctl start keepalived

6. Observe Keepalived logs

tail -f /var/log/messages -n 200

7. Observe the latest added VIPs

ip add show

8.node1 simulates keepalived closed state

systemctl stop keepalived 

9. Use vip address to access rabbitmq cluster

10.4.Federation Exchange (Federation Exchange)

10.4.1. Reasons to use it

(broker Beijing), (broker Shenzhen) are far apart from each other, and network delay is a problem that has to be faced . There is a business in Beijing (Client Beijing) that needs to connect to (broker Beijing) and send a message to the exchangeA in it. The network delay at this time is very small. (Client Beijing) can quickly send the message to exchangeA even if it is turned on. When the publisherconfirm mechanism or transaction mechanism is used, confirmation information can also be received quickly. At this time, there is another business in Shenzhen (Client Shenzhen) that needs to send messages to exchangeA. Then there is a large network delay between (Client Shenzhen) and (broker Beijing). (Client Shenzhen) will experience a certain amount of time when sending messages to exchangeA. Delay, especially when the publisherconfirm mechanism or transaction mechanism is enabled, (Client Shenzhen) will wait for a long delay to receive the confirmation information from (broker Beijing), which will inevitably cause the performance of this sending thread to decrease, and even cause Blockage to a certain extent.

Deploying the business (Client Shenzhen) to the computer room in Beijing can solve this problem, but if other services called by (Client Shenzhen) are deployed in Shenzhen, it will cause new latency problems, and it is not necessarily possible to deploy all the services. In a computer room, how can disaster recovery be achieved? Using the Federation plug-in here can solve this problem very well.

• Problem: The two regions are quite far apart. There is no problem for customers in Beijing to access the switches in the Beijing computer room, but the network delay for customers in Beijing to access the switches in the Shenzhen computer room is relatively large.
• Solution: Synchronize the computer room information in Beijing to Shenzhen, and synchronize the information in Shenzhen computer room to Beijing to ensure that the information in the two computer rooms is consistent. This way, customers in Beijing can quickly access data information when visiting Beijing. ----Use Federation Exchange (Federation Exchange)
  

10.4.2.Building steps

1. It is necessary to ensure that each node runs independently
2. Enable federation related plug-ins on each machine

rabbitmq-plugins enable rabbitmq_federation

rabbitmq-plugins enable rabbitmq_federation_management

3. Schematic diagram (first run consumer to create fed_exchange on node2)

4. Configure upstream (node1) in downstream (node2)

5.Add policy

6. Prerequisites for success

10.5.Federation Queue

• You can use switch synchronization or queue synchronization.

10.5.1. Reasons to use it

Federated queues can provide load balancing for a single queue among multiple Broker nodes (or clusters). A federated queue can connect to one or more upstream queues and obtain messages from these upstream queues to meet the needs of local consumers for consuming messages.

10.5.2.Building steps

1. Schematic diagram

2. Add upstream (same as above)
3. Add policy

10.6.Shovel

10.6.1. Reasons to use it

Federation has a similar data forwarding function. Shovel can reliably and continuously pull data from the queue in one Broker (as the source, that is, the source) and forward it to the exchange in another Broker (as the destination, that is, the destination) . The queue as the source and the exchange as the destination can be located on the same Broker at the same time, or they can be located on different Brokers. Shovel can be translated as "shovel", which is a more vivid metaphor. This "shovel" can "shovel" messages from one party to the other. Shovel behaves like a good client application that is responsible for connecting sources and destinations, reading and writing messages, and handling connection failures.

• The function is the same as that of federated switches and federated queues, except that the source end can be set directly.

10.6.2.Building steps

1. Enable the plug-in (enable all required machines)

rabbitmq-plugins enable rabbitmq_shovel
rabbitmq-plugins enable rabbitmq_shovel_management

2. Principle diagram (messages sent at the source directly enter the destination queue)

3. Add shovel source and destination