RabbitMQ使用 详解

RabbitMQ快速入门

1.初识MQ

1.1.同步和异步通讯

微服务间通讯有同步和异步两种方式:

同步通讯:就像打电话,需要实时响应。

异步通讯:就像发邮件,不需要马上回复。

image-20210717161939695

两种方式各有优劣,打电话可以立即得到响应,但是你却不能跟多个人同时通话。发送邮件可以同时与多个人收发邮件,但是往往响应会有延迟。

1.1.1.同步通讯

我们之前了解的Feign调用就属于同步方式,虽然调用可以实时得到结果,但存在下面的问题:

image-20210717162004285

总结:

同步调用的优点:

  • 时效性较强,可以立即得到结果

同步调用的问题:

  • 耦合度高
  • 性能和吞吐能力下降
  • 有额外的资源消耗
  • 有级联失败问题
1.1.2.异步通讯

异步调用则可以避免上述问题:

我们以购买商品为例,用户支付后需要调用订单服务完成订单状态修改,调用物流服务,从仓库分配响应的库存并准备发货。

在事件模式中,支付服务是事件发布者(publisher),在支付完成后只需要发布一个支付成功的事件(event),事件中带上订单id。

订单服务和物流服务是事件订阅者(Consumer),订阅支付成功的事件,监听到事件后完成自己业务即可。

为了解除事件发布者与订阅者之间的耦合,两者并不是直接通信,而是有一个中间人(Broker)。发布者发布事件到Broker,不关心谁来订阅事件。订阅者从Broker订阅事件,不关心谁发来的消息。

image-20210422095356088

Broker 是一个像数据总线一样的东西,所有的服务要接收数据和发送数据都发到这个总线上,这个总线就像协议一样,让服务间的通讯变得标准和可控。

好处:

  • 吞吐量提升:无需等待订阅者处理完成,响应更快速

  • 故障隔离:服务没有直接调用,不存在级联失败问题

  • 调用间没有阻塞,不会造成无效的资源占用

  • 耦合度极低,每个服务都可以灵活插拔,可替换

  • 流量削峰:不管发布事件的流量波动多大,都由Broker接收,订阅者可以按照自己的速度去处理事件

缺点:

  • 架构复杂了,业务没有明显的流程线,不好管理
  • 需要依赖于Broker的可靠、安全、性能

好在现在开源软件或云平台上 Broker 的软件是非常成熟的,比较常见的一种就是我们今天要了解的MQ技术。

1.2.技术对比:

MQ,中文是消息队列(MessageQueue),字面来看就是存放消息的队列。也就是事件驱动架构中的Broker。

比较常见的MQ实现:

  • ActiveMQ
  • RabbitMQ
  • RocketMQ
  • Kafka

几种常见MQ的对比:

RabbitMQ ActiveMQ RocketMQ Kafka
公司/社区 Rabbit Apache 阿里 Apache
开发语言 Erlang Java Java Scala&Java
协议支持 AMQP,XMPP,SMTP,STOMP OpenWire,STOMP,REST,XMPP,AMQP 自定义协议 自定义协议
可用性 一般
单机吞吐量 一般 非常高
消息延迟 微秒级 毫秒级 毫秒级 毫秒以内
消息可靠性 一般 一般

追求可用性:Kafka、 RocketMQ 、RabbitMQ

追求可靠性:RabbitMQ、RocketMQ

追求吞吐能力:RocketMQ、Kafka

追求消息低延迟:RabbitMQ、Kafka

2.快速入门

2.1.安装RabbitMQ

1.单机部署

我们在Centos7虚拟机中使用Docker来安装。

Docker使用参考:Docker使用详解

1.1.下载镜像

方式一:在线拉取

docker pull rabbitmq:3-management

方式二:从本地加载

将现有镜像文件上传到虚拟机中后,使用命令加载镜像即可:

docker load -i mq.tar
1.2.安装MQ

执行下面的命令来运行MQ容器:

docker run \
 -e RABBITMQ_DEFAULT_USER=test \
 -e RABBITMQ_DEFAULT_PASS=123321 \
 --name mq \
 --hostname mq \
 -p 15672:15672 \
 -p 5672:5672 \
 -d \
 rabbitmq:3.8-management
 
 # 需要插件
 docker run \
 -e RABBITMQ_DEFAULT_USER=rabbitmq \
 -e RABBITMQ_DEFAULT_PASS=1234 \
 -v mq-plugins:/plugins \
 --name mq \
 --hostname mq \
 -p 15672:15672 \
 -p 5672:5672 \
 -d \
 rabbitmq:3.8-management
2.集群部署
2.1.集群分类

在RabbitMQ的官方文档中,讲述了两种集群的配置方式:

  • 普通模式:普通模式集群不进行数据同步,每个MQ都有自己的队列、数据信息(其它元数据信息如交换机等会同步)。例如我们有2个MQ:mq1,和mq2,如果你的消息在mq1,而你连接到了mq2,那么mq2会去mq1拉取消息,然后返回给你。如果mq1宕机,消息就会丢失。
  • 镜像模式:与普通模式不同,队列会在各个mq的镜像节点之间同步,因此你连接到任何一个镜像节点,均可获取到消息。而且如果一个节点宕机,并不会导致数据丢失。不过,这种方式增加了数据同步的带宽消耗。

我们先来看普通模式集群。

设置网络

首先,我们需要让3台MQ互相知道对方的存在。

分别在3台机器中,设置 /etc/hosts文件,添加如下内容:

# 改为自己虚拟主机IP
192.168.48.101 mq1
192.168.48.102 mq2
192.168.48.103 mq3

并在每台机器上测试,是否可以ping通对方:

MQ的基本结构:

image-20210717162752376

RabbitMQ中的一些角色:

  • publisher:生产者
  • consumer:消费者
  • exchange个:交换机,负责消息路由
  • queue:队列,存储消息
  • virtualHost:虚拟主机,隔离不同租户的exchange、queue、消息的隔离,默认/
2.2.获取cookie

RabbitMQ底层依赖于Erlang,而Erlang虚拟机就是一个面向分布式的语言,默认就支持集群模式。集群模式中的每个RabbitMQ 节点使用 cookie 来确定它们是否被允许相互通信。

要使两个节点能够通信,它们必须具有相同的共享秘密,称为Erlang cookie。cookie 只是一串最多 255 个字符的字母数字字符。

每个集群节点必须具有相同的 cookie。实例之间也需要它来相互通信。

我们先在之前启动的mq容器中获取一个cookie值,作为集群的cookie。执行下面的命令:

# 运行mq容器
[root@localhost ~]# docker start mq
mq

# 获取运行mq的cookie值
[root@localhost ~]# docker exec -it mq cat /var/lib/rabbitmq/.erlang.cookie
ACAFGDJNGQVZXBJWBZHP[root@localhost cloud-demo]#

可以看到cookie值如下:

ACAFGDJNGQVZXBJWBZHP

接下来,停止并删除当前的mq容器,我们重新搭建集群。

# 停止mq容器
[root@localhost ~]# docker stop mq
mq
docker rm -f mq
2.3.准备集群配置

在/tmp目录新建一个配置文件 rabbitmq.conf:

cd /tmp
#### 创建文件
touch rabbitmq.conf

文件内容如下:

loopback_users.guest = false
listeners.tcp.default = 5672
cluster_formation.peer_discovery_backend = rabbit_peer_discovery_classic_config
cluster_formation.classic_config.nodes.1 = rabbit@mq1
cluster_formation.classic_config.nodes.2 = rabbit@mq2
cluster_formation.classic_config.nodes.3 = rabbit@mq3

再创建一个文件,记录cookie

cd /tmp
#### 创建cookie文件
touch .erlang.cookie
#### 写入cookie
echo "ACAFGDJNGQVZXBJWBZHP" > .erlang.cookie
#### 修改cookie文件的权限
chmod 600 .erlang.cookie

准备三个目录,mq1、mq2、mq3:

cd /tmp
#### 创建目录
mkdir mq1 mq2 mq3

然后拷贝rabbitmq.conf、cookie文件到mq1、mq2、mq3:

#### 进入/tmp
cd /tmp
#### 拷贝
cp rabbitmq.conf mq1
cp rabbitmq.conf mq2
cp rabbitmq.conf mq3
cp .erlang.cookie mq1
cp .erlang.cookie mq2
cp .erlang.cookie mq3
2.4.启动集群

创建一个网络:

docker network create mq-net

docker volume create

运行命令

集群rabbitmq版本需在3.8以上

若不需要插件:删除 *-v mq-plugins:/plugins * 即可

docker run -d --net mq-net \
-v ${PWD}/mq1/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf \
-v ${PWD}/.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie \
-v mq-plugins:/plugins \
-e RABBITMQ_DEFAULT_USER=test \
-e RABBITMQ_DEFAULT_PASS=123321 \
--name mq1 \
--hostname mq1 \
-p 8071:5672 \
-p 8081:15672 \
rabbitmq:3.8-management
docker run -d --net mq-net \
-v ${PWD}/mq2/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf \
-v ${PWD}/.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie \
-v mq-plugins:/plugins \
-e RABBITMQ_DEFAULT_USER=test \
-e RABBITMQ_DEFAULT_PASS=123321 \
--name mq2 \
--hostname mq2 \
-p 8072:5672 \
-p 8082:15672 \
rabbitmq:3.8-management
docker run -d --net mq-net \
-v ${PWD}/mq3/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf \
-v ${PWD}/.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie \
-v mq-plugins:/plugins \
-e RABBITMQ_DEFAULT_USER=test \
-e RABBITMQ_DEFAULT_PASS=123321 \
--name mq3 \
--hostname mq3 \
-p 8073:5672 \
-p 8083:15672 \
rabbitmq:3.8-management
2.5.测试

在mq1这个节点上添加一个队列:RabbitMQ Management

image-20210717222833196

如图,在mq2和mq3两个控制台也都能看到:

image-20210717223057902

2.5.1.数据共享测试

点击这个队列,进入管理页面:

image-20210717223421750

然后利用控制台发送一条消息到这个队列:

image-20210717223320238

结果在mq2、mq3上都能看到这条消息:

image-20210717223603628

2.5.2.可用性测试

我们让其中一台节点mq1宕机:

docker stop mq1

然后登录mq2或mq3的控制台,发现simple.queue也不可用了:

image-20210717223800203

说明数据并没有拷贝到mq2和mq3。

2.2.RabbitMQ消息模型

RabbitMQ官方提供了5个不同的Demo示例,对应了不同的消息模型:

image-20210717163332646

2.3.创建Demo工程

创建一个Demo工程,mq-demo:

├── consumer
│   ├── pom.xml
│   └── src
│       ├── main
│       └── test
├── pom.xml
└── publisher
    ├── pom.xml
    └── src
        ├── main
        └── test

包括三部分:

  • mq-demo:父工程,管理项目依赖
  • publisher:消息的发送者
  • consumer:消息的消费者

2.4.入门案例

简单队列模式的模型图:

image-20210717163434647

官方的HelloWorld是基于最基础的消息队列模型来实现的,只包括三个角色:

  • publisher:消息发布者,将消息发送到队列queue
  • queue:消息队列,负责接受并缓存消息
  • consumer:订阅队列,处理队列中的消息
2.4.1.publisher implementation

Idea:

  • establish connection
  • CreateChannel
  • declare queue
  • Send a message
  • Close connection and channel

Code:

package com.example.mq.helloworld;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import org.junit.Test;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

public class PublisherTest {
    
    
    @Test
    public void testSendMessage() throws IOException, TimeoutException {
    
    
        // 1.建立连接
        ConnectionFactory factory = new ConnectionFactory();
        // 1.1.设置连接参数,分别是:主机名、端口号、vhost、用户名、密码
        factory.setHost("192.168.48.101");
        factory.setPort(5672);
        factory.setVirtualHost("/");
        factory.setUsername("test");
        factory.setPassword("123321");
        // 1.2.建立连接
        Connection connection = factory.newConnection();

        // 2.创建通道Channel
        Channel channel = connection.createChannel();

        // 3.创建队列
        String queueName = "simple.queue";
        channel.queueDeclare(queueName, false, false, false, null);

        // 4.发送消息
        String message = "hello, rabbitmq!";
        channel.basicPublish("", queueName, null, message.getBytes());
        System.out.println("发送消息成功:【" + message + "】");

        // 5.关闭通道和连接
        channel.close();
        connection.close();

    }
}
2.4.2.consumer implementation

Code idea:

  • establish connection
  • CreateChannel
  • declare queue
  • Subscribe to news

Code:

package com.example.mq.helloworld;

import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

public class ConsumerTest {
    
    

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        // 1.建立连接
        ConnectionFactory factory = new ConnectionFactory();
        // 1.1.设置连接参数,分别是:主机名、端口号、vhost、用户名、密码
        factory.setHost("192.168.48.101");
        factory.setPort(5672);
        factory.setVirtualHost("/");
        factory.setUsername("test");
        factory.setPassword("123321");
        // 1.2.建立连接
        Connection connection = factory.newConnection();

        // 2.创建通道Channel
        Channel channel = connection.createChannel();

        // 3.创建队列
        String queueName = "simple.queue";
        channel.queueDeclare(queueName, false, false, false, null);

        // 4.订阅消息
        channel.basicConsume(queueName, true, new DefaultConsumer(channel){
    
    
            @Override
            public void handleDelivery(String consumerTag, Envelope envelope,
                                       AMQP.BasicProperties properties, byte[] body) throws IOException {
    
    
                // 5.处理消息
                String message = new String(body);
                System.out.println("接收到消息:【" + message + "】");
            }
        });
        System.out.println("等待接收消息。。。。");
    }
}

2.5. Summary

The message sending process of the basic message queue:

  1. Establish connection

  2. create channel

  3. Use channel to declare queue

  4. Use channel to send messages to queue

The message receiving process of the basic message queue:

  1. Establish connection

  2. create channel

  3. Use channel to declare queue

  4. Define the consumer's consumption behavior handleDelivery()

  5. Use channels to bind consumers to queues

3.SpringAMQP

SpringAMQP is a set of templates based on RabbitMQ encapsulation, and it also uses SpringBoot to implement automatic assembly, which is very convenient to use.

The official address of SpringAmqp: https://spring.io/projects/spring-amqp

image-20210717164024967

image-20210717164038678

SpringAMQP provides three functions:

  • Automatically declare queues, switches and their binding relationships
  • Annotation-based listener mode, asynchronously receiving messages
  • Encapsulates the RabbitTemplate tool for sending messages

3.1.Basic Queue simple queue model

Introduce dependencies into the parent project mq-demo

<!--AMQP依赖,包含RabbitMQ-->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
3.1.1.Message sending

First configure the MQ address and add the configuration in application.yml of the publisher service:

spring:
  rabbitmq:
    host: 192.168.48.101 ## 主机名
    port: 5672 ## 端口
    virtual-host: / ## 虚拟主机
    username: test ## 用户名
    password: 123321 ## 密码

Then write the test class SpringAmqpTest in the publisher service and use RabbitTemplate to send messages:

package com.example.mq.spring;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest
public class SpringAmqpTest {
    
    

    @Autowired
    private RabbitTemplate rabbitTemplate;

    @Test
    public void testSimpleQueue() {
    
    
        // 队列名称
        String queueName = "simple.queue";
        // 消息
        String message = "hello, spring amqp!";
        // 发送消息
        rabbitTemplate.convertAndSend(queueName, message);
    }
}
3.1.2. Message reception

First configure the MQ address and add the configuration in application.yml of the consumer service:

spring:
  rabbitmq:
    host: 192.168.48.101 ## 主机名
    port: 5672 ## 端口
    virtual-host: / ## 虚拟主机
    username: test ## 用户名
    password: 123321 ## 密码

Then create a new class SpringRabbitListener in the consumer servicecom.example.mq.listener package, the code is as follows:

package com.example.mq.listener;

import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;

@Component
public class SpringRabbitListener {
    
    

    @RabbitListener(queues = "simple.queue")
    public void listenSimpleQueueMessage(String msg) throws InterruptedException {
    
    
        System.out.println("spring 消费者接收到消息:【" + msg + "】");
    }
}
3.1.3.Testing

Start the consumer service, then run the test code in the publisher service and send MQ messages

3.2.WorkQueue

Work queues, also known as (Task queues), task model. To put it simply, allows multiple consumers to bind to a queue and jointly consume messages in the queue.

image-20210717164238910

When message processing is time-consuming, the speed of message production may be much greater than the speed of message consumption. If things go on like this, more and more messages will accumulate, making it impossible to process them in a timely manner.

At this time, the work model can be used, and multiple consumers can handle message processing together, and the speed can be greatly improved.

3.2.1.Message sending

This time we send it in a loop to simulate the accumulation of a large number of messages.

Add a test method in the SpringAmqpTest class in the publisher service:

/**
     * workQueue
     * 向队列中不停发送消息,模拟消息堆积。
     */
@Test
public void testWorkQueue() throws InterruptedException {
    
    
    // 队列名称
    String queueName = "simple.queue";
    // 消息
    String message = "hello, message_";
    for (int i = 0; i < 50; i++) {
    
    
        // 发送消息
        rabbitTemplate.convertAndSend(queueName, message + i);
        Thread.sleep(20);
    }
}
3.2.2. Message reception

To simulate multiple consumers binding to the same queue, we add 2 new methods in the SpringRabbitListener of the consumer service:

@RabbitListener(queues = "simple.queue")
public void listenWorkQueue1(String msg) throws InterruptedException {
    
    
    System.out.println("消费者1接收到消息:【" + msg + "】" + LocalTime.now());
    Thread.sleep(20);
}

@RabbitListener(queues = "simple.queue")
public void listenWorkQueue2(String msg) throws InterruptedException {
    
    
    System.err.println("消费者2........接收到消息:【" + msg + "】" + LocalTime.now());
    Thread.sleep(200);
}

Notice that this consumer slept for 1000 seconds and the simulation task took time.

3.2.3.Testing

After starting the ConsumerApplication, execute the sending test method testWorkQueue just written in the publisher service.

You can see that consumer 1 quickly completed his 25 messages. Consumer 2 is slowly processing its 25 messages.

In other words, messages are distributed equally to each consumer, without taking into account the consumer's processing capabilities. This is obviously problematic.

3.2.4. Those who can do more work

There is a simple configuration in spring that can solve this problem. We modify the application.yml file of the consumer service and add the configuration:

spring:
  rabbitmq:
    listener:
      simple:
        prefetch: 1 ## 每次只能获取一条消息,处理完成才能获取下一个消息
3.2.5. Summary

Use of Work model:

  • Multiple consumers are bound to a queue, and the same message will only be processed by one consumer.
  • Control the number of messages prefetched by the consumer by setting prefetch

3.3.Publish/Subscribe

The model of publish and subscribe is as follows:

image-20210717165309625

As you can see, in the subscription model, there is an additional exchange role, and the process has changed slightly:

  • Publisher: Producer, that is, the program that wants to send messages, but no longer sends them to the queue, but to X (switch)
  • Exchange: switch, X in the picture. On the one hand, messages sent by producers are received. On the other hand, knowing how to handle the message, such as submitting it to a special queue, submitting it to all queues, or discarding the message. How to do this depends on the type of Exchange. Exchange has the following 3 types:
    • Fanout: Broadcast, delivering messages to all queues bound to the switch
    • Direct: Directed, the message is delivered to the queue that matches the specified routing key.
    • Topic: Wildcard, hand the message to the queue that matches the routing pattern (routing pattern)
  • Consumer: Consumer, as before, subscribes to the queue, no changes
  • Queue: The message queue is the same as before, receiving messages and caching messages.

Exchange (switch) is only responsible for forwarding messages and does not have the ability to store messages, so if there is no queue bound to Exchange, or there is no queue that meets the routing rules, then the message will be lost!

3.4.Fanout

Fanout, the English translation is fanout, I think it is more appropriate to call it broadcast in MQ.

image-20210717165438225

In broadcast mode, the message sending process is as follows:

  • 1) There can be multiple queues
  • 2) Each queue must be bound to Exchange (switch)
  • 3) Messages sent by producers can only be sent to the switch. The switch decides which queue to send to, and the producer cannot decide.
  • 4) The switch sends the message to all bound queues
  • 5) Consumers who subscribe to the queue can get the message

Our plan is this:

  • Create a switch test.fanout, the type is Fanout
  • Create two queues fanout.queue1 and fanout.queue2 and bind them to the switch test.fanout

image-20210717165509467

3.4.1. Declare queues and switches

Spring provides an interface Exchange to represent all different types of switches:

image-20210717165552676

Create a class in consumer and declare queues and switches:

package com.example.mq.config;

import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.FanoutExchange;
import org.springframework.amqp.core.Queue;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class FanoutConfig {
    
    
    /**
     * 声明交换机
     * @return Fanout类型交换机
     */
    @Bean
    public FanoutExchange fanoutExchange(){
    
    
        return new FanoutExchange("test.fanout");
    }

    /**
     * 第1个队列
     */
    @Bean
    public Queue fanoutQueue1(){
    
    
        return new Queue("fanout.queue1");
    }

    /**
     * 绑定队列和交换机
     */
    @Bean
    public Binding bindingQueue1(Queue fanoutQueue1, FanoutExchange fanoutExchange){
    
    
        return BindingBuilder.bind(fanoutQueue1).to(fanoutExchange);
    }

    /**
     * 第2个队列
     */
    @Bean
    public Queue fanoutQueue2(){
    
    
        return new Queue("fanout.queue2");
    }

    /**
     * 绑定队列和交换机
     */
    @Bean
    public Binding bindingQueue2(Queue fanoutQueue2, FanoutExchange fanoutExchange){
    
    
        return BindingBuilder.bind(fanoutQueue2).to(fanoutExchange);
    }
}
3.4.2. Message sending

Add a test method in the SpringAmqpTest class of the publisher service:

@Test
public void testFanoutExchange() {
    
    
    // 队列名称
    String exchangeName = "test.fanout";
    // 消息
    String message = "hello, everyone!";
    rabbitTemplate.convertAndSend(exchangeName, "", message);
}
3.4.3. Message reception

Add two methods in the SpringRabbitListener of the consumer service as consumers:

@RabbitListener(queues = "fanout.queue1")
public void listenFanoutQueue1(String msg) {
    
    
    System.out.println("消费者1接收到Fanout消息:【" + msg + "】");
}

@RabbitListener(queues = "fanout.queue2")
public void listenFanoutQueue2(String msg) {
    
    
    System.out.println("消费者2接收到Fanout消息:【" + msg + "】");
}
3.4.4. Summary

What is the role of a switch?

  • Receive messages sent by publisher
  • Route messages to queues bound to them according to rules
  • The message cannot be cached, routing fails, and the message is lost.
  • FanoutExchange will route messages to each bound queue

What are the beans that declare queues, switches, and binding relationships?

  • Queue
  • FanoutExchange
  • Binding

3.5.Direct

In Fanout mode, a message will be consumed by all subscribed queues. However, in some scenarios, we want different messages to be consumed by different queues. At this time, the Direct type of Exchange is used.

image-20210717170041447

Under Direct model:

  • The binding between the queue and the switch cannot be done arbitrarily, but one must be specifiedRoutingKey (routing key)
  • The sender of the message must also specify the RoutingKey of the message when sending the message to Exchange.
  • Exchange no longer delivers the message to each bound queue, but makes judgments based on the message's Routing Key. Only the queue's Routingkey and the message Routing key are completely consistent, the message will be received

The case requirements are as follows:

  1. Use @RabbitListener to declare Exchange, Queue, and RoutingKey

  2. In the consumer service, write two consumer methods to listen to direct.queue1 and direct.queue2 respectively.

  3. Write a test method in publisher and send a message to test.direct

image-20210717170223317

3.5.1. Declaring queues and switches based on annotations

基于@Bean的方式声明队列和交换机比较麻烦,Spring还提供了基于注解方式来声明。

在consumer的SpringRabbitListener中添加两个消费者,同时基于注解来声明队列和交换机:

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "direct.queue1"),
    exchange = @Exchange(name = "test.direct", type = ExchangeTypes.DIRECT),
    key = {
    
    "red", "blue"}
))
public void listenDirectQueue1(String msg){
    
    
    System.out.println("消费者接收到direct.queue1的消息:【" + msg + "】");
}

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "direct.queue2"),
    exchange = @Exchange(name = "test.direct", type = ExchangeTypes.DIRECT),
    key = {
    
    "red", "yellow"}
))
public void listenDirectQueue2(String msg){
    
    
    System.out.println("消费者接收到direct.queue2的消息:【" + msg + "】");
}
3.5.2.消息发送

在publisher服务的SpringAmqpTest类中添加测试方法:

@Test
public void testSendDirectExchange() {
    
    
    // 交换机名称
    String exchangeName = "test.direct";
    // 消息
    String message = "红色警报!日本乱排核废水,导致海洋生物变异,惊现哥斯拉!";
    // 发送消息
    rabbitTemplate.convertAndSend(exchangeName, "red", message);
}
3.5.3.总结

描述下Direct交换机与Fanout交换机的差异?

  • Fanout交换机将消息路由给每一个与之绑定的队列
  • Direct交换机根据RoutingKey判断路由给哪个队列
  • 如果多个队列具有相同的RoutingKey,则与Fanout功能类似

基于@RabbitListener注解声明队列和交换机有哪些常见注解?

  • @Queue
  • @Exchange

3.6.Topic

3.6.1.说明

Topic类型的ExchangeDirect相比,都是可以根据RoutingKey把消息路由到不同的队列。只不过Topic类型Exchange可以让队列在绑定Routing key 的时候使用通配符!

Routingkey 一般都是有一个或多个单词组成,多个单词之间以”.”分割,例如: item.insert

通配符规则:

#:匹配一个或多个词

*:匹配不多不少恰好1个词

举例:

item.#:能够匹配item.spu.insert 或者 item.spu

item.*:只能匹配item.spu

​ 图示:

image-20210717170705380

解释:

  • Queue1:绑定的是china.# ,因此凡是以 china.开头的routing key 都会被匹配到。包括china.news和china.weather
  • Queue2:绑定的是#.news ,因此凡是以 .news结尾的 routing key 都会被匹配。包括china.news和japan.news

案例需求:

实现思路如下:

  1. 并利用@RabbitListener声明Exchange、Queue、RoutingKey

  2. 在consumer服务中,编写两个消费者方法,分别监听topic.queue1和topic.queue2

  3. 在publisher中编写测试方法,向test. topic发送消息

image-20210717170829229

3.6.2.消息发送

在publisher服务的SpringAmqpTest类中添加测试方法:

/**
     * topicExchange
     */
@Test
public void testSendTopicExchange() {
    
    
    // 交换机名称
    String exchangeName = "test.topic";
    // 消息
    String message = "喜报!孙悟空大战哥斯拉,胜!";
    // 发送消息
    rabbitTemplate.convertAndSend(exchangeName, "china.news", message);
}
3.6.3.消息接收

在consumer服务的SpringRabbitListener中添加方法:

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "topic.queue1"),
    exchange = @Exchange(name = "test.topic", type = ExchangeTypes.TOPIC),
    key = "china.#"
))
public void listenTopicQueue1(String msg){
    
    
    System.out.println("消费者接收到topic.queue1的消息:【" + msg + "】");
}

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "topic.queue2"),
    exchange = @Exchange(name = "test.topic", type = ExchangeTypes.TOPIC),
    key = "#.news"
))
public void listenTopicQueue2(String msg){
    
    
    System.out.println("消费者接收到topic.queue2的消息:【" + msg + "】");
}
3.6.4.总结

描述下Direct交换机与Topic交换机的差异?

  • Topic交换机接收的消息RoutingKey必须是多个单词,以 **.** 分割
  • Topic交换机与队列绑定时的bindingKey可以指定通配符
  • #:代表0个或多个词
  • *:代表1个词

3.7.消息转换器

之前说过,Spring会把你发送的消息序列化为字节发送给MQ,接收消息的时候,还会把字节反序列化为Java对象。

image-20200525170410401

只不过,默认情况下Spring采用的序列化方式是JDK序列化。众所周知,JDK序列化存在下列问题:

  • 数据体积过大
  • 有安全漏洞
  • 可读性差

我们来测试一下。

3.7.1.测试默认转换器

我们修改消息发送的代码,发送一个Map对象:

@Test
public void testSendMap() throws InterruptedException {
    
    
    // 准备消息
    Map<String,Object> msg = new HashMap<>();
    msg.put("name", "Jack");
    msg.put("age", 21);
    // 发送消息
    rabbitTemplate.convertAndSend("simple.queue","", msg);
}

停止consumer服务

发送消息后查看控制台:

image-20210422232835363

3.7.2.配置JSON转换器

显然,JDK序列化方式并不合适。我们希望消息体的体积更小、可读性更高,因此可以使用JSON方式来做序列化和反序列化。

在publisher和consumer两个服务中都引入依赖:

<dependency>
    <groupId>com.fasterxml.jackson.dataformat</groupId>
    <artifactId>jackson-dataformat-xml</artifactId>
    <version>2.9.10</version>
</dependency>

配置消息转换器。

在启动类中添加一个Bean即可:

@Bean
public MessageConverter jsonMessageConverter(){
    
    
    return new Jackson2JsonMessageConverter();
}

RabbitMQ-高级

消息队列在使用过程中,面临着很多实际问题需要思考:

image-20210718155003157

1.消息可靠性

消息从发送,到消费者接收,会经理多个过程:

image-20210718155059371

其中的每一步都可能导致消息丢失,常见的丢失原因包括:

  • 发送时丢失:
    • 生产者发送的消息未送达exchange
    • 消息到达exchange后未到达queue
  • MQ宕机,queue将消息丢失
  • consumer接收到消息后未消费就宕机

针对这些问题,RabbitMQ分别给出了解决方案:

  • 生产者确认机制
  • mq持久化
  • 消费者确认机制
  • 失败重试机制

下面我们就通过案例来演示每一个步骤。

首先,新建demo工程:mq-advanced-demo

├─consumer
│  └─src
│      ├─main
│      │  ├─java
│      │  └─resources
│      └─test
│          └─java
├─publisher
│  └─src
│      ├─main
│      │  ├─java
│      │  └─resources
│      └─test
│          └─java
└─src
    ├─main
    │  ├─java
    │  └─resources
    └─test
        └─java

1.1.生产者消息确认

RabbitMQ提供了publisher confirm机制来避免消息发送到MQ过程中丢失。这种机制必须给每个消息指定一个唯一ID。消息发送到MQ以后,会返回一个结果给发送者,表示消息是否处理成功。

返回结果有两种方式:

  • publisher-confirm,发送者确认
    • 消息成功投递到交换机,返回ack(acknowledge)
    • 消息未投递到交换机,返回nack
  • publisher-return,发送者回执
    • 消息投递到交换机了,但是没有路由到队列。返回ACK,及路由失败原因。

image-20210718160907166

注意:

image-20210718161707992

1.1.1.修改配置

首先,修改publisher服务中的application.yml文件,添加下面的内容:

spring:
  rabbitmq:
    publisher-confirm-type: correlated
    publisher-returns: true
    template:
      mandatory: true
   

说明:

  • publish-confirm-type:开启publisher-confirm,这里支持两种类型:
    • simple:同步等待confirm结果,直到超时
    • correlated:异步回调,定义ConfirmCallback,MQ返回结果时会回调这个ConfirmCallback
  • publish-returns:开启publish-return功能,同样是基于callback机制,不过是定义ReturnCallback
  • template.mandatory:定义消息路由失败时的策略。true,则调用ReturnCallback;false:则直接丢弃消息
1.1.2.定义Return回调

每个RabbitTemplate只能配置一个ReturnCallback,因此需要在项目加载时配置:

修改publisher服务,添加一个:

package com.example.mq.config;

import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.BeansException;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.context.annotation.Configuration;

@Slf4j
@Configuration
public class CommonConfig implements ApplicationContextAware {
    
    
    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
    
    
        // 获取RabbitTemplate
        RabbitTemplate rabbitTemplate = applicationContext.getBean(RabbitTemplate.class);
        // 设置ReturnCallback
        rabbitTemplate.setReturnCallback((message, replyCode, replyText, exchange, routingKey) -> {
    
    
            // 投递失败,记录日志
            log.info("消息发送失败,应答码{},原因{},交换机{},路由键{},消息{}",
                     replyCode, replyText, exchange, routingKey, message.toString());
            // 如果有业务需要,可以重发消息
        });
    }
}
1.1.3.定义ConfirmCallback

ConfirmCallback可以在发送消息时指定,因为每个业务处理confirm成功或失败的逻辑不一定相同。

在publisher服务的com.example.mq.spring.SpringAmqpTest类中,定义一个单元测试方法:

public void testSendMessage2SimpleQueue() throws InterruptedException {
    
    
    // 1.消息体
    String message = "hello, spring amqp!";
    // 2.全局唯一的消息ID,需要封装到CorrelationData中
    CorrelationData correlationData = new CorrelationData(UUID.randomUUID().toString());
    // 3.添加callback
    correlationData.getFuture().addCallback(
        result -> {
    
    
            if(result.isAck()){
    
    
                // 3.1.ack,消息成功,消息到交换机
                log.debug("消息发送成功, ID:{}", correlationData.getId());
            }else{
    
    
                // 3.2.nack,消息失败,消息未到交换机
                log.error("消息发送失败, ID:{}, 原因{}",correlationData.getId(), result.getReason());
            }
        },
        // 消息到交换机,但未到队列,返回ack,并报告异常
        ex -> log.error("消息发送异常, ID:{}, 原因{}",correlationData.getId(),ex.getMessage())
    );
    // 4.发送消息
    rabbitTemplate.convertAndSend("task.direct", "task", message, correlationData);

    // 休眠一会儿,等待ack回执
    Thread.sleep(2000);
}

1.2.消息持久化

生产者确认可以确保消息投递到RabbitMQ的队列中,但是消息发送到RabbitMQ以后,如果突然宕机,也可能导致消息丢失。

要想确保消息在RabbitMQ中安全保存,必须开启消息持久化机制。

  • 交换机持久化
  • 队列持久化
  • 消息持久化
1.2.1.交换机持久化

RabbitMQ中交换机默认是非持久化的,mq重启后就丢失。

SpringAMQP中可以通过代码指定交换机持久化:

@Bean
public DirectExchange simpleExchange(){
    
    
    // 三个参数:交换机名称、是否持久化、当没有queue与其绑定时是否自动删除
    return new DirectExchange("simple.direct", true, false);
}

事实上,默认情况下,由SpringAMQP声明的交换机都是持久化的。

可以在RabbitMQ控制台看到持久化的交换机都会带上D的标示:

image-20210718164412450

1.2.2.队列持久化

RabbitMQ中队列默认是非持久化的,mq重启后就丢失。

SpringAMQP中可以通过代码指定交换机持久化:

@Bean
public Queue simpleQueue(){
    
    
    // 使用QueueBuilder构建队列,durable就是持久化的
    return QueueBuilder.durable("simple.queue").build();
}

事实上,默认情况下,由SpringAMQP声明的队列都是持久化的。

可以在RabbitMQ控制台看到持久化的队列都会带上D的标示:

image-20210718164729543

1.2.3.消息持久化

利用SpringAMQP发送消息时,可以设置消息的属性(MessageProperties),指定delivery-mode:

  • 1:非持久化
  • 2:持久化

用java代码指定:

    @Test
    public void testDurableMessage() {
    
    
        // 1.准备消息
        Message message = MessageBuilder.withBody("hello, spring".getBytes(StandardCharsets.UTF_8))
                .setDeliveryMode(MessageDeliveryMode.PERSISTENT)
                .build();
        // 2.发送消息
        rabbitTemplate.convertAndSend("simple.queue", message);
        log.debug("消息发送成功");
    }

默认情况下,SpringAMQP发出的任何消息都是持久化的,不用特意指定。

1.3.消费者消息确认

RabbitMQ is aburn after reading mechanism. RabbitMQ will delete the message immediately after it confirms that it has been consumed by the consumer.

RabbitMQ uses consumer receipts to confirm whether the consumer has successfully processed the message: after the consumer obtains the message, it should send an ACK receipt to RabbitMQ to indicate that it has processed the message.

Imagine this scenario:

  • 1) RabbitMQ delivers messages to consumers
  • 2) After the consumer obtains the message, it returns ACK to RabbitMQ
  • 3) RabbitMQ deletes messages
  • 4) The consumer is down and the message has not been processed yet

In this way, the message is lost. Therefore, the timing of the consumer returning ACK is very important.

SpringAMQP allows three confirmation modes to be configured:

  • Manual: Manual ack, you need to call the API to send ack after the business code is completed.
  • auto: automatic ack. Spring monitors the listener code for exceptions. If there is no exception, ack is returned; if an exception is thrown, nack is returned.
  • none: Turn off ack. MQ assumes that the consumer will successfully process the message after getting it, so the message will be deleted immediately after delivery.

It can be seen from this:

  • In none mode, message delivery is unreliable and may be lost.
  • The auto mode is similar to the transaction mechanism. When an exception occurs, nack is returned and the message is rolled back to mq; if there is no exception, ack is returned.
  • Manual: You can judge when to ack based on the business situation.

Generally, we just use the default auto.

1.3.1. Demonstrate none mode

Modify the application.yml file of the consumer service and add the following content:

spring:
  rabbitmq:
    listener:
      simple:
        acknowledge-mode: none ## 关闭ack

Modify the method in the SpringRabbitListener class of the consumer service to simulate a message processing exception:

@RabbitListener(queues = "simple.queue")
public void listenSimpleQueue(String msg) {
    
    
    log.info("消费者接收到simple.queue的消息:【{}】", msg);
    // 模拟异常
    System.out.println(1 / 0);
    log.debug("消息处理完成!");
}

The test can find that when the message processing throws an exception, the message is still deleted by RabbitMQ.

1.3.2. Demonstrate auto mode

Change the confirmation mechanism to auto again:

spring:
  rabbitmq:
    listener:
      simple:
        acknowledge-mode: auto ## 关闭ack

Break the point at the abnormal position and send the message again. When the program is stuck at the breakpoint, you can find that the message status is unack (undetermined status):

image-20210718171705383

After the exception is thrown, because Spring will automatically return nack, the message returns to the Ready state and is not deleted by RabbitMQ:

image-20210718171759179

1.4. Retry mechanism for consumption failure

When a consumer encounters an exception, the message will continue to be requeue (requeue) to the queue, and then resent to the consumer. Then it will be abnormal again, requeue again, and the infinite loop will cause MQ's message processing to soar and bring unnecessary pressure:

image-20210718172746378

Solve the problems caused by the retry mechanism after exceptions

1.4.1. Local retry

We can use Spring's retry mechanism to use local retry when the consumer encounters an exception, instead of unlimited requeue to the mq queue.

Modify the application.yml file of the consumer service and add the following content:

spring:
  rabbitmq:
    listener:
      simple:
        retry:
          enabled: true ## 开启消费者失败重试
          initial-interval: 1000 ## 初识的失败等待时长为1秒
          multiplier: 1 ## 失败的等待时长倍数,下次等待时长 = multiplier * last-interval
          max-attempts: 3 ## 最大重试次数
          stateless: true ## true无状态;false有状态。如果业务中包含事务,这里改为false

Restart the consumer service and repeat the previous test. It can be found:

  • After retrying 3 times, SpringAMQP will throw an exception AmqpRejectAndDontRequeueException, indicating that the local retry is triggered.
  • Check the RabbitMQ console and find that the message has been deleted, indicating that SpringAMQP finally returned ack and mq deleted the message.

in conclusion:

  • When local retry is enabled, an exception is thrown during message processing and will not be requeued to the queue, but will be retried locally on the consumer.
  • After the maximum number of retries is reached, Spring will return ack and the message will be discarded.
1.4.2. Failure strategy

In the previous test, the message was discarded after reaching the maximum number of retries, which was determined by Spring's internal mechanism.

After the retry mode is enabled and the number of retries is exhausted, if the message still fails, the MessageRecovery interface needs to be used to handle it, which contains three different implementations:

  • RejectAndDontRequeueRecoverer: After the retries are exhausted, directly reject and discard the message. This is the default

  • ImmediateRequeueMessageRecoverer: After the retries are exhausted, nack is returned and the message is re-enqueued.

  • RepublishMessageRecoverer: After the retries are exhausted, the failed message is delivered to the specified switch.

A more elegant solution is RepublishMessageRecoverer. After failure, the message will be delivered to a designated queue dedicated to storing abnormal messages, and subsequent manual processing will be centralized.

1) Define the switch and queue for processing failed messages in the consumer service

@Bean
public DirectExchange errorMessageExchange(){
    
    
    return new DirectExchange("error.direct");
}
@Bean
public Queue errorQueue(){
    
    
    return new Queue("error.queue", true);
}
@Bean
public Binding errorBinding(Queue errorQueue, DirectExchange errorMessageExchange){
    
    
    return BindingBuilder.bind(errorQueue).to(errorMessageExchange).with("error");
}

2) Define a RepublishMessageRecoverer and associate the queue and switch

@Bean
public MessageRecoverer republishMessageRecoverer(RabbitTemplate rabbitTemplate){
    
    
    return new RepublishMessageRecoverer(rabbitTemplate, "error.direct", "error");
}

Complete code:

package com.example.mq.config;

import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.DirectExchange;
import org.springframework.amqp.core.Queue;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.rabbit.retry.MessageRecoverer;
import org.springframework.amqp.rabbit.retry.RepublishMessageRecoverer;
import org.springframework.context.annotation.Bean;

@Configuration
public class ErrorMessageConfig {
    
    
    @Bean
    public DirectExchange errorMessageExchange(){
    
    
        return new DirectExchange("error.direct");
    }
    @Bean
    public Queue errorQueue(){
    
    
        return new Queue("error.queue", true);
    }
    @Bean
    public Binding errorBinding(Queue errorQueue, DirectExchange errorMessageExchange){
    
    
        return BindingBuilder.bind(errorQueue).to(errorMessageExchange).with("error");
    }

    @Bean
    public MessageRecoverer republishMessageRecoverer(RabbitTemplate rabbitTemplate){
    
    
        return new RepublishMessageRecoverer(rabbitTemplate, "error.direct", "error");
    }
}

1.5. Summary

How to ensure the reliability of RabbitMQ messages?

  • Enable the producer confirmation mechanism to ensure that the producer's messages can reach the queue
  • Turn on the persistence function to ensure that messages will not be lost in the queue before being consumed.
  • Turn on the consumer confirmation mechanism to auto, and spring will complete the ack after confirming that the message processing is successful.
  • Enable the consumer failure retry mechanism and set up MessageRecoverer. After multiple retries fail, the message will be delivered to the abnormal switch for manual processing.

2. Dead letter switch

2.1. First introduction to dead letter switches

2.1.1.What is a dead letter switch?

What is a dead letter?

When a message in a queue meets one of the following conditions, it can become a dead letter:

  • The consumer uses basic.reject or basic.nack to declare consumption failure, and the requeue parameter of the message is set to false
  • The message is an expired message and no one will consume it after timeout.
  • The queue message to be delivered is full and cannot be delivered.

If the queue containing dead letters is configured with the dead-letter-exchange attribute and specifies a switch, then the dead letters in the queue will be delivered to this switch, and this switch is called < /span> (Dead Letter Exchange, check DLX). Dead Letter Exchange

As shown in the figure, a message was rejected by the consumer and became a dead letter:

image-20210718174328383

Because simple.queue is bound to the dead letter switch dl.direct, the dead letter will be delivered to this switch:

image-20210718174416160

If this dead letter exchange is also bound to a queue, the message will eventually enter the queue that stores dead letters:

image-20210718174506856

In addition, when the queue delivers the dead letter to the dead letter switch, it must know two pieces of information:

  • Dead letter switch name
  • The RoutingKey bound to the dead letter switch and the dead letter queue

This ensures that the delivered message can reach the dead-letter switch and be correctly routed to the dead-letter queue.

image-20210821073801398

2.1.2. Use the dead letter switch to receive dead letters (extension)

In the failure retry strategy, the default RejectAndDontRequeueRecoverer will send reject to RabbitMQ after the number of local retries is exhausted, and the message becomes a dead letter and is discarded.

We can add a dead letter switch to simple.queue and bind a queue to the dead letter switch. In this way, the message will not be discarded after it becomes dead letter, but will eventually be delivered to the dead letter switch and routed to the queue bound to the dead letter switch.

image-20210718174506856

We define a set of dead letter switches and dead letter queues in the consumer service:

// 声明普通的 simple.queue队列,并且为其指定死信交换机:dl.direct
@Bean
public Queue simpleQueue2(){
    
    
    return QueueBuilder.durable("simple.queue") // 指定队列名称,并持久化
        .deadLetterExchange("dl.direct") // 指定死信交换机
        .build();
}
// 声明死信交换机 dl.direct
@Bean
public DirectExchange dlExchange(){
    
    
    return new DirectExchange("dl.direct", true, false);
}
// 声明存储死信的队列 dl.queue
@Bean
public Queue dlQueue(){
    
    
    return new Queue("dl.queue", true);
}
// 将死信队列 与 死信交换机绑定
@Bean
public Binding dlBinding(){
    
    
    return BindingBuilder.bind(dlQueue()).to(dlExchange()).with("simple");
}
2.1.3. Summary

What kind of news becomes a dead letter?

  • The message is rejected by the consumer or returns nack
  • Message timed out and was not consumed
  • The queue is full

What are the usage scenarios of dead letter switches?

  • If the queue is bound to a dead letter switch, the dead letter will be delivered to the dead letter switch;
  • You can use the dead letter exchange to collect all messages that consumers failed to process (dead letters) and hand them over to manual processing to further improve the reliability of the message queue.

2.2.TTL

If a message in a queue times out and is not consumed, it will become a dead letter. There are two situations of timeout:

  • The queue where the message is located has a timeout set
  • The message itself has a timeout set

image-20210718182643311

2.2.1. Dead letter switch receiving timeout dead letter

In the SpringRabbitListener of the consumer service, define a new consumer and declare the dead letter switch and dead letter queue:

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "dl.ttl.queue", durable = "true"),
    exchange = @Exchange(name = "dl.ttl.direct"),
    key = "ttl"
))
public void listenDlQueue(String msg){
    
    
    log.info("接收到 dl.ttl.queue的超时未消费的死信消息:{}", msg);
}
2.2.2. Declare a queue and specify TTL

To set a timeout for a queue, you need to configure the x-message-ttl attribute when declaring the queue:

@Bean
public Queue ttlQueue(){
    
    
    return QueueBuilder.durable("ttl.queue") // 指定队列名称,并持久化
        .ttl(10000) // 设置队列的超时时间,10秒
        .deadLetterExchange("dl.ttl.direct") // 指定死信交换机
        .build();
}

Note that this queue has the dead letter switch set todl.ttl.direct

Declare the switch and bind ttl to the switch:

@Bean
public DirectExchange ttlExchange(){
    
    
    return new DirectExchange("ttl.direct");
}
@Bean
public Binding ttlBinding(){
    
    
    return BindingBuilder.bind(ttlQueue()).to(ttlExchange()).with("ttl");
}

Send a message, but do not specify a TTL:

@Test
public void testTTLQueue() {
    
    
    // 创建消息
    String message = "hello, ttl queue";
    // 消息ID,需要封装到CorrelationData中
    CorrelationData correlationData = new CorrelationData(UUID.randomUUID().toString());
    // 发送消息
    rabbitTemplate.convertAndSend("ttl.direct", "ttl", message, correlationData);
    // 记录日志
    log.debug("发送消息成功");
}

Log of sent messages:

09:52:05:307  INFO 6340 --- [           main] cn.test.mq.spring.SpringAmqpTest       : 消息已经成功发送!

Check the log of received messages:

09:52:15:330  INFO 7868 --- [ntContainer#0-1] c.t.mq.listener.SpringRabbitListener     : 接收到 dl.ttl.queue的延迟消息:hello, ttl queue

Because the TTL value of the queue is 10000ms, which is 10 seconds. You can see that the time difference between message sending and receiving is exactly 10 seconds.

2.2.3. When sending a message, set TTL

在发送消息时,也可以指定TTL:

@Test
public void testTTLMsg() {
    
    
    // 创建消息
    Message message = MessageBuilder
        .withBody("hello, ttl message".getBytes(StandardCharsets.UTF_8))
        .setExpiration("5000")
        .build();
    // 消息ID,需要封装到CorrelationData中
    CorrelationData correlationData = new CorrelationData(UUID.randomUUID().toString());
    // 发送消息
    rabbitTemplate.convertAndSend("ttl.direct", "ttl", message, correlationData);
    log.debug("发送消息成功");
}

查看发送消息日志:

09:56:08:820  INFO 16564 --- [           main] cn.test.mq.spring.SpringAmqpTest       : 消息已经成功发送!

接收消息日志:

09:56:13:822  INFO 7868 --- [ntContainer#0-1] c.t.mq.listener.SpringRabbitListener     : 接收到 dl.ttl.queue的延迟消息:hello, ttl messsage

这次,发送与接收的延迟只有5秒。说明当队列、消息都设置了TTL时,任意一个到期就会成为死信。

2.2.4.总结

消息超时的两种方式是?

  • 给队列设置ttl属性,进入队列后超过ttl时间的消息变为死信
  • 给消息设置ttl属性,队列接收到消息超过ttl时间后变为死信

如何实现发送一个消息20秒后消费者才收到消息?

  • 给消息的目标队列指定死信交换机
  • 将消费者监听的队列绑定到死信交换机
  • 发送消息时给消息设置超时时间为20秒

2.3.延迟队列

利用TTL结合死信交换机,我们实现了消息发出后,消费者延迟收到消息的效果。这种消息模式就称为延迟队列(Delay Queue)模式。

延迟队列的使用场景包括:

  • 延迟发送短信
  • 用户下单,如果用户在15 分钟内未支付,则自动取消
  • 预约工作会议,20分钟后自动通知所有参会人员

因为延迟队列的需求非常多,所以RabbitMQ的官方也推出了一个插件,原生支持延迟队列效果。

这个插件就是DelayExchange插件。

2.3.1.安装DelayExchange插件

官方的安装指南地址为:https://blog.rabbitmq.com/posts/2015/04/scheduling-messages-with-rabbitmq

上述文档是基于linux原生安装RabbitMQ,然后安装插件。

因为我们之前是基于Docker安装RabbitMQ,所以也使用Docker来安装RabbitMQ插件。

1).下载插件

RabbitMQ有一个官方的插件社区,地址为:https://www.rabbitmq.com/community-plugins.html

其中包含各种各样的插件,包括我们要使用的DelayExchange插件:

image-20210713104511055

大家可以去对应的GitHub页面下载3.8.9版本的插件,地址为https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/releases/tag/3.8.9这个对应RabbitMQ的3.8.5以上版本。

2).上传插件

因为我们是基于Docker安装,所以需要先查看RabbitMQ的插件目录对应的数据卷。配置插件数据卷参考安装篇(安装RabbitMQ时加 -v mq-plugins:/plugins参数即可),重新创建Docker容器。

我们之前设定的RabbitMQ的数据卷名称为mq-plugins,所以我们使用下面命令查看数据卷:

 安装示例 需要插件
 docker run \
 -e RABBITMQ_DEFAULT_USER=rabbitmq \
 -e RABBITMQ_DEFAULT_PASS=1234 \
 -v mq-plugins:/plugins \
 --name mq \
 --hostname mq \
 -p 15672:15672 \
 -p 5672:5672 \
 -d \
 rabbitmq:3.8-management

 查看mq-plugins数据卷
[root@localhost RabbitMQ] docker volume inspect mq-plugins
[
    {
        "CreatedAt": "2023-04-06T18:19:38+08:00",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/mq-plugins/_data",
        "Name": "mq-plugins",
        "Options": null,
        "Scope": "local"
    }
]

接下来,将下载的插件上传到这个目录即可:这里使用secureCRT

sftp> cd /var/lib/docker/volumes/mq-plugins/_data
sftp> pwd
/var/lib/docker/volumes/mq-plugins/_data
sftp> put e:/tmp/rabbitmq_delayed_message_exchange-3.8.9-0199d11c.ez
Uploading rabbitmq_delayed_message_exchange-3.8.9-0199d11c.ez to /var/lib/docker/volumes/mq-plugins/_data/rabbitmq_delayed_message_exchange-3.8.9-0199d11c.ez
  100% 49KB     49KB/s 00:00:00     
e:\tmp\rabbitmq_delayed_message_exchange-3.8.9-0199d11c.ez: 51047 bytes transferred in 0 seconds (49 KB/s)
3).安装插件

最后就是安装了,需要进入MQ容器内部来执行安装。我的容器名为mq,所以执行下面命令:

docker exec -it mq bash

执行时,请将其中的 -it 后面的mq替换为你自己的容器名.

进入容器内部后,执行下面命令开启插件:

 启用插件
root@mq:/ rabbitmq-plugins enable rabbitmq_delayed_message_exchange
Enabling plugins on node rabbit@mq:
rabbitmq_delayed_message_exchange
The following plugins have been configured:
  rabbitmq_delayed_message_exchange
  rabbitmq_management
  rabbitmq_management_agent
  rabbitmq_prometheus
  rabbitmq_web_dispatch
Applying plugin configuration to rabbit@mq...
The following plugins have been enabled:
  rabbitmq_delayed_message_exchange

started 1 plugins.
2.3.2.DelayExchange原理

DelayExchange需要将一个交换机声明为delayed类型。当我们发送消息到delayExchange时,流程如下:

  • 接收消息
  • 判断消息是否具备x-delay属性
  • 如果有x-delay属性,说明是延迟消息,持久化到硬盘,读取x-delay值,作为延迟时间
  • 返回routing not found结果给消息发送者
  • x-delay时间到期后,重新投递消息到指定队列
2.3.3.使用DelayExchange

插件的使用也非常简单:声明一个交换机,交换机的类型可以是任意类型,只需要设定delayed属性为true即可,然后声明队列与其绑定即可。

1)声明DelayExchange交换机

基于注解方式(推荐):

    @RabbitListener(bindings = @QueueBinding(
            value = @Queue(name = "delay.queue", durable = "true"),
            exchange = @Exchange(name = "delay.direct", delayed = "true"),
            key = "delay"
    ))
    public void listenDelayExchange(String msg) {
    
    
        log.info("消费者接收到了delay.queue的延迟消息");
    }

也可以基于@Bean的方式:

	@Bean
    public DirectExchange delayedExchange(){
    
    
        return ExchangeBuilder.directExchange("delay.direct")
                .delayed() //设置delay属性为true
                .durable(true) //持久化
                .build();
    }

    @Bean
    public Queue delayQueue(){
    
    
        return QueueBuilder.durable("delay.queue").build();
    }

    @Bean
    public Binding ttlBinding(){
    
    
        return BindingBuilder.bind(delayQueue()).to(delayedExchange()).with("delay");
    }
2)发送消息

发送消息时,一定要携带x-delay属性,指定延迟的时间:

    @Test
    public void testSendDelayMessage() throws InterruptedException {
    
    
        // 1.准备消息
        Message message = MessageBuilder
                .withBody("hello, ttl messsage".getBytes(StandardCharsets.UTF_8))
                .setDeliveryMode(MessageDeliveryMode.PERSISTENT)
                .setHeader("x-delay", 5000)
                .build();
        // 2.准备CorrelationData
        CorrelationData correlationData = new CorrelationData(UUID.randomUUID().toString());
        // 3.发送消息
        rabbitTemplate.convertAndSend("delay.direct", "delay", message, correlationData);

        log.info("发送消息成功");
    }

执行结果

发送结果:
10:52:51:312  INFO 16632 --- [           main] cn.test.mq.spring.SpringAmqpTest       : 发送消息成功
接收结果:
10:52:56:337  INFO 13208 --- [ntContainer#0-1] c.t.mq.listener.SpringRabbitListener     : 消费者接收到了delay.queue的延迟消息
2.3.4.总结

延迟队列插件的使用步骤包括哪些?

  • 声明一个交换机,添加delayed属性为true
  • 发送消息时,添加x-delay头,值为超时时间

3.惰性队列

3.1.消息堆积问题

当生产者发送消息的速度超过了消费者处理消息的速度,就会导致队列中的消息堆积,直到队列存储消息达到上限。之后发送的消息就会成为死信,可能会被丢弃,这就是消息堆积问题。

image-20210718194040498

解决消息堆积有两种思路:

  • 增加更多消费者,提高消费速度。也就是我们之前说的work queue模式
  • 扩大队列容积,提高堆积上限

要提升队列容积,把消息保存在内存中显然是不行的。

3.2.惰性队列

从RabbitMQ的3.6.0版本开始,就增加了Lazy Queues的概念,也就是惰性队列。惰性队列的特征如下:

  • 接收到消息后直接存入磁盘而非内存
  • 消费者要消费消息时才会从磁盘中读取并加载到内存
  • 支持数百万条的消息存储
3.2.1.基于命令行设置lazy-queue

而要设置一个队列为惰性队列,只需要在声明队列时,指定x-queue-mode属性为lazy即可。可以通过命令行将一个运行中的队列修改为惰性队列:

 rabbitmqctl set_policy Lazy "^lazy-queue$" '{"queue-mode":"lazy"}' --apply-to queues  
root@mq1:/ rabbitmqctl set_policy Lazy "^lazy-queue$" '{"queue-mode":"lazy"}' --apply-to queues  
Setting policy "Lazy" for pattern "^lazy-queue$" to "{"queue-mode":"lazy"}" with priority "0" for vhost "/" ...

命令解读:

  • rabbitmqctl :RabbitMQ的命令行工具
  • set_policy :添加一个策略
  • Lazy :策略名称,可以自定义
  • "^lazy-queue$" :用正则表达式匹配队列的名字
  • '{"queue-mode":"lazy"}' :设置队列模式为lazy模式
  • --apply-to queues :策略的作用对象,是所有的队列
3.2.2.基于@Bean声明lazy-queue
    @Bean
    public Queue lazyQueue() {
    
    
        return QueueBuilder.durable("lazy.queue")
                .lazy() //开启x-queue-mode为lazy
                .build();
    }
3.2.3.基于@RabbitListener声明LazyQueue
    @RabbitListener(queuesToDeclare =@Queue(
            name = "lazy.queue",
            durable = "true",
            arguments = @Argument(name = "x-queue-mode",value = "lazy")
    ))
    public void listenLazyExchange(String msg) {
    
    
        log.info("消费者接收到了lazy.queue的消息:{}",msg);
    }
3.3.总结

消息堆积问题的解决方案?

  • 队列上绑定多个消费者,提高消费速度
  • 使用惰性队列,可以再mq中保存更多消息

惰性队列的优点有哪些?

  • 基于磁盘存储,消息上限高
  • 没有间歇性的page-out,性能比较稳定

惰性队列的缺点有哪些?

  • 基于磁盘存储,消息时效性会降低
  • 性能受限于磁盘的IO

4.MQ集群

集群部署

接下来,我们看看如何安装RabbitMQ的集群。

1.集群分类

在RabbitMQ的官方文档中,讲述了两种集群的配置方式:

  • 普通模式:普通模式集群不进行数据同步,每个MQ都有自己的队列、数据信息(其它元数据信息如交换机等会同步)。例如我们有2个MQ:mq1,和mq2,如果你的消息在mq1,而你连接到了mq2,那么mq2会去mq1拉取消息,然后返回给你。如果mq1宕机,消息就会丢失。
  • 镜像模式:与普通模式不同,队列会在各个mq的镜像节点之间同步,因此你连接到任何一个镜像节点,均可获取到消息。而且如果一个节点宕机,并不会导致数据丢失。不过,这种方式增加了数据同步的带宽消耗。

我们先来看普通模式集群。

设置网络

首先,我们需要让3台MQ互相知道对方的存在。

分别在3台机器中,设置 /etc/hosts文件,添加如下内容:

 改为自己虚拟主机IP
192.168.48.101 mq1
192.168.48.102 mq2
192.168.48.103 mq3

并在每台机器上测试,是否可以ping通对方:

这里为了方便,使用同一台虚拟机,在Docker中安装,Docker使用参考:Docker使用详解

假定已安装过单机Rabbitmq

  单机Rabbitmq安装示例
 docker run \
 -e RABBITMQ_DEFAULT_USER=rabbitmq \
 -e RABBITMQ_DEFAULT_PASS=1234 \
 -v mq-plugins:/plugins \
 --name mq \
 --hostname mq \
 -p 15672:15672 \
 -p 5672:5672 \
 -d \
 rabbitmq:3.8-management

MQ的基本结构:

image-20210717162752376

RabbitMQ中的一些角色:

  • publisher:生产者
  • consumer:消费者
  • exchange个:交换机,负责消息路由
  • queue:队列,存储消息
  • virtualHost:虚拟主机,隔离不同租户的exchange、queue、消息的隔离,默认/
2.获取cookie

The bottom layer of RabbitMQ relies on Erlang, and the Erlang virtual machine is a distributed-oriented language that supports cluster mode by default. Each RabbitMQ node in cluster mode uses cookies to determine whether they are allowed to communicate with each other.

For two nodes to be able to communicate, they must have the same shared secret, called an Erlang cookie. A cookie is simply a string of alphanumeric characters of up to 255 characters.

Each cluster node must havethe same cookie. It is also required between instances to communicate with each other.

We first obtain a cookie value in the previously started mq container as a cluster cookie. Execute the following command:

 运行mq容器
[root@localhost ~] docker start mq
mq

 获取运行mq的cookie值
[root@localhost ~] docker exec -it mq cat /var/lib/rabbitmq/.erlang.cookie
ACAFGDJNGQVZXBJWBZHP[root@localhost cloud-demo]#

You can see that the cookie value is as follows:

ACAFGDJNGQVZXBJWBZHP

Next, stop and delete the current mq container, and we rebuild the cluster.

 停止mq容器
[root@localhost ~] docker stop mq
mq
docker rm -f mq
3. Prepare cluster configuration

Create a new configuration file rabbitmq.conf in the /tmp directory:

cd /tmp
### 创建文件
touch rabbitmq.conf

The contents of the file are as follows:

loopback_users.guest = false
listeners.tcp.default = 5672
cluster_formation.peer_discovery_backend = rabbit_peer_discovery_classic_config
cluster_formation.classic_config.nodes.1 = rabbit@mq1
cluster_formation.classic_config.nodes.2 = rabbit@mq2
cluster_formation.classic_config.nodes.3 = rabbit@mq3

Create another file to record cookies

cd /tmp
### 创建cookie文件
touch .erlang.cookie
### 写入cookie
echo "ACAFGDJNGQVZXBJWBZHP" > .erlang.cookie
### 修改cookie文件的权限
chmod 600 .erlang.cookie

Prepare three directories, mq1, mq2, mq3:

cd /tmp
### 创建目录
mkdir mq1 mq2 mq3

Then copy rabbitmq.conf and cookie files to mq1, mq2, mq3:

### 进入/tmp
cd /tmp
### 拷贝
cp rabbitmq.conf mq1
cp rabbitmq.conf mq2
cp rabbitmq.conf mq3
cp .erlang.cookie mq1
cp .erlang.cookie mq2
cp .erlang.cookie mq3
4. Start the cluster

Create a network:

docker network create mq-net

docker volume create

Run command

The cluster rabbitmq version must be above 3.8

docker run -d --net mq-net \
-v ${PWD}/mq1/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf \
-v ${PWD}/.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie \
-e RABBITMQ_DEFAULT_USER=test \
-e RABBITMQ_DEFAULT_PASS=123321 \
--name mq1 \
--hostname mq1 \
-p 8071:5672 \
-p 8081:15672 \
rabbitmq:3.8-management
docker run -d --net mq-net \
-v ${PWD}/mq2/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf \
-v ${PWD}/.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie \
-e RABBITMQ_DEFAULT_USER=test \
-e RABBITMQ_DEFAULT_PASS=123321 \
--name mq2 \
--hostname mq2 \
-p 8072:5672 \
-p 8082:15672 \
rabbitmq:3.8-management
docker run -d --net mq-net \
-v ${PWD}/mq3/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf \
-v ${PWD}/.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie \
-e RABBITMQ_DEFAULT_USER=test \
-e RABBITMQ_DEFAULT_PASS=123321 \
--name mq3 \
--hostname mq3 \
-p 8073:5672 \
-p 8083:15672 \
rabbitmq:3.8-management
5. Test

Add a queue on the mq1 node:RabbitMQ Management

image-20210717222833196

As shown in the figure, you can also see it in both mq2 and mq3 consoles:

image-20210717223057902

5.1. Data sharing test

Click this queue to enter the management page:

image-20210717223421750

Then use the console to send a message to this queue:

image-20210717223320238

As a result, this message can be seen on mq2 and mq3:

image-20210717223603628

5.2. Usability testing

We let one of the nodes mq1 go down:

docker stop mq1

Then log in to the mq2 or mq3 console and find that simple.queue is also unavailable:

image-20210717223800203

It means that the data is not copied to mq2 and mq3.

4.1.Cluster classification

RabbitMQ is written based on the Erlang language, and Erlang is a concurrency-oriented language that naturally supports cluster mode. RabbitMQ clusters have two modes:

Ordinary cluster: It is a distributed cluster that disperses the queues to various nodes in the cluster, thereby improving the concurrency capability of the entire cluster. .

Mirror cluster: It is a master-slave cluster. Based on the ordinary cluster, the master-slave backup function is added to improve the cluster's performance. Data availability.

Although the mirror cluster supports master-slave, master-slave synchronization is not strongly consistent, and there may be a risk of data loss in some cases. Therefore, after version 3.8 of RabbitMQ, new features were introduced: Arbitration Queue to replace the mirror cluster. The bottom layer uses the Raft protocol to ensure master-slave data consistency. .

4.2. Ordinary cluster

4.2.1. Cluster structure and characteristics

Ordinary clusters, or classic clusters, have the following characteristics:

  • Some data will be shared among various nodes in the cluster, including: switch and queue meta information. Does not include messages in the queue.
  • When accessing a node in the cluster, if the queue is not on that node, it will be passed from the node where the data is to the current node and returned.
  • If the node where the queue is located is down, the messages in the queue will be lost.

The structure is as shown in the figure:

image-20210718220843323

4.2.2.Deployment

Reference: Cluster deployment chapter

4.3. Mirror cluster

4.3.1. Cluster structure and characteristics

Mirror cluster: essentially a master-slave mode, with the following characteristics:

  • The switches, queues, and messages in the queues will be synchronously backed up between the mirror nodes of each mq.
  • The node that creates the queue is called the queue'sprimary node,The other nodes backed up to are called the queue's< a i=3>Mirrornode.
  • The master node of one queue may be the mirror node of another queue
  • All operations are completed by the master node and then synchronized to the mirror node.
  • After the master goes down, the mirror node will be replaced by the new master node.

The structure is as shown in the figure:

image-20210718221039542

4.3.2.Deployment

In a normal cluster, once the host that created the queue goes down, the queue will be unavailable. Does not have high availability capabilities. If you want to solve this problem, you must use the official mirror cluster solution.

Official document address: https://www.rabbitmq.com/ha.html

4.3.2.1. Characteristics of mirror mode

By default, the queue is only saved on the node where the queue was created. In mirroring mode, the node that creates the queue is called the master node of the queue. The queue will also be copied to other nodes in the cluster, also called the master node. The queue'smirrornode.

However, different queues can be created on any node in the cluster, so the master nodes of different queues can be different. Eventhe master node of one queue may be the mirror node of another queue.

All requests sent by users to the queue, such as sending messages and message receipts, will be completed on the master node by default. If the request is received from the slave node, it will also be routed to the master node for completion. Mirror nodes only serve to back up data.

When the master node receives the ACK from the consumer, all mirrors delete the data in the node.

Summarized as follows:

  • The mirror queue structure is one master and multiple slaves (the slaves are mirrors)
  • All operations are completed by the master node and then synchronized to the mirror node.
  • After the master goes down, the mirror node will be replaced by the new master (if the master has gone down before the master-slave synchronization is completed, data loss may occur)
  • It does not have load balancing function, because all operations will be completed by the master node (but different queues can have different master nodes, which can be used to improve throughput)
4.3.2.2. Configuration of mirror mode

There are three modes for mirror mode configuration:

ha-mode ha-params Effect
exact mode exactly The number of copies of the queue count The number of queue replicas (master and mirror servers combined) in the cluster. A count of 1 means a single replica: the queue master. A count value of 2 means 2 copies: 1 queue master and 1 queue mirror. In other words: count = number of images + 1. If there are less than count nodes in the cluster, the queue will be mirrored to all nodes. If the total number of clusters is greater than count+1 and the node containing the mirror fails, a new mirror will be created on another node.
all (none) Queues are mirrored across all nodes in the cluster. The queue will be mirrored to any newly joined nodes. Mirroring to all nodes will place additional stress on all cluster nodes, including network I/O, disk I/O, and disk space usage. It is recommended to use exactly and set the number of copies to (N / 2 +1).
nodes node names Specify which nodes the queue is created to. If all the specified nodes do not exist, an exception will occur. If the specified node exists in the cluster, but is temporarily unavailable, a node will be created to the node to which the current client is connected.

Here we use the rabbitmqctl command as an example to explain the configuration syntax.

Syntax example:

exactly mode

rabbitmqctl set_policy ha-two "^two\." '{"ha-mode":"exactly","ha-params":2,"ha-sync-mode":"automatic"}'
  • rabbitmqctl set_policy: Fixed writing method
  • ha-two: Strategy name, customized
  • "^two\.": Regular expression matching the queue. Only queues that meet the naming rules will take effect. Here is any queue name starting with two.
  • '{"ha-mode":"exactly","ha-params":2,"ha-sync-mode":"automatic"}': Strategy content
    • "ha-mode":"exactly": Strategy mode, here is exactly mode, specify the number of copies
    • "ha-params":2: Strategy parameter, here is 2, which means the number of copies is 2, 1 master and 1 mirror
    • "ha-sync-mode":"automatic": Synchronization strategy, the default is manual, that is, the newly added mirror node will not synchronize old messages. If set to automatic, the newly added mirror node will synchronize all messages in the master node, which will cause additional network overhead.

all mode

rabbitmqctl set_policy ha-all "^all\." '{"ha-mode":"all"}'
  • ha-all: Strategy name, customized
  • "^all\.": Matches all queue names starting with all.
  • '{"ha-mode":"all"}': Strategy content
    • "ha-mode":"all": Strategy mode, here is all mode, that is, all nodes will be called mirror nodes

nodes mode

rabbitmqctl set_policy ha-nodes "^nodes\." '{"ha-mode":"nodes","ha-params":["rabbit@nodeA", "rabbit@nodeB"]}'
  • rabbitmqctl set_policy: Fixed writing method
  • ha-nodes: Strategy name, customized
  • "^nodes\.": Regular expression matching the queue. Only queues that meet the naming rules will take effect. Here is any queue name starting with nodes.
  • '{"ha-mode":"nodes","ha-params":["rabbit@nodeA", "rabbit@nodeB"]}': Strategy content
    • "ha-mode":"nodes": Strategy mode, here is nodes mode
    • "ha-params":["rabbit@mq1", "rabbit@mq2"]: Policy parameter, here specifies the name of the node where the replica is located
4.3.2.3.Testing

We use mirroring in exact mode because the number of cluster nodes is 3, so the number of mirrors is set to 2.

Run the following command:

docker exec -it mq1 rabbitmqctl set_policy ha-two "^two\." '{"ha-mode":"exactly","ha-params":2,"ha-sync-mode":"automatic"}'

Next, we create a new queue:

image-20210717231751411

View the queue on any mq console:

image-20210717231829505

Test data sharing

Send a message to two.queue:

image-20210717231958996

Then view the messages on any console of mq1, mq2, or mq3:

image-20210717232108584

Test high availability

Now, we let the master node mq1 of two.queue go down:

docker stop mq1

Check the cluster status:

image-20210717232257420

View queue status:

image-20210717232322646

Found that he is still healthy! And its main node was switched to rabbit@mq2

4.4. Arbitration queue

4.4.1. Cluster characteristics

Arbitration queue: Arbitration queue is a new feature only available after version 3.8. It is used to replace the mirror queue and has the following characteristics:

  • Like the mirror queue, both are in master-slave mode and support master-slave data synchronization.
  • Very simple to use, no complicated configuration
  • Master-slave synchronization is based on Raft protocol and is strongly consistent
4.4.2.Deployment

Starting from RabbitMQ version 3.8, a new arbitration queue has been introduced. It has similar functions to the mirror queue, but is more convenient to use.

4.4.2.1. Add arbitration queue

To add a queue in any console, be sure to select the queue type as Quorum.

image-20210717234329640

View the queue on any console:

image-20210717234426209

You can see the words + 2 in the arbitration queue. It means that this queue has 2 mirror nodes.

Because the default number of mirrors in the arbitration queue is 5. If your cluster has 7 nodes, then the number of mirrors must be 5; and our cluster only has 3 nodes, so the number of mirrors is 3.

4.4.2.2.Testing

You can refer to the test on the mirror cluster, the effect is the same.

4.4.2.3.Cluster expansion

Join the cluster

1) Start a new MQ container:

docker run -d --net mq-net \
-v ${PWD}/.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie \
-e RABBITMQ_DEFAULT_USER=itcast \
-e RABBITMQ_DEFAULT_PASS=123321 \
--name mq4 \
--hostname mq5 \
-p 8074:15672 \
-p 8084:15672 \
rabbitmq:3.8-management

2) Enter the container console:

docker exec -it mq4 bash

3) Stop the mq process

rabbitmqctl stop_app

4) Reset data in RabbitMQ:

rabbitmqctl reset

5) Join mq1:

rabbitmqctl join_cluster rabbit@mq1

6) Start the mq process again

rabbitmqctl start_app

image-20210718001909492

Add a copy of the arbitration queue

Let’s first check the current copy status of the queue quorum.queue and enter the mq1 container:

docker exec -it mq1 bash

Excuting an order:

rabbitmq-queues quorum_status "quorum.queue"

result:

image-20210718002118357

Now, let's let mq4 join in:

rabbitmq-queues add_member "quorum.queue" "rabbit@mq4"

result:

image-20210718002253226

Check it out again:

rabbitmq-queues quorum_status "quorum.queue"

image-20210718002342603

Check the console and find that the number of mirrors of quorum.queue has also changed from +2 to +3:

image-20210718002422365

4.4.3.Java code creates arbitration queue
@Bean
public Queue quorumQueue() {
    
    
    return QueueBuilder
        .durable("quorum.queue") // 持久化
        .quorum() // 仲裁队列
        .build();
}
4.4.4.SpringAMQP connects to MQ cluster

Note that address is used here instead of host and port.

spring:
  rabbitmq:
    addresses: 192.168.48.105:8071, 192.168.48.105:8072, 192.168.48.105:8073
    username: test
    password: 123321
    virtual-host: /

Guess you like

Origin blog.csdn.net/lxlsygxs2017/article/details/129992980