Detailed explanation of the use of RabbitMQ message middleware

RabbitMQ official website: https://www.rabbitmq.com

I. Introduction

        Message Queue (Message Queue), understood literally: first of all, it is a queue. FIFO first-in-first-out data structure - queue. A message queue is a so-called queue that stores messages . The message queue does not solve the purpose of the queue to store messages, but solves the communication problem .

        For example, taking the e-commerce order system as an example, if synchronous communication is used between services, it will not only take a long time, but also be affected by network fluctuations during the process, and a high success rate cannot be guaranteed. Therefore, use asynchronous communication methods to transform the architecture.

        Using asynchronous communication to decouple calls between modules can quickly improve system throughput. The upstream system obtains the result immediately after executing the message sending business, and multiple downstream services subscribe to the message and consume it themselves . Through the message queue, the underlying communication protocol is shielded, so that decoupling and parallel consumption can be realized.

2. Introduction to RabbitMQ

Several popular MQs on the market:

ActiveMQ，RocketMQ，Kafka，RabbitMQ。

• Language support: ActiveMQ and RocketMQ only support Java language, Kafka can support multiple languages, and RabbitMQ supports multiple languages.
• In terms of efficiency: ActiveMQ, RocketMQ, and Kafka are all at the millisecond level, while RabbitMQ is at the microsecond level.
• Message loss and message duplication problems: RabbitMQ has relatively mature solutions for message persistence and duplication problems.
• Learning cost: RabbitMQ is very simple. RabbitMQ is powered by Rabbit

RabbitMQ is developed and maintained by Rabbit Company, ultimately at Pivotal.

RabbitMQ strictly follows the AMQP protocol, an advanced message queue protocol, to help us deliver asynchronous messages between processes.

3. RabbitMQ installation (Docker installation)

1. Start the container

docker run -d -p 15672:15672 -p 5672:5672 \
    -e RABBITMQ_DEFAULT_VHOST=rabbitmq \
    -e RABBITMQ_DEFAULT_USER=admin \
    -e RABBITMQ_DEFAULT_PASS=admin \
    --hostname myRabbit --name rabbitmq \
    rabbitmq

 Parameter Description:

• -d : Indicates running the container in the background;
• -p : Map the container's ports 5672 ( pass port ) and 15672 ( backend management port ) to the host;
• -e : Specify environment variables:
  • RABBITMQ_DEFAULT_VHOST : default virtual machine name;
  • RABBITMQ_DEFAULT_USER : Default username;
  • RABBITMQ_DEFAULT_PASS : Default user password;
• --hostname   : Specify the hostname ( an important note with RabbitMQ is that it stores data based on so-called  node names  , which default to the hostname );
• --name rabbitmq  : Set the container name;
• rabbitmq  : the image name used by the container;

2. Start rabbitmq_management

docker exec -it rabbitmq rabbitmq-plugins enable rabbitmq_management

3. Access  RabbitMQ backend management

• Enter the address in the browser: http://虚拟机IP地址:15672 you can access the background management page
• The default username and password are both  admin( specify the username and password when the container is created );

 Note: If it is a cloud server, remember to open the relevant ports.

4. RabbitMQ architecture

• Publisher - Producer: publishes messages to Exchange in RabbitMQ
• Consumer - Consumer: listens to messages in the Queue in RabbitMQ
• Exchange - Switch: Establish a connection with the producer and receive messages from the producer
• Queue - Queue: Exchange will distribute messages to the specified Queue, and the Queue and consumers will interact.
• Routes - Routing: What strategy does the switch use to publish messages to the Queue?

1. Simple architecture

 2. Complete architecture diagram of RabbitMQ

3. View the graphical interface and create a Virtual Host

Virtual hosts are used to divide a rabbitmq internally into multiple hosts for use by different users without conflict.

 Create a new test user, add /test Virtual Host, and set the test user to have permissions to operate /test.

 5. Queue mode of RabbitMQ

1. RabbitMQ communication method

2.HelloWorld mode-simple queue mode 

1) Create a new Maven project to facilitate the management of producers and consumers.

2) Create a message producer (send a message)

step:

• Create a SpringBoot project named my-priduer-demo
• Introduce dependencies

<dependency>
    <groupId>com.rabbitmq</groupId>
    <artifactId>amqp-client</artifactId>
    <version>>5.10.0</version>
</dependency>

• Write producer

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.TimeoutException;

public class MyProducerDemoApplication {
    public static final String QUEUE_NAME = "my_queue";

    public static void main(String[] args) throws IOException, TimeoutException {
        // 1.连接Broker
        // 1.1 获得连接工厂
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("虚拟机地址");
        factory.setPort(5672);
        factory.setUsername("test_user");
        factory.setPassword("test");
        factory.setVirtualHost("/test"); //设置连接的虚拟主机
        factory.setHandshakeTimeout(3000000);
        // 1.2 从连接工厂获得连接对象
        Connection connection = factory.newConnection();
        // 1.3 获取chanel，用于之后发送消息的对象
        Channel channel = connection.createChannel();
        // 1.4 声明队列 (队列不存在则创建，存在则使用)
        /*
         * queue – 队列的名称 the name of the queue
         * durable – 是否开启持久化 true if we are declaring a durable queue (the queue will survive a server restart)
         * exclusive – 是否独占连接(只允许当前客户端连接) true if we are declaring an exclusive queue (restricted to this connection)
         * autoDelete – 是否自动删除(长时间空闲未使用) true if we are declaring an autodelete queue (server will delete it when no longer in use)
         * arguments – 用于封装描述队列中的其他数据 other properties (construction arguments) for the queue
         */
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        // 1.5 定义消息
        String message = "hello,rabbitmq!";
        // 1.6 发送消息
        /*
         * exchange – 交换机(Hello world模式下，一定是空串，不能为Null) the exchange to publish the message to
         * routingKey – 路由键(当exchange为空串时，路由键为队列名称) the routing key
         * immediate – 立即的 true if the 'immediate' flag is to be set. Note that the RabbitMQ server does not support this flag.
         * mandatory – 强制的 true if the 'mandatory' flag is to be set
         * props – 封装描述消息的数据 other properties for the message - routing headers etc
         * body – 消息体 the message body
         */
        channel.basicPublish("",QUEUE_NAME,null,message.getBytes(StandardCharsets.UTF_8));
        System.out.println("发送完毕！");
        // 1.7 断开连接
        channel.close();
        connection.close();

    }

}

View on client:

3) Create a message consumer

• Introduce dependencies
• Write consumer

import java.nio.charset.StandardCharsets;
import com.rabbitmq.client.*;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.TimeoutException;
public class MyConsumer {
    public static final String QUEUE_NAME = "my_queue";

    public static void main(String[] args) throws IOException, TimeoutException {
        // 1.连接Broker
        // 1.1 获得连接工厂
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("虚拟机ip");
        factory.setPort(5672);
        factory.setUsername("test_user");
        factory.setPassword("test");
        factory.setVirtualHost("/test"); //设置连接的虚拟主机
        factory.setHandshakeTimeout(3000000);
        // 1.2 从连接工厂获得连接对象
        Connection connection = factory.newConnection();
        // 1.3 获取chanel，用于之后发送消息的对象
        Channel channel = connection.createChannel();
        // 1.4 创建一个Consumer对象，来处理消息----打印消息
        Consumer consumer = new DefaultConsumer(channel) {
            @Override
            public void handleDelivery(String consumerTag,
                                       Envelope envelope,
                                       AMQP.BasicProperties properties,
                                       byte[] body) throws IOException {

                System.out.println(new String(body));

            }
        };

        // 设置消费者监听某个队列
        channel.basicConsume(QUEUE_NAME,consumer);

    }
}

• Start and view the console

• Open the management page and find that the message has not been confirmed for consumption. When we restart the consumer, we will still receive this message.

 Just set AutoAck to true when the consumer is listening to the queue.

// 设置消费者监听某个队列
channel.basicConsume(QUEUE_NAME,true,consumer);

Problems with simple queues:

        When multiple consumers consume the same queue. At this time, rabbitmq's fair scheduling mechanism is activated. Therefore, regardless of the consumer's consumption ability, each consumer can receive the same number of messages fairly, and there cannot be a situation where  the able ones work more .

Problems with manual ACK:

       Regardless of whether the consumer has finished consuming, an ACK will be sent immediately to notify the Broker that the consumption is completed, which means that the Broker will immediately push the next message to the consumer. If the consumer's consumption ability is weak, it will cause message accumulation or affect the entire message queue. Spending power.

Solution: Manual ACK

3. Work queue mode: Those who can do more work mode   

Change automatic ack to manual ack

• Consumer 1

public class MyConsumer2 {
    public static final String QUEUE_NAME = "my_work_queue";

    public static void main(String[] args) throws Exception {
        // 1.连接Broker
        // 1.1 获得连接工厂
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("虚拟机ip");
        factory.setPort(5672);
        factory.setUsername("test_user");
        factory.setPassword("test");
        factory.setVirtualHost("/test"); //设置连接的虚拟主机
        factory.setHandshakeTimeout(3000000);
        // 1.2 从连接工厂获得连接对象
        Connection connection = factory.newConnection();
        // 1.3 获取chanel，用于之后发送消息的对象
        Channel channel = connection.createChannel();
        // 声明队列
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        // 声明一次只能消费一条消息
        channel.basicQos(1);
        // 1.4 创建一个Consumer对象，来处理消息----打印消息
        Consumer consumer = new DefaultConsumer(channel) {
            @Override
            public void handleDelivery(String consumerTag,
                                       Envelope envelope,
                                       AMQP.BasicProperties properties,
                                       byte[] body) throws IOException {
                System.out.println(new String(body));
                // 手动ASC，告诉Broker这条消息已经被消费，可以被移除队列，并且不需要批量确认消费
                channel.basicAck(envelope.getDeliveryTag(),false);
            }
        };

        // 设置消费者监听某个队列,并修改ASC模式为手动
        channel.basicConsume(QUEUE_NAME,false,consumer);

    }
}

• Consumer 2

        On the basis of consumer 1, sleep the current thread for 3 seconds to reflect that consumer 2's consumption power is weaker than consumer 1

        channel.basicQos(1) : declares that only one message can be consumed at a time

        channel.basicAck(envelope.getDeliveryTag(),false)  : Manual ASC, telling the Broker that this message has been consumed and can be removed from the message queue, and no batch confirmation of consumption is required.

• Producer (send 100 messages to the queue)

public class MyProducerDemoApplication {
    public static final String QUEUE_NAME = "my_work_queue";

    public static void main(String[] args) throws IOException, TimeoutException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("虚拟机ip地址");
        factory.setPort(5672);
        factory.setUsername("test_user");
        factory.setPassword("test");
        factory.setVirtualHost("/test"); //设置连接的虚拟主机
        factory.setHandshakeTimeout(3000000);
        // 1.2 从连接工厂获得连接对象
        Connection connection = factory.newConnection();
        // 1.3 获取chanel，用于之后发送消息的对象
        Channel channel = connection.createChannel();  
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);        
        for (int i = 0;i<=99;i++) {
            // 1.5 定义消息
            String message = "hello,rabbitmq!"+i;
            channel.basicPublish("",QUEUE_NAME,null,message.getBytes(StandardCharsets.UTF_8));
        }
        System.out.println("发送完毕！");
        // 1.7 断开连接
        channel.close();
        connection.close();

    }

}

Result : Consumer 1 (with normal spending power) consumed 97 messages, and Consumer 2 (with weak spending power) consumed 3 messages, which reflects the " people who can do more work " under the Work mode.

4. Publish and subscribe model-fanout

        For the previous queue model, there was no way to solve the problem of one message being consumed by multiple consumers at the same time. So we use the publish-subscribe model to implement it.

step:

        The producer declares the exchange and sends messages to the exchange (no more messages are sent to the queue)

        -->

        The consumer declares the queue and the switch and binds the queue to the switch

• Write producer

public class MyProducer {

    // 定义交换机名称
    public static final String EXCHANGE_NAME = "my_fanout_exchange";

    public static void main(String[] args) throws IOException, TimeoutException {

        // 获取连接对象
        Connection connection = RabbitUtil.getConnection();
        // 获取channel通道
        Channel channel = connection.createChannel();
        // 1、声明交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"fanout");
        // 2、生产消息，发送给交换机
        for (int i = 0; i < 10; i++) {
            String message = "message:"+i;
            channel.basicPublish(EXCHANGE_NAME,"",null,message.getBytes(StandardCharsets.UTF_8));
        }
        System.out.println("消息已全部发送！");
    }
}

• Write consumer 1

        Key actions:

                Create queue

                Create switch

                Bind the queue to the switch

                Let consumers listen to the queue

public class MyConsumer1 {

    private static  String EXCHANGE_NAME = "my_fanout_exchange";
    private static  String QUEUE_NAME = "my_fanout_queue_1";

    public static void main(String[] args) throws Exception {
        Connection connection = RabbitUtil.getConnection();
        Channel channel = connection.createChannel();
        // 1、声明交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"fanout");
        // 2、声明队列
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        // 3、将队列与交换机进行绑定
        channel.queueBind(QUEUE_NAME,EXCHANGE_NAME,"");
        // 4、创建消费者
        Consumer consumer = new DefaultConsumer(channel){
            @Override
            public void handleDelivery(String consumerTag,
                                       Envelope envelope,
                                       AMQP.BasicProperties properties,
                                       byte[] body) throws IOException {
                // 消费消息
                System.out.println("消费者1："+new String(body));
            }
        };
        // 5、让消费者监听队列
        channel.basicConsume(QUEUE_NAME,true,consumer);
    }

}

• Write consumer 2

public class MyConsumer2 {

    private static  String EXCHANGE_NAME = "my_fanout_exchange";
    private static  String QUEUE_NAME = "my_fanout_queue_2";

    public static void main(String[] args) throws Exception {
        Connection connection = RabbitUtil.getConnection();
        Channel channel = connection.createChannel();
        // 1、声明交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"fanout");
        // 2、声明队列
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        // 3、将队列与交换机进行绑定
        channel.queueBind(QUEUE_NAME,EXCHANGE_NAME,"");
        // 4、创建消费者
        Consumer consumer = new DefaultConsumer(channel){
            @Override
            public void handleDelivery(String consumerTag,
                                       Envelope envelope,
                                       AMQP.BasicProperties properties,
                                       byte[] body) throws IOException {
                // 消费消息
                System.out.println("消费者2："+new String(body));
            }
        };
        // 5、让消费者监听队列
        channel.basicConsume(QUEUE_NAME,true,consumer);
    }

}

Start two consumers and one producer respectively, and find that both consumers have received all messages.

5.routing mode-direct 

Key actions:

Specify the routing-key          when the producer sends the message

         When the consumer declares the binding relationship between the queue and the switch, specify the routing-key

• Writing Producer

        When sending a message to the switch, specify routing-key as apple.

public class MyProducer {
    // 定义交换机名称
    public static final String EXCHANGE_NAME = "my_routing_exchange";

    public static void main(String[] args) throws IOException, TimeoutException {
        // 获取连接对象
        Connection connection = RabbitUtil.getConnection();
        // 获取channel通道
        Channel channel = connection.createChannel();
        // 1、声明路由模式的交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"direct");
        // 2、生产消息，发送给交换机
        String message = "apple-message:";
channel.basicPublish(EXCHANGE_NAME,"apple",null,message.getBytes(StandardCharsets.UTF_8));
        System.out.println("消息已全部发送！");
        // 3、关闭连接
        channel.close();
        connection.close();
    }
}

• Write consumer 1

        When binding switches and queues, specify routing-key as apple

public class MyConsumer1 {
    private static  String EXCHANGE_NAME = "my_routing_exchange";
    private static  String QUEUE_NAME = "my_routing_queue_1";
    private static  String ROUTING_KEY = "apple";

    public static void main(String[] args) throws Exception {
        Connection connection = RabbitUtil.getConnection();
        Channel channel = connection.createChannel();
        // 1、声明交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"direct");
        // 2、声明队列
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        // 3、将队列与交换机进行绑定，并指定routingKey
        channel.queueBind(QUEUE_NAME,EXCHANGE_NAME,ROUTING_KEY);
        // 4、创建消费者
        Consumer consumer = new DefaultConsumer(channel){
            @Override
            public void handleDelivery(String consumerTag,
                                       Envelope envelope,
                                       AMQP.BasicProperties properties,
                                       byte[] body) throws IOException {
                // 消费消息
                System.out.println(ROUTING_KEY+"："+new String(body));
            }
        };
        // 5、让消费者监听队列
        channel.basicConsume(QUEUE_NAME,true,consumer);
    }
}

• Write consumer 2

        When binding switches and queues, specify routing-key as banana

public class MyConsumer2 {
    private static  String EXCHANGE_NAME = "my_routing_exchange";
    private static  String QUEUE_NAME = "my_routing_queue_2";
    private static  String ROUTING_KEY = "banana";

    public static void main(String[] args) throws Exception {
        Connection connection = RabbitUtil.getConnection();
        Channel channel = connection.createChannel();
        // 1、声明交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"direct");
        // 2、声明队列
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        // 3、将队列与交换机进行绑定，并指定routingKey
        channel.queueBind(QUEUE_NAME,EXCHANGE_NAME,ROUTING_KEY);
        // 4、创建消费者
        Consumer consumer = new DefaultConsumer(channel){
            @Override
            public void handleDelivery(String consumerTag,
                                       Envelope envelope,
                                       AMQP.BasicProperties properties,
                                       byte[] body) throws IOException {
                // 消费消息
                System.out.println(ROUTING_KEY+"："+new String(body));
            }
        };
        // 5、让消费者监听队列
        channel.basicConsume(QUEUE_NAME,true,consumer);
    }
}

Start two consumers and one producer respectively, and find that only the consumer with the routing-key bound to the queue is apple receives the message, then the message is sent to the specified queue .

6.topics mode

On the basis of routing mode, wildcards are used for routing-key, which increases the matching scope and playability.

- *.orange.*

- *.*.rabbit only supports single level

- lazy.# can support multi-level routing-key 

If product.* is used in the binding relationship, then when sending a message:

• product.add ok product.del ok
• product.add.one is not ok

If product.# is used in the binding relationship, then when sending a message:

• product.add ok
• product.add.one ok 

Write producer:

        Use topic mode and set routing-key to multi-level

public class MyProducer {

    public static final String EXCHANGE_NAME = "my_topic_exchange";

    public static void main(String[] args) throws Exception {
        // 获得连接对象与通道
        Connection connection = RabbitUtil.getConnection();
        Channel channel = connection.createChannel();
        // 声明交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"topic");
        // 发送消息
        channel.basicPublish(EXCHANGE_NAME,"product.add.one",false,false,null,"hello,topic".getBytes(StandardCharsets.UTF_8));
        // 关闭连接
        channel.close();
        connection.close();
    }
}

Write consumer 1:

        Use topic mode and set routing-key to product.* (single level)

public class MyConsumer1 {
    // 交换机名称
    private static  String EXCHANGE_NAME = "my_topic_exchange";
    // 队列名称
    private static  String QUEUE_NAME = "my_topic_queue_1";
    public static void main(String[] args) throws Exception {
        Connection connection = RabbitUtil.getConnection();
        Channel channel = connection.createChannel();
        // 1、声明交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"topic");
        // 2、声明队列
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        // 3、将队列与交换机进行绑定，并指定routingKey，为 product.任意字符 都能接收
        channel.queueBind(QUEUE_NAME,EXCHANGE_NAME,"product.*");
        // 4、创建消费者
        Consumer consumer = new DefaultConsumer(channel){
            @Override
            public void handleDelivery(String consumerTag,
                                       Envelope envelope,
                                       AMQP.BasicProperties properties,
                                       byte[] body) throws IOException {
                // 消费消息
                System.out.println("product.*消费者："+"："+new String(body));
            }
        };
        // 5、让消费者监听队列
        channel.basicConsume(QUEUE_NAME,true,consumer);
    }
}

Write consumer 2:

        Use topic mode and set routing-key to product.# (multi-level)

public class MyConsumer2 {
    // 交换机名称
    private static  String EXCHANGE_NAME = "my_topic_exchange";
    // 队列名称
    private static  String QUEUE_NAME = "my_topic_queue_2";
    public static void main(String[] args) throws Exception {
        Connection connection = RabbitUtil.getConnection();
        Channel channel = connection.createChannel();
        // 1、声明交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"topic");
        // 2、声明队列
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        // 3、将队列与交换机进行绑定，并指定routingKey，为 product.任意字符 都能接收
        channel.queueBind(QUEUE_NAME,EXCHANGE_NAME,"product.#");
        // 4、创建消费者
        Consumer consumer = new DefaultConsumer(channel){
            @Override
            public void handleDelivery(String consumerTag,
                                       Envelope envelope,
                                       AMQP.BasicProperties properties,
                                       byte[] body) throws IOException {
                // 消费消息
                System.out.println("product.*消费者："+"："+new String(body));
            }
        };
        // 5、让消费者监听队列
        channel.basicConsume(QUEUE_NAME,true,consumer);
    }
}

Two consumers and one producer were started respectively, and it was found that only the consumer whose queue-bound routing-key was product.# received the message.

6. Using RabbitMQ in Springboot

1.Introduce dependencies

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-amqp</artifactId>
        </dependency>

  2. Write configuration file      

server.port=8091
spring.rabbitmq.addresses=虚拟机ip地址
spring.rabbitmq.port=5672
spring.rabbitmq.username=test_user
spring.rabbitmq.password=test
spring.rabbitmq.virtual-host=/test #虚拟主机名

3. Use publish-subscribe model

Publish and subscribe review:

        The consumer defines the switch and the queue and binds the two;

        The producer sends a message to the switch;

1) Write consumer

• Write configuration class

        Define switches and queues, and bind switches and queues

/**
 * RabbitMQ消费者配置类
 * springBoot实现消息订阅模式
 */
@Configuration
public class MyRabbitConfig {

    public static final String EXCHANGE_NAME = "my_boot_fanout_exchange";
    public static final String QUEUE_NAME = "my_boot_fanout_queue1";

    /**
     * 声明交换机
     */
    @Bean
    public FanoutExchange exchange(){
        return new FanoutExchange(EXCHANGE_NAME,true,false);
    }

    /**
     * 声明队列
     */
    @Bean
    public Queue queue(){
      return   new Queue(QUEUE_NAME,true,false,false);
    }

    /**
     * 绑定交换机与队列
     */
    @Bean
    public Binding queueBinding(Queue queue,FanoutExchange exchange){
        return BindingBuilder.bind(queue).to(exchange);
    }

}

• How to write consumption messages

        Key: Use this annotation to specify the listening queue @RabbitListener(queues = "Name of the queue to be listened to")

@Component
public class MyConsumer {

    /**
     * 监听队列：当队列中有消息，则监听器工作，处理接收到的消息
     * @param message 消息体
     */
    @RabbitListener(queues = "my_boot_fanout_queue1")
    public void process(Message message){
        byte[] messageBody = message.getBody();
        System.out.println(new String(messageBody));
    }

}

2) Write the producer

• Write configuration class

        Because the producer only needs to send messages to the switch, it only needs to declare the switch

@Configuration
public class MyProducerConfig {
    public static final String EXCHANGE_NAME = "my_boot_fanout_exchange";
    /**
     * 声明交换机
     */
    @Bean
    public FanoutExchange exchange(){
        return new FanoutExchange(EXCHANGE_NAME,true,false);
    }
}

• Use RabbitTemplate to send messages

    @Autowired
    RabbitTemplate rabbitTemplate;

    public static final String EXCHANGE_NAME = "my_boot_fanout_exchange";

    @Test
    void testSendMsg(){
        String msg = "Hello,SpringBootRabbitMQ!";
        rabbitTemplate.convertAndSend(EXCHANGE_NAME,"",msg);
        System.out.println("消息发送成功！");
    }

4. Use topic mode

        Compared with the publish-subscribe mode, the topic mode has the additional use of routing-key.

1) Adjust the consumer configuration class (routing-key must be specified)

/**
 * RabbitMQ消费者配置类
 * springBoot实现Topic模式
 */
@Configuration
public class MyRabbitTopicConfig {

    public static final String TOPIC_EXCHANGE_NAME = "my_boot_topic_exchange";
    public static final String TOPIC_QUEUE_NAME = "my_boot_topic_queue";

    /**
     * 声明交换机
     */
    @Bean
    public TopicExchange exchange(){
        return new TopicExchange(TOPIC_EXCHANGE_NAME,true,false);
    }

    /**
     * 声明队列
     */
    @Bean
    public Queue queue(){
      return   new Queue(TOPIC_QUEUE_NAME,true,false,false);
    }

    /**
     * 绑定交换机与队列
     */
    @Bean
    public Binding queueBinding(Queue queue,TopicExchange exchange){
        return BindingBuilder.bind(queue).to(exchange).with("product.*"); // 能接受到routing-key为product.任意字符的消息（*单层 #多层）
    }

}

        You also need to modify the queue that the consumer listens to to the queue name declared here.

 2) Write the producer

• Adjust the producer’s configuration class

public class MyTopicProducerConfig {
    public static final String TOPIC_EXCHANGE_NAME = "my_boot_topic_queue";

    /**
     * 声明交换机
     */
    @Bean
    public TopicExchange exchange(){
        return new TopicExchange(TOPIC_EXCHANGE_NAME,true,false);
    }
}

• Carry routing-key when sending messages

    @Autowired
    RabbitTemplate rabbitTemplate;

    public static final String TOPIC_EXCHANGE_NAME = "my_boot_fanout_exchange";

    @Test
    void testSendMsg(){
        String msg = "Hello,SpringBootRabbitMQ!";
        rabbitTemplate.convertAndSend(TOPIC_EXCHANGE_NAME,"product.add",msg); //指定routing-key
        System.out.println("消息发送成功！");
    }

5. Implementation of manual ack

• Add manual ack configuration to the configuration file

spring.rabbitmq.listener.direct.acknowledge-mode=manual

• Perform manual ack after consumption in the consumer

    @RabbitListener(queues = "my_boot_topic_queue")
    public void process(Message message, Channel channel) throws IOException {

        System.out.println("接收到的消息"+message.toString());

        // 手动ACK,告知Broker确认已被消费的消息id（DeliveryTag）
        channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);

    }

 Among them, the two key-value pairs in the request header of the message are:

• spring_listener_return_correlation : This attribute is used to determine which listener is called when a message is returned.
• spring_returned_message_correlation : This attribute refers to the unique identifier of the returned message to be confirmed.

7. Reliable delivery of messages

Three guarantees for message reliability:

        1. The producer accurately delivers the message to the switch (using the Confirm mechanism)

        2. The switch accurately delivers the message to the queue (using the Return mechanism)

        3. The queue accurately pushes the message to the consumer (consumer manual ACK)

1. Ensure that producer messages can be delivered to MQ through the confirm mechanism 

• Use confirm when sending messages to producers in spring projects

public class MyProducer {

    public static final String EXCHANGE_NAME = "my_topic_exchange";

    public static void main(String[] args) throws Exception {
        // 获得连接对象与通道
        Connection connection = RabbitUtil.getConnection();
        Channel channel = connection.createChannel();
        // 声明交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"topic");
        // 开启confirm机制
        channel.confirmSelect();
        // 设置confirm监听器
        channel.addConfirmListener(new ConfirmListener() {
            // 消息被Broker确认接收了，将会回调此方法
            @Override
            public void handleAck(long l, boolean b) throws IOException {
                // 消息发送成功
                System.out.println("消息被成功投递！");
            }
            // 消息被Broker接收失败了，将会回调此方法
            @Override
            public void handleNack(long l, boolean b) throws IOException {
                //开启重试机制，重试达到阈值，则考虑人工介入
                System.out.println("消息投递失败！");
            }
        });
        byte[] msg = "hello,confirm message".getBytes(StandardCharsets.UTF_8);
        // 发送消息
        channel.basicPublish(EXCHANGE_NAME,"product.add",false,false,null,msg);
    }
}

Use channel.confirmSelect()  to turn on the confirm mechanism before sending a message   ;

                       Use  channel.addConfirmListener  to set up the confirm listener

                                Among them,  handleAck  is the callback function of the message successfully delivered to the switch. 

                                        handleNack  is the callback function for messages that are not successfully delivered to the switch.

• Using Confirm in SpringBoot

Step 1: Modify the producer configuration:

server.port=8091
spring.rabbitmq.addresses=虚拟机ip地址
spring.rabbitmq.port=5672
spring.rabbitmq.username=test_user
spring.rabbitmq.password=test
spring.rabbitmq.virtual-host=/test
spring.rabbitmq.publisher-confirm-type: correlated

publisher-confirm-type: There are three configurations:

• Simple: Simple execution of ack judgment; after successfully publishing the message, use rabbitTemplate to call the waitForConfirms or waitForConfirmsOrDie method to wait for the broker node to return the sending result, and judge the next step logic based on the return result. But it should be noted that if the waitForConfirmsOrDie method returns false, the channel will be closed.
• Correlated: When executing ack, data (metadata of the message) will also be carried;
• none: Disable release confirmation mode, default

Step 2: Write an implementation class (listener) of ConfirmCallback and inject it into RabbitTemplate

@Component
public class MyConfirmCallfack implements RabbitTemplate.ConfirmCallback {

    @Autowired
    private RabbitTemplate rabbitTemplate;

    // 将监听器注入到RabbitTemplate中
    @PostConstruct
    public void init(){
        rabbitTemplate.setConfirmCallback(this);
    }


    /**
     * @param correlationData 消息元数据（消息id，消息内容）
     * @param ack 布尔值，Broker是否成功接收到消息
     * @param cause 投递失败的原因
     */
    @Override
    public void confirm(CorrelationData correlationData, boolean ack, String cause) {
        // 消息的id
        String id = correlationData.getId();
        if (ack){
            // 消息投递成功
            System.out.println("消息投递成功，id为："+id);
        }else{
            // 消息投递失败，可对失败的消息进行定时重试
            System.out.println("消息投递失败，原因为："+cause);
        }
    }
}

 We found that when changing the switch to a non-existent switch, we will get feedback of message failure. However, if the wrong routing-key is modified and the message is not successfully delivered, we will not receive feedback of message delivery failure. This is because Confirm will only pay attention to the message delivery situation between the producer and the switch.

2. Use the return mechanism to ensure that messages can be successfully delivered to the queue in rabbitmq

The producer delivers the message to the mq switch - guaranteed by the Confirm mechanism.

If the switch cannot deliver the message to the queue, it can retry through the Return mechanism. 

Step 1: Modify the configuration file 

If the return mechanism is turned on. Mandatory needs to be set to true.

server.port=8091
spring.rabbitmq.addresses=虚拟机IP地址
spring.rabbitmq.port=5672
spring.rabbitmq.username=test_user
spring.rabbitmq.password=test
spring.rabbitmq.virtual-host=/test
spring.rabbitmq.publisher-confirm-type: correlated
spring.rabbitmq.publisher-returns: true

Step 2: Implement the RabbitTemplate.ReturnCallback interface in the listening class

import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.connection.CorrelationData;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;

@Component
public class MyConfirmCallfack implements RabbitTemplate.ConfirmCallback,RabbitTemplate.ReturnCallback {



    @Autowired
    private RabbitTemplate rabbitTemplate;

    // 将监听器注入到RabbitTemplate中
    @PostConstruct
    public void init(){
        rabbitTemplate.setConfirmCallback(this);
        rabbitTemplate.setReturnCallback(this);
    }


    /**
     * @param correlationData 消息元数据（消息id，消息内容）
     * @param ack 布尔值，Broker是否成功接收到消息
     * @param cause 投递失败的原因
     */
    @Override
    public void confirm(CorrelationData correlationData, boolean ack, String cause) {
        // 消息的id
        String id = correlationData.getId();
        if (ack){
            // 消息投递成功
            System.out.println("消息投递成功，id为："+id);
        }else{
            // 消息投递失败，可对失败的消息进行定时重试
            System.out.println("消息投递失败，原因为："+cause);
        }
    }

    /**
        当消息未成功被投递到队列，调用此方法
     */
    @Override
    public void returnedMessage(Message message, int i, String s, String s1, String s2) {
        System.out.println("消息"+new String(message.getBody()+"没有被成功投递到队列"));
    }
}

3. The difference between manual ack, nack and reject

1) Don’t do any ack

        RabbitMQ will mark the message as unacked. At this time, MQ is waiting for the consumer to ack. If the consumer loses the session, the message will return to the ready state and be consumed by other consumers.

2）ack

        After confirming the receipt, the message will be removed from the queue .

3）reject

        Reject means rejecting this message.

        Reject only supports processing one message at a time. After the message is rejected and requeue is set to false, it will enter the dead letter queue . If requeue is set to true, it will return to the queue, but this situation is rarely used.

4）nack 

public class MyConsumer1 {
    // 交换机名称
    private static  String EXCHANGE_NAME = "my_topic_exchange";
    // 队列名称
    private static  String QUEUE_NAME = "my_topic_queue_1";
    public static void main(String[] args) throws Exception {
        Connection connection = RabbitUtil.getConnection();
        Channel channel = connection.createChannel();
        // 1、声明交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"topic");
        // 2、声明队列
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        // 3、将队列与交换机进行绑定，并指定routingKey，为 product.任意字符 都能接收
        channel.queueBind(QUEUE_NAME,EXCHANGE_NAME,"product.*");
        // 4、创建消费者
        Consumer consumer = new DefaultConsumer(channel){
            @Override
            public void handleDelivery(String consumerTag,
                                       Envelope envelope,
                                       AMQP.BasicProperties properties,
                                       byte[] body) throws IOException {
                // 消费消息
                System.out.println("product.*消费者："+"："+new String(body));
                  //⼿动ack
//                channel.basicAck(envelope.getDeliveryTag(),false);
                  //reject拒签消息 ⼀次只⽀持处理⼀条消息
//                channel.basicReject(envelope.getDeliveryTag(),false);
                //nack 拒签消息 ⽀持批处理多条消息
                channel.basicNack(envelope.getDeliveryTag(), true,false);
            }
        };
        // 5、让消费者监听队列
        channel.basicConsume(QUEUE_NAME,true,consumer);
    }
}

 5) Encapsulation of message metadata

        Before the producer sends the message, the metadata of the message can be constructed, such as whether the message is persistent, the message expiration time, the message ID and customized Map data.

Producer side: 

public class MyProducer {

    public static final String EXCHANGE_NAME = "my_topic_exchange";

    public static void main(String[] args) throws Exception {
        // 获得连接对象与通道
        Connection connection = RabbitUtil.getConnection();
        Channel channel = connection.createChannel();
        // 声明交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"topic");

        // 创建消息的元数据
        HashMap<String, Object> map = new HashMap<>();
        map.put("name","zhangsan");
        map.put("age","18");
        AMQP.BasicProperties properties = new AMQP.BasicProperties.Builder()
                .deliveryMode(2) //消息是否支持持久化：1不支持2支付
                .messageId(UUID.randomUUID().toString()) //定义消息的业务id
                .expiration("100000000") // 定义消息的过期时间
                .headers(map) // 头信息
                .build();
        // 发送消息
        channel.basicPublish(EXCHANGE_NAME,"product.#",false,false,properties,"hello,topic".getBytes(StandardCharsets.UTF_8));
        // 关闭连接
        channel.close();
        connection.close();
    }
}

Construct message metadata through new AMQP.BasicProperties.Builder

Consumer side:

public class MyConsumer1 {
    // 交换机名称
    private static  String EXCHANGE_NAME = "my_topic_exchange";
    // 队列名称
    private static  String QUEUE_NAME = "my_topic_queue_1";
    public static void main(String[] args) throws Exception {
        Connection connection = RabbitUtil.getConnection();
        Channel channel = connection.createChannel();
        // 1、声明交换机
        channel.exchangeDeclare(EXCHANGE_NAME,"topic");
        // 2、声明队列
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);
        // 3、将队列与交换机进行绑定，并指定routingKey，为 product.任意字符 都能接收
        channel.queueBind(QUEUE_NAME,EXCHANGE_NAME,"product.add");
        // 4、创建消费者
        Consumer consumer = new DefaultConsumer(channel){
            @Override
            public void handleDelivery(String consumerTag,
                                       Envelope envelope,
                                       AMQP.BasicProperties properties,
                                       byte[] body) throws IOException {
                // 消费消息
                Map<String, Object> map = properties.getHeaders();//获取消息元数据
                System.out.println(map);
                //⼿动ack
                channel.basicAck(envelope.getDeliveryTag(),false);
            }
        };
        // 5、让消费者监听队列
        channel.basicConsume(QUEUE_NAME,false,consumer);
    }
}

Get message metadata through AMQP.BasicProperties

8. The problem of repeated consumption of messages

1. What is idempotence?

        Idempotence: The results of multiple operations are consistent . How to ensure idempotence for non-idempotent operations? ——Use distributed locks.

        Use distributed locks to solve the problem of repeated consumption of messages without manual ACK due to network jitter. Idea: Generate a globally unique ID for the message. The producer will carry this ID when sending the message. After the consumer successfully consumes the message, the ID of the message will be used. Through Redis's setnx cache, when preparing for repeated consumption, it is judged from redis whether there is the ID.

9. Dead letter queue - "delay" queue 

1. Introduction to the dead letter queue 

        The dead letter queue allows a message, if certain conditions are met, to become a dead letter and be sent to another switch for consumption. This process is the role of the dead letter queue. The dead letter queue can have the effect of "delaying" the queue. For example, if the order times out and is not paid, the order status can be changed to "cancelled". This operation can be completed using the dead letter queue. Set the timeout of the message. When the message times out, the message becomes a dead letter, so the consumer who monitors the dead letter queue cancels the order. 

 There are two things to know:

• How does a message become a dead letter? Conditions for becoming a dead letter
• How to create a dead letter queue and achieve the effect of a dead letter queue

2. Conditions for a message to become a dead letter

• If the message is rejected and is not returned to the queue, the message will become a dead letter. (nack, reject and requeue are false)
• The message has expired and will become a dead letter.
• The length of the queue is limited and no messages can be stored. Messages that cannot be stored will become dead letters.

3. Create a dead letter queue 

        Key point: Just let the normal queue be bound to the dead letter switch. Note: This dead letter switch is actually a normal switch.

consumer):

        Connection connection = RabbitUtil.getConnection();
        Channel channel = connection.createChannel();

        // 声明普通交换机、普通队列 声明死信交换机、死信队列 建立他们的关系
        String normalExchangeName = "normal.exchange";
        String exchangeType = "topic";
        String normalQueueName = "normal.queue";
        String routingKey = "dlx.#";
        // 声明死信队列
        String dlxExchangeName = "dlx.exchange";
        String dlxQueueName = "dlx.queue";
        // 声明普通交换机
        channel.exchangeDeclare(normalExchangeName,exchangeType,true,false,null);

        // 为队列绑定死信交换机
        Map<String,Object> queueArgs = new HashMap<>();
        queueArgs.put("x-dead-letter-exchange",dlxExchangeName);//正常队列绑定⼀个交换机，让该交换机是死信交换机
        queueArgs.put("x-max-length",4);   //设置队列的⻓度是4
        // 声明普通队列，并将带有死信交换机的消息元数据
        channel.queueDeclare(normalQueueName,true,false,false,queueArgs);
        channel.queueBind(normalQueueName,normalExchangeName,routingKey);
        //创建死信队列
        channel.exchangeDeclare(dlxExchangeName,exchangeType,true,false,null);
        channel.queueDeclare(dlxQueueName,true,false,false,null);
        channel.queueBind(dlxQueueName,dlxExchangeName,"#");

4. Delay queue

        Create a consumer that listens to the dead letter queue. When a message enters the dead letter queue, it takes out the metadata of the message for business processing (such as canceling the order after timeout), and manually ACKs after successfully consuming the message from the dead letter queue.