[RabbitMQ] RabbitMQ quick start and easy to understand beginners’ introduction

Both methods have their own pros and cons. You can get an immediate response when you call, but you can't talk to multiple people at the same time. Sending emails allows you to send and receive emails to multiple people at the same time, but there is often a delay in response.

1.1.1. Synchronous communication

The Feign call we learned before is a synchronous method. Although the call can get results in real time, there are the following problems:

Summarize:

Advantages of synchronous calls:

Strong timeliness, results can be obtained immediately

Problems with synchronous calls:

High degree of coupling
Reduced performance and throughput
There is additional resource consumption
There is a cascading failure problem

1.1.2.Asynchronous communication

Asynchronous calls can avoid the above problems:

Let's take the purchase of goods as an example. After paying, the user needs to call the order service to complete the order status modification, call the logistics service, allocate the corresponding inventory from the warehouse and prepare for shipment.

In the event model, the payment service is the event publisher (publisher). After the payment is completed, it only needs to publish a successful payment event (event) with the order id in the event.

The order service and logistics service are event subscribers (Consumers). They subscribe to the event of successful payment and complete their own business after listening to the event.

In order to decouple the event publisher and subscriber, the two do not communicate directly, but there is an intermediary (Broker). The publisher publishes events to the Broker and does not care who subscribes to the event. Subscribers subscribe to events from the Broker and do not care who sends the message.

Broker is something like a data bus. All services that need to receive and send data are sent to this bus. This bus is like a protocol, making communication between services standard and controllable.

benefit:

Throughput improvement: no need to wait for subscriber processing to complete, faster response
Fault isolation: the service is not directly called, and there is no cascading failure problem
There is no blocking between calls and no invalid resource usage.
The degree of coupling is extremely low, and each service can be flexibly plugged in and replaced.
Traffic peak shaving: No matter how much the traffic of published events fluctuates, it will be received by the Broker, and subscribers can process events at their own speed.

shortcoming:

The structure is complex, the business has no obvious process lines, and it is difficult to manage.
Need to rely on the reliability, security and performance of the Broker

Fortunately, the Broker software on open source software or cloud platforms is very mature. One of the more common ones is the MQ technology we are going to learn today.

1.2.Technical comparison:

MQ, Chinese is Message Queue (MessageQueue), which literally means a queue for storing messages. That is the Broker in the event-driven architecture.

More common MQ implementations:

ActiveMQ
RabbitMQ
RocketMQ
Kafka

Comparison of several common MQs:

	RabbitMQ	ActiveMQ	RocketMQ	Kafka
Company/Community	Rabbit	Apache	Ali	Apache
Development language	Erlang	Java	Java	Scala&Java
Protocol support	AMQP，XMPP，SMTP，STOMP	OpenWire,STOMP，REST,XMPP,AMQP	Custom protocol	Custom protocol
Availability	high	generally	high	high
Single machine throughput	generally	Difference	high	very high
message delay	Microsecond level	millisecond level	millisecond level	Within milliseconds
message reliability	high	generally	high	generally

Pursuing availability: Kafka, RocketMQ, RabbitMQ

Pursuing reliability: RabbitMQ, RocketMQ

Pursuing throughput capacity: RocketMQ, Kafka

Pursuing low message latency: RabbitMQ, Kafka

2. Quick Start

2.1. Install RabbitMQ

To install RabbitMQ, refer to the pre-course materials:

The basic structure of MQ:

Some roles in RabbitMQ:

publisher: producer
consumer：consumer
exchange: switch, responsible for message routing
queue: Queue, stores messages
virtualHost: virtual host, isolates the exchange, queue, and messages of different tenants

2.2.RabbitMQ message model

RabbitMQ officially provides 5 different Demo examples, corresponding to different message models:

2.3.Import Demo project

The pre-course material provides a Demo project, mq-demo:

After importing, you can see the structure as follows:

Includes three parts:

mq-demo: parent project, manages project dependencies
publisher: the sender of the message
consumer: the consumer of the message

2.4. Getting Started Case

Model diagram of simple queue mode:

The official HelloWorld is implemented based on the most basic message queue model and only includes three roles:

publisher: message publisher, sends messages to queue queue
queue: message queue, responsible for accepting and caching messages
consumer: subscribe to the queue and process messages in the queue

2.4.1.publisher implementation

Idea:

establish connection
CreateChannel
declare queue
Send a message
Close connection and channel

Code:

package cn.itcast.mq.helloworld;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import org.junit.Test;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

public class PublisherTest {
    @Test
    public void testSendMessage() throws IOException, TimeoutException {
        // 1.建立连接
        ConnectionFactory factory = new ConnectionFactory();
        // 1.1.设置连接参数，分别是：主机名、端口号、vhost、用户名、密码
        factory.setHost("192.168.150.101");
        factory.setPort(5672);
        factory.setVirtualHost("/");
        factory.setUsername("itcast");
        factory.setPassword("123321");
        // 1.2.建立连接
        Connection connection = factory.newConnection();

        // 2.创建通道Channel
        Channel channel = connection.createChannel();

        // 3.创建队列
        String queueName = "simple.queue";
        channel.queueDeclare(queueName, false, false, false, null);

        // 4.发送消息
        String message = "hello, rabbitmq!";
        channel.basicPublish("", queueName, null, message.getBytes());
        System.out.println("发送消息成功：【" + message + "】");

        // 5.关闭通道和连接
        channel.close();
        connection.close();

    }
}

2.4.2.consumer implementation

Code idea:

establish connection
CreateChannel
declare queue
Subscribe to news

Code:

package cn.itcast.mq.helloworld;

import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

public class ConsumerTest {

    public static void main(String[] args) throws IOException, TimeoutException {
        // 1.建立连接
        ConnectionFactory factory = new ConnectionFactory();
        // 1.1.设置连接参数，分别是：主机名、端口号、vhost、用户名、密码
        factory.setHost("192.168.150.101");
        factory.setPort(5672);
        factory.setVirtualHost("/");
        factory.setUsername("itcast");
        factory.setPassword("123321");
        // 1.2.建立连接
        Connection connection = factory.newConnection();

        // 2.创建通道Channel
        Channel channel = connection.createChannel();

        // 3.创建队列
        String queueName = "simple.queue";
        channel.queueDeclare(queueName, false, false, false, null);

        // 4.订阅消息
        channel.basicConsume(queueName, true, new DefaultConsumer(channel){
            @Override
            public void handleDelivery(String consumerTag, Envelope envelope,
                                       AMQP.BasicProperties properties, byte[] body) throws IOException {
                // 5.处理消息
                String message = new String(body);
                System.out.println("接收到消息：【" + message + "】");
            }
        });
        System.out.println("等待接收消息。。。。");
    }
}

2.5. Summary

The message sending process of the basic message queue:

Establish connection
create channel
Use channel to declare queue
Use channel to send messages to queue

The message receiving process of the basic message queue:

Establish connection
create channel
Use channel to declare queue
Define the consumer's consumption behavior handleDelivery()
Use channels to bind consumers to queues

3.SpringAMQP

SpringAMQP is a set of templates based on RabbitMQ encapsulation, and it also uses SpringBoot to implement automatic assembly, which is very convenient to use.

The official address of SpringAmqp: Spring AMQP

SpringAMQP provides three functions:

Automatically declare queues, switches and their binding relationships
Annotation-based listener mode, asynchronously receiving messages
Encapsulates the RabbitTemplate tool for sending messages

3.1.Basic Queue simple queue model

Introduce dependencies into the parent project mq-demo

<!--AMQP依赖，包含RabbitMQ-->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>

3.1.1.Message sending

First configure the MQ address and add the configuration in application.yml of the publisher service:

spring:
  rabbitmq:
    host: 192.168.150.101 # Host name
    port: 5672 #Port
    virtual-host: / # virtual host
    username: itcast # username
    password: 123321 # Password

Then write the test class SpringAmqpTest in the publisher service and use RabbitTemplate to send messages:

package cn.itcast.mq.spring;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest
public class SpringAmqpTest {

    @Autowired
    private RabbitTemplate rabbitTemplate;

    @Test
    public void testSimpleQueue() {
        // 队列名称
        String queueName = "simple.queue";
        // 消息
        String message = "hello, spring amqp!";
        // 发送消息
        rabbitTemplate.convertAndSend(queueName, message);
    }
}

3.1.2. Message reception

First configure the MQ address and add the configuration in application.yml of the consumer service:

spring:
  rabbitmq:
    host: 192.168.150.101 # Host name
    port: 5672 #Port
    virtual-host: / # virtual host
    username: itcast # username
    password: 123321 # Password

cn.itcast.mq.listenerThen create a new class SpringRabbitListener in the consumer service package, the code is as follows:

package cn.itcast.mq.listener;

import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;

@Component
public class SpringRabbitListener {

    @RabbitListener(queues = "simple.queue")
    public void listenSimpleQueueMessage(String msg) throws InterruptedException {
        System.out.println("spring 消费者接收到消息：【" + msg + "】");
    }
}

3.1.3.Testing

Start the consumer service, then run the test code in the publisher service and send MQ messages

3.2.WorkQueue

Work queues, also known as (Task queues), task model. To put it simply, multiple consumers are bound to a queue and jointly consume messages in the queue .

When message processing is time-consuming, the speed of message production may be much greater than the speed of message consumption. If things continue like this, more and more messages will accumulate and cannot be processed in a timely manner.

At this time, the work model can be used, and multiple consumers can handle message processing together, and the speed can be greatly improved.

3.2.1.Message sending

This time we send it in a loop to simulate the accumulation of a large number of messages.

Add a test method in the SpringAmqpTest class in the publisher service:

/**
     * workQueue
     * 向队列中不停发送消息，模拟消息堆积。
     */
@Test
public void testWorkQueue() throws InterruptedException {
    // 队列名称
    String queueName = "simple.queue";
    // 消息
    String message = "hello, message_";
    for (int i = 0; i < 50; i++) {
        // 发送消息
        rabbitTemplate.convertAndSend(queueName, message + i);
        Thread.sleep(20);
    }
}

3.2.2. Message reception

To simulate multiple consumers binding to the same queue, we add 2 new methods in the SpringRabbitListener of the consumer service:

@RabbitListener(queues = "simple.queue")
public void listenWorkQueue1(String msg) throws InterruptedException {
    System.out.println("消费者1接收到消息：【" + msg + "】" + LocalTime.now());
    Thread.sleep(20);
}

@RabbitListener(queues = "simple.queue")
public void listenWorkQueue2(String msg) throws InterruptedException {
    System.err.println("消费者2........接收到消息：【" + msg + "】" + LocalTime.now());
    Thread.sleep(200);
}

Notice that this consumer slept for 1000 seconds and the simulation task took time.

3.2.3.Testing

After starting the ConsumerApplication, execute the sending test method testWorkQueue just written in the publisher service.

You can see that consumer 1 quickly completed his 25 messages. Consumer 2 is slowly processing its 25 messages.

In other words, messages are distributed equally to each consumer, without taking into account the consumer's processing capabilities. This is obviously problematic.

3.2.4. Those who can do more work

There is a simple configuration in spring that can solve this problem. We modify the application.yml file of the consumer service and add the configuration:

spring:
  rabbitmq:
    listener:
      simple:
        prefetch: 1 # Only one message can be obtained at a time, and the next message cannot be obtained until the processing is completed.

3.2.5. Summary

Use of Work model:

Multiple consumers are bound to a queue, and the same message will only be processed by one consumer.
Control the number of messages prefetched by the consumer by setting prefetch

3.3.Publish/Subscribe

The model of publish and subscribe is as follows:

As you can see, in the subscription model, there is an additional exchange role, and the process has changed slightly:

Publisher: Producer, that is, the program that wants to send messages, but no longer sends them to the queue, but to X (switch)
Exchange: switch, X in the picture. On the one hand, messages sent by producers are received. On the other hand, knowing how to handle the message, such as submitting it to a special queue, submitting it to all queues, or discarding the message. How to do this depends on the type of Exchange. Exchange has the following 3 types:
- Fanout: Broadcast, delivering messages to all queues bound to the switch
- Direct: Directed, the message is delivered to the queue that matches the specified routing key.
- Topic: Wildcard, hand the message to the queue that matches the routing pattern (routing pattern)
Consumer: Consumer, as before, subscribes to the queue, no changes
Queue: The message queue is the same as before, receiving messages and caching messages.

Exchange (switch) is only responsible for forwarding messages and does not have the ability to store messages . Therefore, if there is no queue bound to Exchange, or there is no queue that conforms to the routing rules, the message will be lost!

3.4.Fanout

Fanout, the English translation is fanout, I think it is more appropriate to call it broadcast in MQ.

In broadcast mode, the message sending process is as follows:

1) There can be multiple queues
2) Each queue must be bound to Exchange (switch)
3) Messages sent by producers can only be sent to the switch. The switch decides which queue to send to, and the producer cannot decide.
4) The switch sends the message to all bound queues
5) Consumers who subscribe to the queue can get the message

Our plan is this:

Create a switch itcast.fanout, the type is Fanout
Create two queues fanout.queue1 and fanout.queue2 and bind them to the switch itcast.fanout

3.4.1. Declare queues and switches

Spring provides an interface Exchange to represent all different types of switches:

Create a class in consumer and declare queues and switches:

package cn.itcast.mq.config;

import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.FanoutExchange;
import org.springframework.amqp.core.Queue;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class FanoutConfig {
    /**
     * 声明交换机
     * @return Fanout类型交换机
     */
    @Bean
    public FanoutExchange fanoutExchange(){
        return new FanoutExchange("itcast.fanout");
    }

    /**
     * 第1个队列
     */
    @Bean
    public Queue fanoutQueue1(){
        return new Queue("fanout.queue1");
    }

    /**
     * 绑定队列和交换机
     */
    @Bean
    public Binding bindingQueue1(Queue fanoutQueue1, FanoutExchange fanoutExchange){
        return BindingBuilder.bind(fanoutQueue1).to(fanoutExchange);
    }

    /**
     * 第2个队列
     */
    @Bean
    public Queue fanoutQueue2(){
        return new Queue("fanout.queue2");
    }

    /**
     * 绑定队列和交换机
     */
    @Bean
    public Binding bindingQueue2(Queue fanoutQueue2, FanoutExchange fanoutExchange){
        return BindingBuilder.bind(fanoutQueue2).to(fanoutExchange);
    }
}

3.4.2. Message sending

Add a test method in the SpringAmqpTest class of the publisher service:

@Test
public void testFanoutExchange() {
    // 队列名称
    String exchangeName = "itcast.fanout";
    // 消息
    String message = "hello, everyone!";
    rabbitTemplate.convertAndSend(exchangeName, "", message);
}

3.4.3. Message reception

Add two methods in the SpringRabbitListener of the consumer service as consumers:

```java
@RabbitListener(queues = "fanout.queue1")
public void listenFanoutQueue1(String msg) {
    System.out.println("消费者1接收到Fanout消息：【" + msg + "】");
}

@RabbitListener(queues = "fanout.queue2")
public void listenFanoutQueue2(String msg) {
    System.out.println("消费者2接收到Fanout消息：【" + msg + "】");
}
```

3.4.4. Summary

What is the role of a switch?

Receive messages sent by publisher
Route messages to queues bound to them according to rules
The message cannot be cached, routing fails, and the message is lost.
FanoutExchange will route messages to each bound queue

What are the beans that declare queues, switches, and binding relationships?

Queue
FanoutExchange
Binding

3.5.Direct

In Fanout mode, a message will be consumed by all subscribed queues. However, in some scenarios, we want different messages to be consumed by different queues. At this time, Direct type Exchange will be used.

Under Direct model:

The binding between the queue and the switch cannot be done arbitrarily, but a RoutingKey(routing key) must be specified.
The sender of the message must also specify the message when sending the message to Exchange RoutingKey.
Exchange no longer delivers messages to each bound queue, but Routing Keymakes judgments based on the message. Only when the queue is completely consistent Routingkeywith the message Routing keywill the message be received.

The case requirements are as follows :

Use @RabbitListener to declare Exchange, Queue, and RoutingKey
In the consumer service, write two consumer methods to listen to direct.queue1 and direct.queue2 respectively.
Write a test method in the publisher and send a message to itcast.direct

3.5.1. Declare queues and switches based on annotations

It is troublesome to declare queues and switches based on @Bean. Spring also provides annotation-based declaration.

Add two consumers to the consumer's SpringRabbitListener, and declare queues and switches based on annotations:

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "direct.queue1"),
    exchange = @Exchange(name = "itcast.direct", type = ExchangeTypes.DIRECT),
    key = {"red", "blue"}
))
public void listenDirectQueue1(String msg){
    System.out.println("消费者接收到direct.queue1的消息：【" + msg + "】");
}

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "direct.queue2"),
    exchange = @Exchange(name = "itcast.direct", type = ExchangeTypes.DIRECT),
    key = {"red", "yellow"}
))
public void listenDirectQueue2(String msg){
    System.out.println("消费者接收到direct.queue2的消息：【" + msg + "】");
}

3.5.2. Message sending

Add a test method in the SpringAmqpTest class of the publisher service:

@Test
public void testSendDirectExchange() {
    // 交换机名称
    String exchangeName = "itcast.direct";
    // 消息
    String message = "红色警报！日本乱排核废水，导致海洋生物变异，惊现哥斯拉！";
    // 发送消息
    rabbitTemplate.convertAndSend(exchangeName, "red", message);
}

3.5.3. Summary

Describe the difference between Direct switches and Fanout switches?

Fanout switch routes messages to each queue bound to it
The Direct switch determines which queue to route to based on the RoutingKey.
Similar to Fanout functionality if multiple queues have the same RoutingKey

What are the common annotations for declaring queues and switches based on the @RabbitListener annotation?

@Queue
@Exchange

3.6.Topic

3.6.1.Description

TopicExchangeCompared with other types Direct, messages can be RoutingKeyrouted to different queues. It’s just that Topicthe type Exchangeallows the queue Routing keyto use wildcards when binding!

RoutingkeyGenerally, it consists of one or more words, and multiple words are separated by ".", for example:item.insert

Wildcard rules:

#: Match one or more words

*: Match no more, no less, exactly 1 word

Example:

item.#: can match item.spu.insertoritem.spu

item.*:can only matchitem.spu

Illustration:

explain:

Queue1: It is bound to china.#, so anything china.starting with routing keywill be matched. Including china.news and china.weather
Queue2: Binding is #.news, so anything .newsending with routing keywill be matched. Including china.news and japan.news

Case requirements:

The implementation idea is as follows:

And use @RabbitListener to declare Exchange, Queue, and RoutingKey
In the consumer service, write two consumer methods to listen to topic.queue1 and topic.queue2 respectively.
Write a test method in publisher to send a message to itcast.topic

/**
     * topicExchange
     */
@Test
public void testSendTopicExchange() {
    // 交换机名称
    String exchangeName = "itcast.topic";
    // 消息
    String message = "喜报！孙悟空大战哥斯拉，胜!";
    // 发送消息
    rabbitTemplate.convertAndSend(exchangeName, "china.news", message);
}

3.6.2. Message sending

Add a test method in the SpringAmqpTest class of the publisher service:

3.6.3. Message reception

Add methods in the SpringRabbitListener of the consumer service:

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "topic.queue1"),
    exchange = @Exchange(name = "itcast.topic", type = ExchangeTypes.TOPIC),
    key = "china.#"
))
public void listenTopicQueue1(String msg){
    System.out.println("消费者接收到topic.queue1的消息：【" + msg + "】");
}

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "topic.queue2"),
    exchange = @Exchange(name = "itcast.topic", type = ExchangeTypes.TOPIC),
    key = "#.news"
))
public void listenTopicQueue2(String msg){
    System.out.println("消费者接收到topic.queue2的消息：【" + msg + "】");
}

3.6.4. Summary

Describe the difference between Direct switches and Topic switches?

The message RoutingKey received by the Topic switch must be multiple words, **.**separated by
Wildcard characters can be specified for the bindingKey when binding a Topic switch to a queue.
#: represents 0 or more words
*: represents 1 word

3.7.Message converter

As mentioned before, Spring will serialize the message you send into bytes and send them to MQ. When receiving the message, it will also deserialize the bytes into Java objects.

However, by default, the serialization method used by Spring is JDK serialization. As we all know, JDK serialization has the following problems:

Data size is too large
There is a security vulnerability
Poor readability

Let's test it out.

3.7.1. Test the default converter

We modify the message sending code and send a Map object:

@Test
public void testSendMap() throws InterruptedException {
    // 准备消息
    Map<String,Object> msg = new HashMap<>();
    msg.put("name", "Jack");
    msg.put("age", 21);
    // 发送消息
    rabbitTemplate.convertAndSend("simple.queue","", msg);
}

Stop the consumer service

Check the console after sending the message:

3.7.2. Configure JSON converter

Obviously, the JDK serialization method is not suitable. We hope that the message body will be smaller and more readable, so we can use JSON for serialization and deserialization.

Introduce dependencies in both publisher and consumer services:

<dependency>
    <groupId>com.fasterxml.jackson.dataformat</groupId>
    <artifactId>jackson-dataformat-xml</artifactId>
    <version>2.9.10</version>
</dependency>

Configure the message converter.

Just add a Bean to the startup class:

@Bean
public MessageConverter jsonMessageConverter(){
    return new Jackson2JsonMessageConverter();
}