Introduction to the five common models of RabbitMQ and the use of SpringAMQP

Introduction to MQ

异步通信MQ, MessageQuene is a message queue, which is a method between programs .
Why use MQ?
MQ is used for asynchronous communication between programs. The question of why to use MQ is also why it is necessary to introduce asynchronous communication between programs.
What is synchronous communication?
The simple understanding of synchronous communication is that the code to be executed in the next line cannot 完全执行be executed until the previous line is completed. For example: I want to query user information according to the order Id and return it. At this time, it is necessary to first call the order system to query the order information through the id, obtain the user id from the order information, call the user system to pass the user id to query the user information and return it.
If you want to query user information, you must first check the order information. The execution of the next line of code depends on the execution result of the previous line of code. We usually use synchronous communication in the code, which is executed step by step from top to bottom. jump. Just like when adding async to axios.
An example of using synchronous communication is unreasonable: After placing an order and paying, the order system will wait for the payment to be completed, and then call the storage system to perform operations such as deducting inventory. After the storage system is successfully executed, it will be handed over to the logistics system. After all executions are successful, the order success information is returned to the user.
Synchronous communication : Each system executes like a chain. If one part is broken, all will be broken. Adding or deleting requires modification. Asynchronous

communication : The next line of code does not have to wait for the execution result of the previous line of code, as long as it is called, it will eventually be executed. Similar to sub-threads, axios, etc.

Through the comparison of synchronous and asynchronous communication in this example, can we find the problems caused by synchronization?

• 性能和吞吐能力下降: The user will not get the response until all these systems are executed successfully in order. The user experience is greatly reduced. Just like ordering food in a restaurant, as long as the customer tells the dish to eat, the waiter can tell the customer that the order is successful (although it has not been done, but it can be successful), instead of waiting until the dish is served, the waiter tells the customer that the order is successful. The user orders successfully. This will make the user feel that it takes too long to order cauliflower. Back to the previous example: after the user has paid, it is already possible to ensure that the order must be successful, and return a response message to the user. The rest of the operations can be handed over to the background, as long as the success is guaranteed, and the user does not have to wait.

• 资源浪费: After the user's payment is successful, not only the user result is not returned in time, but this thread also participates in calling other systems, resulting in a waste of thread resources.

• 耦合度高: If a short message system is added to the warehousing system and logistics system, then the code of the order system will be modified, and a short message system will be inserted into the entire code result. To use asynchronous communication, you only need to develop an independent SMS system, connect to the middleware, and listen to the unified notification from the order system.
  The caller is only responsible for sending information to the queue, and the callee is only responsible for taking out messages from the message queue for calling. The two realize code decoupling through this middleman (message queue).
  
  级联失败: In synchronous communication, if an error occurs during this process, the following code will not be executed, and errors will occur in the entire business. But with asynchronous communication, businesses get messages through middleware, and there is no connection between them. Even if one business execution fails, it will not affect other businesses.

Summary :
Using asynchronous communication, you can quickly respond to user operations (although the execution is not completed, it can ensure the final execution is successful), and release resources for the next user. Asynchronous services are relatively independent, and they are uniformly called by middleware. It is very convenient to add and delete a certain service. At the same time, when a service fails, it will not affect other services.

Since asynchronous communication is so good, why use synchronous communication? :
Still the previous example, the payment system must be called synchronously in the order system, because subsequent calls are all waiting for the payment result. If you want to use asynchronous communication, and finally wait for the result below the code, it may not be as high as synchronous communication calls 时效性.

How to achieve asynchronous communication?
If it is a service, you can start a sub-thread to call it, but this can only improve the response speed of the user. What if there are many services? What about service dynamic scaling? This requires a message queue.
Usually, the call message is sent to a queue, and each callee listens to the message in the queue. Once there is a message in the queue, it obtains the parameters from the message queue and executes the corresponding method.

Benefits of using MQ

解耦: Pluggable effect can be achieved between service calls, each service is isolated from each other and does not affect each other
异步: no need to wait for follow-up processing, which improves system throughput, improves response speed, and reduces blocking
削峰: large traffic is blocked by the message queue first in each service In the front, messages can be kept in the queue as a buffer, and the queue can be called at an appropriate speed,
just like a dam. When there is a large flood, the dam can keep the flood in the reservoir, and then release it at an appropriate speed. Play a buffer role.

Disadvantages of using MQ :

• The complexity of the system has increased: there are a series of problems such as repeated consumption problems, order problems, message reliability problems, dead letter problems, and so on. Business processes are more chaotic than synchronous calls.
• Usability issues: When MQ is introduced, it is necessary to ensure that MQ can be used normally. Asynchronous calls depend on the performance, security, and availability of MQ. one more risk
• consistency problem

Note : It is not MQ that calls each service, but each service listens to the MQ message and calls the corresponding method through callback .
Thinking :

• Doesn't the call between services go through the service registry? Initiating a Ribbon call through the service center is a synchronous call
• How to use MQ to achieve a service instance call? Multiple service calls for an instance?

technical comparison

Common MQs are Rabbit MQ, Rocket MQ, Active MQ, Kafka. in these technologiesActive MQ较弱

	RabbitMQ	ActiveMQ	RocketMQ	Kafka
Company/Community	Rabbit	Apache	Ali	Apache
Development language	Erlang	Java	Java	Scala&Java
protocol support	AMQP，XMPP，SMTP，STOMP	OpenWire,STOMP，REST,XMPP,AMQP	custom protocol	custom protocol
availability	高	generally	高	高
Stand-alone throughput	generally	Difference	high	非常高
message delay	微秒级	Millisecond	Millisecond	within milliseconds
message reliability	高	generally	高	generally

RabbitMQ and RocketMQ are a trade-off 消息延迟between 吞吐量and Kafka 高吞吐, which is usually used in big data.

Use of RabbitMQ

Docker start

use command

docker run \
 -e RABBITMQ_DEFAULT_USER=用户名 \
 -e RABBITMQ_DEFAULT_PASS=密码 \
 --name mq \
 --hostname myhost \
 -p 15672:15672 \
 -p 5672:5672 \
 -d \
 Rabbit镜像id

RabbitMQ exposes two ports, 15672 is used for browser access to provide visual management, and 5672 is used to be connected by various services.
Access 15672 through a browser, enter the configured user name and password to go to the management page

The management panel has many functions, not only for viewing, such as: adding messages to queues, viewing queue messages, adding queues, managing queues, adding switches, managing switches, etc.

MQ role introduction

The most basic of MQ is the four roles in the figure
publisher: the message publisher is the caller of each service, that is, it sends the message to the message queue, and other services (the following consumers) listen to the callback: the switch is
exchangeresponsible for broadcasting the message to individual queues. It feels like a concave lens diverges light, broadcasting the message of the message publisher to the subscribed queue. Note that the same message is sent to the specified queue. If the publisher only sends messages to one queue, no exchange is needed. There is a publish and subscribe relationship with the queue if there is an exchange. The switch cannot cache data, the process of sending to the queue fails, and the message is directly lost.
quene: Message queue, which has the function of pushing messages to consumers and temporarily storing messages. Messages in the queue can only be consumed by one consumer.
consumer: Message consumer, the consumer must obtain the message from the queue , and then monitor the message in the queue internally, and call back as soon as there is a message in the monitored queue, so as to achieve the purpose of calling the method in the service.

A service uses a queue, and each instance in the service acts as a consumer under a queue.

Five common message models

There can also be multiple producers, but in the model, one and multiple are the same as long as they are sent to a queue in the same message format. For simplicity, one instance is used to represent no switch: no switch,
suitable for
consumers Yes 一个服务下有多或一个实例, each instance competes to consume the same message.

• 1、基本消息队列: One queue and one consumer, which is equivalent to only one instance under one service
  
• 2、工作消息队列: A queue corresponds to multiple consumers, which is equivalent to having multiple instances under one service
  

There are switches
. The switch can send a message to each queue to send the call information to an instance corresponding to each service. But what about
those queues that can get the message of the switch? There are three common

• 广播 Fanout: As long as all queues under a host will get the information of this switch, it is equivalent to calling an instance in all services under this host
  

• 路由 Direct: Only 完美the corresponding queue can receive the message
  

• 主题 Topic: Only the matching queue can receive the message. Unlike the previous one, the matching does not require one-to-one correspondence. It can be a range or a .level

Thinking : What should I do when there are more than one group of service publishers and consumers in a RabbitMQ server? In other words, how to isolate multiple identical and different business queue models?
One is introduced in RabbitMQ VirtualHost, and the effect is achieved through this 多租户, that is, a RabbitMQ is logically divided into multiple small RabbitMQs, VirtualHostand they are completely isolated.
Therefore, when connecting, not only the ip and port of RabbitMQ must be specified, but also which one is specificVirtualHost

Use the Spring AMQP template

What is AMQP : AMQP, an application layer standard advanced message queuing protocol that provides unified messaging services, is an open standard for application layer protocols designed for message-oriented middleware. The client and message middleware based on this protocol can transmit messages, and it is not limited by different client/middleware products, different development languages ​​and other conditions.

SpringAMQP simplifies the use of RabbitMQ in code

1. To introduce dependencies between producers and servers

 <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>

2. VirtualHostSpecify the ip and port of the RabbitMQ service and the corresponding (default /) in the configuration file

spring:
  rabbitmq:
    host: 192.168.83.100 
    port: 5672
    virtual-host: /
    username: yan
    password: 1234

In the simulation demonstration process, the consumer directly outputs the monitored messages. In practical applications, it is realized by using the callback to call the service method according to the message as a parameter 异步调用.

simple queue model

In the simple message model, sending a message needs to know 队列名字, and the consumer 队列名字determines to listen to the queue according to

• producer

• The producer usually sends messages to the queue in the service business, and the controller calls the service. Here is to write the business directly in the controller for simplicity
  

• consumer
  

It is worth noting that this method requires the declaration of the queue

work queue model

That is, one queue, multiple consumers

So there are multiple consumers in a queue, how are messages distributed? After one consumer consumes the message, will the other consume it again?
Example: When there are 10 messages in the queue, both consumers A and B listen to the queue. Consumers take out the messages on average first, and then consume them . Once a message is fetched, other consumers cannot consume the message.
There is a problem in taking out averagely before consuming: , as many messages with poor performance and good performance are taken out, but the consumption time of the same amount of messages is different, and there will be idle consumers with good performance and poor performance. Consumers can't handle it. The overall service processing efficiency will be reduced, and they should be allowed to fetch messages according to their processing capabilities.
What if it is taken out according to the processing capacity?
It's like eating a buffet. Those with small appetites get the same amount as those with large appetites. To make them take according to their appetites, you have to let them eat before taking them. The same is true for the consumption queue. Each consumer can only get one message (the number can be configured) each time. After processing, he will go to get the next message. More reasonable, the overall processing efficiency is improved.
Change setting:

Broadcast Model Fanout

The switch is introduced to spread messages to each queue.

Consumers still specify the queue name to get messages from the queue, but the queue is not directly connected to the producer, but the switch . The publisher sends information to the exchange, and the exchange is broadcasting information to the bound queue.
Therefore, the producer must declare the exchange, declare the queue, bind the exchange and the queue, and finally send the message to the exchange.

Here, fan1the exchange named is used to bind the exchange with two queues named , and each queue has a consumer q1. q2I send a message to the exchange, and the consumers under both queues will receive the message

Method 1: Use the @Bean method to bind the queue to the switch

---

consumer side

Still get messages from the queue on

the producer side
Producers still send messages to the queue

Process summary :

Method 2 : Using annotations
The producer still sends messages to the switch, but it does not need to manage the queue and bind the switch to the queue.
When the consumer listens to the queue, it specifies its switch and binds the relationship between the queue and the switch

Switch summary :

1. With the switch, the producer wants to send the message to the switch
2. The switch must be bound to the queue

Note :
3. The switch cannot cache data. If the switch fails to broadcast the message to the queue, the message will be lost.

Routing Mode Direct

Different from the broadcast mode, not all the queues bound to the switch must be able to obtain the messages pushed by the switch, and on this basis, the password ( ) routing Keyand 当前消息the password of the switch ( routing Key) 完全匹配. This mode enables the switch to broadcast messages to certain queues in the queues bound to it.
For example: I only want to call the warehousing system when I send a message in the order system this time, but there is also a logistics system bound to the switch, which needs to be specified in the message, and only the queue that matches it can be routingKeyobtained, making the call more flexible .

Usage In Faout mode, the producer side specifies when sending a messageroutingKey

The consumer specifies the routingKey when binding the queue and the switch

In DIRCT mode, the queue wants to obtain the conditions that the data in the switch meets

1. Bind to the switch
2. The queue routingKeycontains the messages bound to the exchangeroutingKey

Even if the routingKey is the same, the switches bound to the listening queues are different, and the data cannot be obtained.
Direct mode When the routingKey of each queue contains the routingKey of this message, then the Faout mode is reached.

Theme mode Topic

Compared with Direct mode, routingKey can be used in this mode 通配符. Each .represents a level across a level, representing any level *For example: = = but ! = because it can only represent one level (one word) Note that here is less, only one#
a.b.ca.*.ca.#a.b.ca.**
*

This kind of wildcard can be specified not only in the producer's message, but also in the consumer binding queue. It
can

also

be declared and bound in the way of @Bean. Only one is demonstrated here.

Note : Not only messages of String type can be sent to the queue, but all types of messages are supported. When sending an object as a message, it must be serialized due to the transmission of the object.

So how to pass object data through the message queue?
The sender can convert the object into a JSON string format to serialize the object, and the consumer can deserialize the JSON string into an object.

Although springAMQP can automatically create switches and queues, producers must ensure that they exist before sending to queues and switches. The statement can be on the consumer side or the producer side, but it should be on the consumer side, so that the producer can be unaware when there is a business increase or decrease. The declaration of queues, switches and their binding relationship can be 注解declared in this way, or 注入beandeclared in the container

In RabbitMQ, once the switch is created and the type is specified, it is not allowed to be modified unless the virtual machine is deleted.
For example: I created a switch named a1 and specified it as faout type, but I want to change a1 to direct. At this time It is unsuccessful, the a1 switch is still of faout type, and such an error will be reported. inequivalent arg 'type' for exchange 'e1' in vhost '/': received 'direct' but current is 'fanout'
The solution is either to change the name of the switch, or delete and create a new type.