Table of contents
2. The three main functions of the message queue are: asynchronous, peak cutting, and decoupling
3. Disadvantages of message queues
4. Examine this design from the perspective of three high levels
4. Several common message queues
1. Definition
Message Queue: Generally, we will refer to it as MQ (Message Queue) for short.
Message Query (MQ), message queue middleware, many beginners believe that MQ realizes asynchrony and decoupling of programs through sending and receiving messages. MQ is mainly used for asynchronous operations. This is not the real purpose of mq, but just mq application, the real purpose of mq is for communication.
It shields complex communication protocols, like the commonly used dubbo, http protocols are all synchronous.
It is difficult for these two protocols to achieve double-ended communication. A calls B, and B can also actively call A, and does not support long connections. What mq does is to build a simple protocol on top of these protocols—the producer and consumer models. What mq brings us is not the underlying communication protocol, but a higher-level communication model. He defines two objects: those who send data are called producers, and those who receive messages are called consumers. We can ignore the underlying communication protocol, and we can define producers and consumers ourselves.
2. The three main functions of the message queue are: asynchronous, peak cutting, and decoupling
The message queue is like a Three Gorges Dam, used to intercept the pressure from the upstream.
Asynchronous: What was done in the previous step has been changed to multi-step
Peak shaving: (for example, the traffic suddenly increased by 100 times, and the program is designed for the size of the data in between) the pressure is piled into the mq, and the consumer does not need to resist the surge of data, and the message will be placed in the message queue , Consumers can consume slowly. Block the surge in traffic through mq so as not to overwhelm downstream services. If a service in the server is down, it is likely to have a chain reaction that paralyzes the entire system
Decoupling: Decoupling the producer and the consumer, as long as the producer puts the message into mq, the producer's task is completed, and the consumers are not related to each other
(Change from the above general form to the following mq implementation)
Mq is generally used for modification operations (when my modification operation, the call chain is too long, I need to use mq)
3. Disadvantages of message queues
①, increasing the complexity of the system.
So if your business volume is not large and the concurrency is not high, there is no need to use message queues.
②, business issues.
The transaction problem is actually a problem that must exist in a distributed system, but the message queue is more serious. There are two general solutions. The first is to use distributed transactions. All the services involved in the order are put into one transaction, and either all succeed or all fail. The second is that consumers take reasonable data compensation measures, such as message retry, manual data refresh, and so on.
③ Availability
The decoupling I just talked about is actually the decoupling between the various modules of the system, but these modules are all associated with the message queue. If the message queue hangs up, it is really impossible to place an order. In order to ensure availability, we can use message queue clusters, front-end traffic limit, etc., which will be introduced later.
4. Examine this design from the perspective of three high levels
High availability: When a module in my system goes down, it will not affect my other services. (The final consistency of data can be guaranteed through data compensation or distributed transactions)
High performance: The user places an order, puts all the data required for the order into the message queue, and returns it directly. All the time-consuming is equivalent to the time-consuming of network transmission.
High concurrency: Since the message queue does not handle any business logic, all the concurrency it supports is at the million level. If there are 1 million users placing orders, and 1 million data are placed in the message queue, the service connected to the message queue can be consumed slowly, and it will not cause millions of requests to come in instantly and overwhelm my service.
4. Several common message queues
Among them, kafka and rocketmq are currently the most widely used in major Internet companies because of their high throughput.