I have read a lot of articles about cloud native, either cloud and fog, or high and low. Therefore, the author has the idea of writing a series of articles "Dahua Cloud Native", hoping to use the most popular, simple language, and easy-to-remember scenarios to explain: the composition and application relationships in the cloud native ecosystem.
In the previous articles, we have introduced docker, k8s, grayscale publishing, microservices, load balancing, etc. in the vernacular. In this article, we will take a look at the component often used in the cloud native environment - message queue. When it comes to message queues, some friends may often hear some terms from others, such as: decoupling service programs, processing traffic peaks, improving user experience through asynchronous processing, and buffering batch processing to improve processing performance. The author is good at explaining in the vernacular, so I do not use professional terms to explain professional issues. I have always felt that the function of the message queue is very similar to that of the express cabinet. What is the similarity method? Let me tell you in detail.
The wonderful relationship between message queues and express lockers
Article directory
1. The message queue is a courier cabinet
Let's compare express cabinets with message queues
- The message queue is compared to express cabinets : there are many manufacturers producing express cabinets, such as: Fengchao (apache kafka), express (alibaba RocketMQ), neighbor treasure (ActiveMQ), etc. Anyway, these are the commonly used ones. The express cabinet is responsible for temporarily saving mail, and the message queue is responsible for temporarily saving message data.
- The courier is likened to a message producer : the courier is responsible for delivering mail to the express cabinet, and the producer is responsible for delivering messages to the message queue. What a wonderful thing to do!
- Consumers are compared to message consumers : It may be that this example is so apt that this sentence is nonsense any way you look at it. Nonsense is still to be said, consumers in life get mail, consumers in the program get message data.
Second, the benefits of express cabinets (message queues)
Let's first review how we spent the days without a courier cabinet : one morning, we suddenly received a call from a courier: "Brother, I have your courier." I thought it was really bad: "You won't come early or late, I'll be going to work soon, you come at this time. Well, I'll wait for you for a while." As a result, it is possible that the courier is very unreliable, sometimes saying that there is a traffic jam, sometimes saying that he will be here soon, and waiting for you to be late for work. This situation breaks you down!
Suddenly one day, a thing called a courier cabinet suddenly appeared in the community. This thing is good. "Brother, there is your courier." I thought to myself who is your brother: "Ah, you can put it in the courier cabinet, I will come back after get off work at night to pick it up." The courier happily put the courier in the courier cabinet and started to make a phone call, 10 mails in the morning. If each is delivered to the door, the courier will take at least half an hour. Okay now, 9 express cabinets, 1 user requested to be delivered to the door, and it was done in 10 minutes. The courier thinks this stuff is really good!
The couriers are happy, and consumers are actually very satisfied. Some shopaholics may receive 10 or more emails a day. When there is no courier cabinet, the courier will pick up the courier once (wait once) with a single call. With the express locker, I can take it all together when I get off work. The above example reflects several advantages of the message queue (express cabinet), please comment carefully:
- Asynchronous decoupling: With express cabinets, consumers do not have to wait for couriers, and the user experience is enhanced. The decoupling between consumers and producers (couriers) will not affect their own independent procedures because of the operation behavior of the other party. Users do not have to wait and receive events to block time-consuming.
- Traffic clipping: We assume an extreme situation, you buy 1,000 books through various channels, and suddenly call you in a certain hour. You definitely don't have the ability to receive 1,000 mails in an hour, so you ask the courier to put the mail in the courier locker. So you can pick up emails on your own schedule, as you can handle. Similarly, when our consumer program faces multi-user and high-concurrency requests, storing data in message queues can reduce the peak traffic data, and consume data according to the capabilities and resources that the program can handle.
- Buffer batch processing: Producer batch delivery, cool! Consumers can get multiple emails at one time, cool!
3. Disadvantages brought by the introduction of express cabinets
Having said so many advantages, are there any disadvantages to express cabinets? Of course there is
-
Introduce complexity. There is no doubt that the express cabinet (message queue) is extra, and it did not exist in the original collection process. Therefore, it needs a place to put it, and it also needs to be fireproof, anti-theft and moisture-proof, and it needs to be maintained. The message queue middleware is the same, you need to install it in the server area and maintain it.
-
Temporary data inconsistencies will result. If there is no courier locker, when you receive the mail, it is really received. But after using the express locker, you will receive the "message that the mail is put into the express locker", but there will be a certain delay between when you actually get the mail. Of course, you will still get the mail eventually, and choosing the "message queue" express cabinet is to endure temporary inconsistency and accept "eventual consistency".
-
Of course, in extreme cases, the courier locker is broken, and you have to accept the fact that "mail may be lost", which hardly happens in a community with a high security factor.