The basics of message queues
1. Endurance
Simply put, it is to store the data on the disk, instead of storing it in the memory and disappearing when the server restarts and disconnects, so that the data can be stored permanently.
Two, common persistence methods
Third, the distribution strategy of the message
01. Message distribution strategy
MQ message queue has the following roles
1: Producer
2: Store messages
3: Consumers
Then after the producer generates the message, MQ stores it, how does the consumer get the message? Generally, the way to get data is nothing more than push or pull. A typical git has a push-pull mechanism. The http request we send is a typical process of pulling database data back. The message queue MQ is a push process, and these push mechanisms are applicable to many business scenarios and there are many corresponding push mechanism strategies.
Scenario Analysis One For
example, I placed an order on the APP, and we have many systems and services. How do we know that the message is consumed by that system or those services or systems? At this time, we need a distribution strategy. This requires consumption strategies. Or call it the methodology of consumption.
Scene analysis two
There may be exceptions in the process of sending messages, or network jitter, failures, etc., which cause the message to be unable to be consumed. For example, the user is placing an order, the consumption MQ is accepted, the order system is malfunctioning, and the user payment fails. Then it is necessary at this time The message middleware must support the message retry mechanism strategy. That is to say, support: In the case of problems and failures, the message can be retransmitted without loss.
Fourth, the mechanism and comparison of message distribution strategies
Five, the message queue is highly available and highly reliable
1. What is a high-availability mechanism
The so-called high availability: refers to the ability of a product to perform a specified function under specified conditions and at a specified time or time.
When the business volume increases, the request is too large, a message middleware server will touch the limit of the hardware (CPU, memory, disk), a message server you can no longer meet the needs of the business, so the message middleware must support clustering deploy. To achieve the purpose of high availability.
2. Cluster mode 1-Master-slave master-slave shared data deployment mode.
Producers talk about consumption and send to the Master node. All of them are connected to this message queue to share this data area. The Master node is responsible for writing. Once the Master hangs up, the slave The node continues to serve. So as to form high availability
Cluster Mode 2-Master-slave synchronization deployment method
This mode of writing messages is also on the master master node, but the master node will synchronize data to the slave node to form a copy, which is very similar to the zookeeper or redis master-slave mechanism. This can achieve the effect of load balancing. If consumers have multiple, they can go to different nodes to consume, thinking that the copy and synchronization of messages will temporarily use a lot of bandwidth and network resources. It will be used in subsequent rabbtmq.
Cluster Mode 3-Multi-Master Cluster Synchronous Deployment Mode
The difference from the above is not particularly big, but its writing can be written to any node.
Cluster mode 4-multi-master cluster forwarding deployment mode
If the data you insert is in broker-1, the metadata information will store the relevant description of the data and the location (queue) where the record is stored.
It synchronizes the description information, that is, the metadata information. If consumers consume in broker-2 and find that they have no corresponding messages at some points, they can query from the corresponding metadata information, and then return the corresponding message information. , Scenario: For example, buying a train ticket or buying a concert ticket for a scalper. For example, the first scalper has a concert ticket that a customer said to buy, but there is no concert ticket, but he will contact other scalpers for inquiries and return if there is one.
6. Cluster mode 5 The combination of Master-slave and Breoker-cluster
realizes the multi-master and multi-slave hot backup mechanism to complete the high availability of messages and the hot backup mechanism of data. When the production scale reaches a certain stage, this use The frequency is relatively high.
Such a cluster mode will be specifically analyzed and explained in subsequent courses. Their ultimate goal is to ensure that the message server will not hang up, and the message service can still be used if there is a failure.
Anyway, it comes down to three sentences:
1: Either message sharing,
2: Either message synchronization
3: Either metadata sharing
6. What is a highly reliable mechanism
The so-called high availability refers to the fact that the system can run continuously without failure and low failure. For example, a system suddenly crashes, reports errors, exceptions, etc., and does not affect the normal operation of online business. The probability of error is extremely low, so it is called: high reliability .
In high-concurrency business scenarios, if the high reliability of the system cannot be guaranteed, the hidden dangers and losses caused are very serious.
How to ensure the reliability of middleware messages? It can be considered from two aspects:
1: Message transmission: The correctness of data analysis between systems is ensured through protocols.
2: Reliable storage of messages: The reliability of messages is guaranteed through persistence.
