1. Some knowledge of RabbitMQ
1. Message attributes
RabbitMQ is a message middleware based on the AMQP message transmission protocol; similar to HTTP, there are two parts of data, header and body, and Message is the concept of message body in RabbitMQ.
Message is composed of Properties and Body. The former is some meta-information, such as message priority, persistence, transmission format (such as JSON), delay and other advanced features, and Body is the message data entity transmitted.
2. Message delivery
The three concepts of Exchange, Queue and Routing Key are the key to understanding RabbitMQ message delivery. A core principle in RabbitMQ is that messages cannot be delivered directly to Queue.
Producer can only deliver its own message to Exchange, and Exchange delivers it to the corresponding Queue according to the routing_key.
3. Message reliability
Unlike HTTP synchronous access, in RabbitMQ, the Producer does not know whether the message has been reliably delivered to the Consumer for processing. So, how does RabbitMQ ensure reliable delivery of messages?
There are two main points: first, the message confirmation mechanism. After the Consumer finishes processing the message, it needs to send a confirmation message to the Broker Server. There are three ways to choose "confirm receipt", "discard" and "redelivery". If the Consumer hangs up before the Broker Server receives the confirmation message, the Broker Server will re-deliver the message.
Second, you can choose data persistence so that even if RabbitMQ restarts, messages will not be lost
2. RabbitMQ usage scenarios
1. Data access
Assume that there is a user behavior collection system responsible for collecting user click behavior data from the App side. Usually, data reporting and data processing are separated, that is, the App side reports data through the REST API, and the backend receives the data and puts it in the queue and returns immediately, while the data processing uses Worker to fetch data from the queue, as follows As shown in the figure.
The benefits of doing this are:
First, the separation of functions, the reported API interface does not care about the data processing function, and is only responsible for accessing data;
Second, data buffering, the rate of data reporting is uncontrollable, depending on the user's frequency of use, using this mode can buffer data to a certain extent;
Third, it is easy to expand. When the amount of data is large, it can be expanded by adding data processing Workers to increase the processing rate. This is a typical producer-consumer model, where data is reported as a producer and data is processed as a consumer.
2. Event distribution
Assume that there is an e-commerce system. User behaviors such as "collecting", "ordering", and "paying" are very important events. Usually, the back-end service needs to do a lot on these event points in addition to completing the corresponding functional processing. Other processing actions, such as sending SMS notifications, recording user points, and so on.
If we put these actions in each module, it is not conducive to functional decoupling and code maintenance. At this time, we need an event distribution system to publish corresponding events in each functional module, and the processors who are interested in it to process.
There are two roles involved here: A is interested in B, A is the handler, and B is the event. The event handler completes the binding of the two and subscribes the event to the message center. The service module is the back-end business logic service, which publishes events at different event points, and the events are distributed to the handlers corresponding to the event handlers through the message center.
The whole process is shown in the figure below, and here is a typical subscription-publishing mode.
3. Six working modes of RabbitMQ
1. Simple mode: only one queue and one consumer
default AMQP交换机In simple mode, we don’t need to specify a switch . RabbitMQ will deliver our messages to the specified queue by default. It is a Direct type switch. The binding key when the queue is bound to it is actually the name of the queue .
2. WorkQueues: work queue mode, only one queue, multiple consumers
The work queue mode also uses the default AMQP switch, and the messages in the Queue will be evenly distributed to multiple consumers for processing. There are two ways of message distribution:
Round-robin distribution : One consumer consumes one item and is distributed equally . In the woek mode, the round-robin distribution method is adopted by default. Polling distribution does not write code demonstration, it is relatively simple, for example, if a producer sends 6 messages to the queue, if there are 3 consumers listening to this queue at the same time, then each of these 3 consumers will share Got 2 messages. The following mainly introduces fair distribution.
Fair distribution : Fair distribution is carried out according to the consumption capacity of consumers, the ones that are processed quickly get more, the ones that are processed slowly get less, and those who can do more work .
3.Publish/Subscribe: publish subscription mode, there are N queues, N consumers, you need to declare the binding switch
There are three types of switches in RabbitMQ: fanout, direct, topic
(1) fanout: fan-out mode, the message is broadcast to all queues bound to the switch
(2) direct: routing mode, the message publisher specifies a key, and the message is routed to all queues that match the key
(3) topic: topic mode, similar to routing mode, can be fuzzy matched by wildcards
4. RPC mode: support the situation where the producer and the consumer are not in the same system, that is, remote calls are allowed
- In the RPC mode, the consumer is usually placed in a remote system as a server and provides an interface; the producer calls the interface and sends a message.
- The RPC mode is a remote call mode, because it requires http requests, so the speed is slower than the internal call of the system. Moreover, in rpc mode, it is usually not easy to distinguish which are external requests and which are internal requests, resulting in a slower overall speed. Therefore, the rpc mode cannot be abused.
- There are two properties in MessageProperties:
reply_to: used to define the name of the callback queue
correlation_id: Used to correlate rpc message request sending and message response receiving.
- To implement the RPC mode, the producer needs to send a callback queue, the workflow:
1. After the producer (Client) starts to produce messages, it creates an anonymous and exclusive callback queue.
2. When the producer (Client) sends a request, it contains two attributes: reply_to, which is the callback queue; correlation_id, which is the ID of this request
3. The request (request) is sent to the rpc_queue queue.
4. The consumer (The RPC worker) waits for the request on the rpc_queue, when received, it processes the message and sends the result back to the client using the queue in the replyTo field
5. The producer waits for the message on the callback queue. When the message appears, check the correlation_id, and return the response data to the program if it matches the value in the request.