1. What is rabbitmq
RabbitMQ is to achieve the Advanced Message Queuing Protocol (AMQP) is an open source message broker software;
Using AMQP (Advanced Message Queuing Protocol) Advanced Message Queuing Protocol a message queue technology, the biggest feature is that the consumer does not need to ensure that providers exist to achieve a high degree of decoupling between service;
RabbitMQ is an open source, written in Erlang, based on AMQP protocol, messaging middleware;
2. Why use rabbitmq
1 includes a number of advanced features asynchronous, clipping, at load balancing (distributed system);
2. has a persistent mechanism to process the message, a message queue can also be preserved.
3. To achieve decoupling between consumers and producers.
4. For a highly concurrent scenario, the use of such synchronized access to the message queue may become serial access limiting certain amount, is conducive to the operation of the database.
The message queue can be used to achieve the effect asynchronous orders, queue, the background logic orders.
3. Use rabbitmq scene
1. asynchronous communication between services
2. The order of consumption
3. regular tasks
4. Request clipping
4. How to ensure that messages are correctly transmitted to RabbitMQ? How to ensure that the message recipient consume the news?
Transmission confirmation pattern:
To confirm the channel setting mode (transmission confirmation mode), all messages are posted channel is assigned a unique ID.
Once the message is delivered to the destination queue, or after the message is written to disk (persistent message may be), the channel sends an acknowledgment to the producer (containing the unique message ID).
If an internal error occurred causing RabbitMQ message is lost, it sends a nack (not acknowledged, unacknowledged) messages.
Transmission confirmation pattern is asynchronous, while waiting for the application producer acknowledgment, can continue to send the message. When the confirmation message arrives producer application, the application callback method producers will be triggered to process the confirmation message.
Receiving confirmation mechanism
Mechanism acknowledgment message recipient: the consumer receives each message must be acknowledged (message reception and acknowledgment message are two different operating). Consumers only confirmed the news, RabbitMQ can safely delete the message from the queue.
Here and did not use a timeout mechanism, RabbitMQ only to confirm that the need to re-send the message by connecting the Consumer interrupt. In other words, as long as the connection is not interrupted, RabbitMQ to the Consumer sufficient length of time to process the message. Ensure that the final data consistency;
The following lists some special circumstances:
If the consumer receives a message, disconnected or unsubscribe, RabbitMQ think that message was not distributed, then re-distributed to consumers under a subscription before confirming. (There may be a message repeated consumption of risks, need to re)
If the consumer has not received the message confirmation message, the connection has not disconnected, the RabbitMQ think the consumer is busy, will not give the consumer more news distribution.
5. How to avoid duplicate message delivery or repeated consumption?
When message production, internal MQ message generated for each transmission of a producer inner-msg-id, as a basis to weight (and retransmits the message delivery failure), to avoid duplicate message into the queue;
When news consumption, requires the message body must have a bizId (for the same business globally unique, such as payment ID, order ID, message ID, etc.) as a basis to weight, to avoid the same message is repeated consumption.
One solution is the idea: to ensure the uniqueness of the message, even if multiple transmissions, do not let the message many times the consumer impact; ensure power messages and so on;
For example: when writing the message that uniquely identifies the data queue do, message consumer, according to the unique identification is determined whether post-consumer;
6. What message based on the transmission?
Since the creation and destruction overhead of TCP connections larger, and the number of concurrent limited by system resources will cause performance bottlenecks. RabbitMQ way channel used to transmit data. Channel virtual connection is established in the real TCP connections, and the number of channels per TCP connection is not limited.
7. How message distribution?
If the queue has at least one consumer subscription, the message will be sent in the cycle (round-robin) fashion to consumers. Each message will only be distributed to a consumer subscription (provided that the consumer can properly process the message and confirm).
Can achieve more consumed by the routing function
channel.basicQos (1) ;. How fast consumption, and more spending. Pull disposed just take a message. (Each consumer while only deal with a message
）
8. how the message routing?
Conceptually, the message routing must have three parts: switches, routing binding. Producers publish to the message switch; binding from the router determines how to route the message to a particular queue; final message queue, and the consumer receives.
When news release to the switch, the message will have a routing key (routing key), set in the message creation.
Routing keys through the queue, the queue can be bound to the exchanger.
After the message arrives at the switch, RabbitMQ will route the message queue routing keys and key match (with different rules for different routing switch). If the pattern matches to the queue, the message will be delivered to the respective queue; if not matched to any queue, messages will enter the "black hole."
Common exchanger is divided into about three types:
direct: If the routing keys exact match, the message is delivered to the appropriate queue
fanout: if the exchange receives the message will be broadcast to all queues bound
topic: messages from different sources can be made to reach the same queue. When using the topic exchanger can use wildcards, for example: "*" matches any text in a particular location, the routing keys divided into several parts, "#" matches all the rules. "." Special attention: topic sent to switch messages can not be arbitrarily set selection key (routing_key), it must be spaced apart by a series of identifiers consisting of. "."
9. How to ensure that messages are not lost?
1. Manual ack determined manually (manual consumption is job-complete confirmation)
2. The message persistence, of course, the premise is to be persistent queue
RabbitMQ way to ensure a persistent message from the server restart in recovery is to write them a persistent log files on the disk when it was released a persistent message to the persistence on the switch, Rabbit will be submitted in a message to the log file it sends a response.
Once the consumer spending a persistent message from the persistent queue, RabbitMQ will put the messages are flagged for garbage collection in the persistent log. If persistent messages RabbitMQ restart before being consumed, the Rabbit will automatically rebuild switches and queue (and binding), and republish persistent log file messages to the appropriate queue.
10. Use RabbitMQ What are the benefits?
Room service highly decoupled,
High performance asynchronous communication,
Traffic clipping
Message Distribution
Message Buffer
…
The disadvantage of 12.mq
Reduce system availability
The more dependent on external systems introduced, the more easy to hang, you had A system call interface is BCD three systems like, four people ABCD system properly, nothing issue, you add a partial MQ come in, in case MQ hung up zezheng? MQ hung up, the entire system crashes, you do not what would be finished.
Increase system complexity:
Add MQ abruptly come in, how do you ensure that the message is not repeated consumption? How to deal with the case of message loss? How to ensure that the order of message delivery? The bulk of the big head, a lot of questions, endless suffering
Consistency:
A processing system over direct return is successful, people think that you request will be successful; but the problem is that if the three systems where BCD, BD two write library system is successful, the results of C write library system failed, zezheng? You data is inconsistent.
---------------------
11.rabbitmq cluster
Mirroring cluster model
queue you created, or whether the metadata inside the message queue will exist on more than one instance, then every time you write a message to the queue, the queue will automatically messages to multiple instances of messages in sync.
Benefit is that you either machine goes down, all right, other machines can be used. The downside is that, first, this performance overhead is too big now, synchronized news all the machines, resulting in network bandwidth consumption and heavy pressure! Second, such a play, there is no extension at all, and if a queue is heavily loaded, you add a machine, the new machine also contains all the data of the queue, and there is no way to expand your queue linear
12, how to ensure reliable transmission RabbitMQ message? (Messages are not lost?)
A: The message is unreliable messages may be lost, hijacking and other reasons;
Loss is divided into: the producers lost messages, message list is missing messages, consumers lose messages;
Producers lost messages :( producer confirmation message mechanism) lost data from producers this perspective, RabbitMQ provides transaction and confirm model to ensure that producers do not lose the message;
That transaction mechanism: Before sending a message, open a transaction (channel.txSelect ()), then sends a message that if anything unusual occurs during sending, the transaction will be rolled back (channel.txRollback ()), if the transaction is committed successfully sent (channel.txCommit ()). However, this approach has a drawback: the throughput-down;
Mostly used confirm mode: once enters confirm channel mode, all messages are posted in the channel will be assigned a unique ID (starting at 1), once the message is delivered to the queue after all matching;
rabbitMQ sends an ACK to the producer (the unique ID contained in the message), which message has the right to know the manufacturers reach the destination queue;
If rabbitMQ could not process the message, it will send a Nack message to you, you can perform a retry operation.
Lost data message queue: message persistence.
Where lost data message queue processing, typically is on persistent disks.
This configuration can confirm persistence mechanism used in conjunction with, you can give the producer sends a message Ack signal after persistent disk.
In this way, if the message before persistent disk, rabbitMQ killed, so producers can not receive Ack signal, the producers will be automatically re-issued.
So how persistent it?
By the way, right here, in fact, very easy, just two steps below
The persistent queue identifier durable set to true, represents a durable queue is
sending messages, deliveryMode = 2 will be
arranged such later, even rabbitMQ hung up, restart can recover data
Consumers lost message: Consumers lose the data is usually because the use of automatic confirmation message mode, a confirmation message can be changed manually!
Consumers After receiving the message, before processing messages, RabbitMQ will automatically reply message has been received;
If you then process the message fails, the message will be lost;
Solution: After successfully processing a message, manual reset confirmation message.
13, how to assure an orderly RabbitMQ message?
A: The single-threaded consumption guarantee the order of the message; the message number, the message is processed according to number of consumers process messages;
14. How to specify priority of the message
Max priority queue set parameters, the default priority message RabbitMQ is 0, the maximum is 10
15. The persistent message is how to achieve
RabbitMQ divided persistence: persistence, persistence exchanger persistent message queues and
And switch queues persistence parameters are set by the durable statement is true when implemented
Persistent message is a message sent to the designated 2 deliveryMode achieved
16. How to set the message expiration time
Attribute setting queue, the queue all the messages have the same expiration time
of the message itself separately provided, each message can be different TTL
If used with both methods, the TTL of the message to a small value that is between the two quasi
17. how to deal with the message delivery failure
First, there is the case of two delivery failure
When the switch can not find qualified queue according to their type and routing keys
if the exchange is found when a message is routed to the queue does not exist any consumer on the queue
Solution:
When you specify mandatory or imrnediate parameter to true when the producer delivered messages, RabbitMQ will undeliverable messages Basic.Return command returns a message to producers, producers need at this time to call channel.addReturnListener to add an ReturnListener monitor Xin for message delivery failure monitor
If the above two parameters necessary to add ReturnListener logic, and the logic becomes complicated producers, RabbitMQ backup switch can handle this problem
In a statement at the time by the switch (call channel.exchangeDeclare method) adding alternate-exchange parameters to achieve
For backup switch, comprising the use of several special cases:
If there is no backup switch, RabbitMQ client and server will not be an exception, then the message will be lost
If the backup switch is not bound to any queue, the client and the server RabbitMQ will not be an exception, then the message will be lost
If the backup switch does not match any of the queue, the client and the server RabbitMQ will not be an exception, then the message will be lost
If the backup switch and mandatory parameters used together, then the mandatory parameters is invalid
18.Spring AMQP
The lower is rabbitMq
(RabbitTemplate.convertAndSend (conversion and transmission))
AmqpTemplate.convertAndSend (conversion and transmission)
Spring to help us perform serialization deserialization.
by
ObjectOutputStream serialize
ObjectOutputStream deserialization
19. ** template (template) to automatically generate the code level switch queue and bound to persist.
Or create your own console
19. The producer sends message retry is provided
The global unique ID idempotent message 20.RabbitMQ
MQ resolution message retransmission - do idempotency
https://blog.csdn.net/wo1901446409/article/details/97011754
The second half to ensure idempotency not repeat consumption.
Generating a unique ID, the binding MYsql uniqueness.
Original link: https: //blog.csdn.net/wo1901446409/article/details/97389212