1. What are the benefits of using RabbitMQ?
1. Decoupling, system A directly calls the codes of system B and system C in the code. If system D is connected in the future, system A needs to modify the code, too troublesome!
2. Asynchronous, write messages to the message queue, non-essential business logic runs asynchronously, speeding up the response speed
3. When the peak is cut and the concurrency is large, all requests are directly sent to the database, causing the database connection to be abnormal
2. What is the broker in RabbitMQ? What does cluster mean?
Broker refers to the logical grouping of one or more erlang nodes, and the RabbitMQ application is running on the node. The cluster is based on the broker and adds the constraint of sharing metadata between nodes.
3. Is the channel, exchange and queue in the concept of RabbitMQ a logical concept, or does it correspond to the process entity? What role do they play?
queue has its own erlang process;
Exchange is internally implemented as a lookup table to preserve binding relationships;
The channel is the entity that actually performs the routing work, that is, it is responsible for delivering the message to the queue according to the routing_key .
According to the description of the AMQP protocol, the channel is a virtual connection above the real TCP connection. All AMQP commands are sent through the channel , and each channel has a unique ID.
A channel can only be used by a single operating system thread, so messages delivered to a particular channel are in order .
However, multiple channels are allowed on one operating system thread.
4. What is vhost? What role does it play?
vhost can be understood as a virtual broker, that is, mini-RabbitMQ server. It contains independent queue, exchange, binding, etc., but the most important thing is that it has an independent permission system, which can achieve user control within the scope of vhost. Of course, from the global perspective of RabbitMQ, vhost can be used as a hand with different permissions.
Segment (a typical example is that different applications can run in different vhosts).
5. What transmission is the message based on?
Because the TCP connection creation and destruction overhead is large, and the number of concurrency is limited by system resources, it will cause performance bottlenecks. RabbitMQ uses channels to transmit data. Channels are virtual connections established within real TCP connections, and there is no limit to the number of channels on each TCP connection.
6. How is the message distributed?
If at least one consumer subscribes to the queue, the message will be sent to the consumer in a round-robin manner. Each message will only be distributed to a subscribed consumer (provided that the consumer can process the message and confirm it normally).
7. How to route messages?
Conceptually, message routing must have three parts: switch, routing, and binding. The producer publishes the message to the switch; the binding determines how the message is routed from the router to a specific queue; the message finally reaches the queue and is received by the consumer.
When a message is published to the switch, the message will have a routing key, which is set when the message is created.
Through the queue routing key, you can bind the queue to the switch.
After the message reaches the switch, RabbitMQ will match the routing key of the message with the routing key of the queue (different routing rules for different switches).
If it can be matched to the queue, the message will be delivered to the corresponding queue; if it cannot be matched to any queue, the message will enter the "black hole".
Commonly used switches are mainly divided into the following three types:
direct: If the routing keys match exactly, the message is delivered to the corresponding queue
fanout: if the switch receives a message, it will be broadcast to all bound queues
topic: Can enable messages from different sources to reach the same queue. When using the topic switch, you can use wildcards, such as: "*" matches any text in a specific position, "." Divides the routing key into parts, and "#" matches all rules. Special note: the message sent to the topic switch cannot be set at random (routing_key),
Must be a series of identifiers separated by ".".
8. What is metadata? What are the types of metadata? What is included? What are the metadata related to cluster? How is the metadata saved? How is the metadata distributed in the cluster?
In non-cluster mode, the metadata is mainly divided into Queue metadata (queue name and attributes, etc.), Exchange metadata (exchange name, type and attributes, etc.), Binding metadata (lookup table for storing routing relationships), Vhost metadata ( Namespace constraints and security attribute settings for the first three within the scope of vhost).
In cluster mode, it also includes node location information and node relationship information in the cluster. The metadata depends on the type of erlang node to determine whether it is stored only in RAM or on both RAM and disk. Metadata is distributed across nodes in the cluster.
9. What is the difference between declaring queue, exchange, and binding in a cluster system consisting of a single node system and multiple nodes?
A: When you declare a queue on a single node, as long as the relevant metadata on the node has been changed, you will get a Queue.Declare-ok response; while declaring a queue on the cluster requires all nodes on the cluster to be Queue.Declare-ok response will be obtained only after the metadata is successfully updated. In addition, if the node type
For the RAM node, the changed data is only saved in the memory, and if the type is disk node, the data saved on the disk must also be changed.
Dead letter queue & dead letter exchange: DLX (Dead-Letter-Exchange), which is called dead letter exchange. When a message becomes a dead letter, if the queue where the message is located exists x-dead-letter-exchange Parameter, then it will be sent to the exchange corresponding to x-dead-letter-exchange, this exchange is called dead letter exchange
Exchanger, the queue bound to this dead letter exchange is the dead letter queue.
10. How to ensure that the message is sent to RabbitMQ correctly?
RabbitMQ uses the sender confirmation mode to ensure that the message is sent to RabbitMQ correctly .
Sender confirmation mode: Set the channel to confirm mode (sender confirmation mode), then all messages posted on the channel will be assigned a unique ID. Once the message is delivered to the destination queue, or after the message is written to disk (a persistent message), the channel will send an acknowledgment to the producer (containing the message's unique ID).
If an internal error occurs in RabbitMQ and the message is lost, a nack (not acknowledged) message will be sent. The sender confirmation mode is asynchronous, and the producer application can continue to send messages while waiting for confirmation. When the confirmation message reaches the producer application, the callback method of the producer application will be
Trigger to process confirmation messages.
11. How to ensure that the message receiver consumes the message?
Receiver message confirmation mechanism: The consumer must confirm each message after receiving it (message reception and message confirmation are two different operations).
Only if the consumer confirms the message, RabbitMQ can safely delete the message from the queue. The timeout mechanism is not used here. RabbitMQ only confirms whether the message needs to be resent by interrupting the connection of the Consumer. In other words, as long as the connection is not interrupted, RabbitMQ gives the Consumer enough time to process the message.
Several special cases are listed below:
If the consumer receives the message, disconnects or unsubscribes before confirming it, RabbitMQ will think that the message has not been distributed, and then redistribute it to the next subscribed consumer. (There may be hidden dangers of repeated message consumption, which needs to be deduplicated according to bizId )
If the consumer receives the message but does not confirm the message, and the connection is not disconnected, RabbitMQ believes that the consumer is busy and will not distribute more messages to the consumer.
12. How to avoid repeated delivery or consumption of messages?
During message production, MQ internally generates an inner-msg-id for each message sent by the producer, as a basis for deduplication and idempotence (message delivery fails and retransmits), to avoid duplicate messages entering the queue; when the message is consumed , Requires a bizId in the message body (globally unique for the same business, such as payment ID, order ID, post ID, etc.)
As a basis for deduplication and idempotency, to avoid the same message being consumed repeatedly.
This question answers the following points for business scenarios:
1. For example, you get this message to do the insert operation of the database. That's easy. Make a unique primary key for this message. Even if repeated consumption occurs, it will lead to primary key conflicts and avoid dirty data in the database.
2. For another example, if you get this message to do the redis set operation, it will be easy and you don't need to solve it, because the result is the same no matter how many times you set, the set operation is originally considered to be an idempotent operation.
3. If the above two situations are not enough, go for the big move. Prepare a third-party medium for consumption records. Taking redis as an example, assign a global id to the message. Once the message has been consumed, write <id, message> to redis in KV form. Before consumers start to consume, first go to redis to check whether there is a consumption record.
13. How to solve the problem of losing data?
1. Producers lose data
What if the producer's message is not delivered to MQ? From the perspective of producers losing data, RabbitMQ provides transaction and confirm modes to ensure that producers do not lose messages.
The transaction mechanism means that before sending the message, open the transaction (channel.txSelect ()) and then send the message. If something abnormal occurs during the transmission process, the transaction will be rolled back (channel.txRollback ()). (channel.txCommit ()).
However, the disadvantage is that the throughput drops. Therefore, according to the experience of bloggers, the majority of confirm mode is used in production. Once the channel enters confirm mode, all messages posted on the channel will be assigned a unique ID (starting from 1). Once the message is delivered to all matching queues, rabbitMQ will send an Ack to
Produced by (a unique ID containing the message), which message has the right to know the manufacturers reach the destination queue if rabiitMQ could not process the message, will send a Nack message to you, you can perform a retry operation.
2. Message queue lost data
To deal with the loss of data in the message queue, the configuration of the persistent disk is generally turned on. This persistence configuration can be used in conjunction with the confirm mechanism. You can send an Ack signal to the producer after the message is persisted to disk . In this way, if the rabbitMQ is killed before the message is persisted to the disk, the producer will not receive the Ack signal and the producer will automatically resend it.
So how to persist, here by the way, it is actually very easy, just the following two steps:
① Set the durable identifier of the queue to true, which means that it is a persistent queue
② DeliveryMode = 2 when sending a message
After this setting, even if rabbitMQ hangs, the data can be restored after restarting. When the message has not been persisted to the hard disk, the service may have died. In this case, the mirrored queue can be introduced by introducing mirrored-queue, but there is no guarantee that the message is not lost 100% (the entire cluster is hung up)
3. Consumers lose data
Enabling manual confirmation mode can solve this problem
①In the automatic confirmation mode, the consumer hangs up, and the message to be ack returned to the queue. If the consumer throws an exception, the message will be retransmitted continuously until it is successfully processed. No message will be lost, even if the service hangs, the unprocessed message will be returned to the queue, but the exception will make the message retry continuously.
②Manual confirmation mode, if the consumer dies before it can be processed, a message will be repeatedly sent to other consumers when there is no response to the ack; if the listener handles the exception and the exception is not captured, the message will be received repeatedly, and then Always throw an exception; if the exception is caught, but not acked in finally, it will always be sent repeatedly
Message (retry mechanism).
③Unconfirmed mode, acknowledge = "none" does not use the confirmation mechanism, as soon as the message is sent, it will be removed from the queue immediately. No matter the client is abnormal or disconnected, it will be removed as soon as it is sent, and will not be resent.
14. Use of dead letter queue and delayed queue
Dead letter news:
The message is rejected (Basic.Reject or Basic.Nack) and the value of the requeue parameter is set to false
Message expired
The queue reaches the maximum length
Expired message:
There are two types of rabbitmq to set the expiration time of the message. The first one is to set the queue. After this setting, all the messages in the queue have the same expiration time. The second is to set the message itself. , Then the expiration time of each message is different. If these two methods are used at the same time, then the smaller the expiration time
The numerical value shall prevail. When the message reaches the expiration time and has not been consumed, then the message becomes a dead letter message.
Queue setting: use the x-message-ttl parameter when the queue is declared, in milliseconds
Single message setting: set the value of the expiration parameter of the message attribute, the unit is milliseconds
Delay queue: There is no delay queue in rabbitmq, but we can simulate the delay queue by setting the expiration time of the message and the dead letter queue. Consumers listen to the queue bound to the dead letter exchange, not to the queue where the message is sent.
With the above basic knowledge, we complete the following requirements:
demand:
The user creates an order in the system, and if the user does not make a payment over the time, the order is automatically cancelled.
analysis:
1. In the above situation, we are suitable to use delay queue to achieve, then how to create a delay queue
2. The delay queue can be timed by the expired message + dead letter queue
3. Expired messages are implemented by setting the x-message-ttl parameter in the queue
4. The dead letter queue sets the x-dead-letter-exchange parameter to the queue when the queue is declared, and then declares a queue to bind the switch corresponding to x-dead-letter-exchange.
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("127.0.0.1");
factory.setPort(AMQP.PROTOCOL.PORT);
factory.setUsername("guest");
factory.setPassword("guest");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
// 声明一个接收被删除的消息的交换机和队列
String EXCHANGE_DEAD_NAME = "exchange.dead";
String QUEUE_DEAD_NAME = "queue_dead";
channel.exchangeDeclare(EXCHANGE_DEAD_NAME, BuiltinExchangeType.DIRECT);
channel.queueDeclare(QUEUE_DEAD_NAME, false, false, false, null);
channel.queueBind(QUEUE_DEAD_NAME, EXCHANGE_DEAD_NAME, "routingkey.dead");
String EXCHANGE_NAME = "exchange.fanout";
String QUEUE_NAME = "queue_name";
// delete the message to a designated push switch corresponding routing key
channel.exchangeDeclare (EXCHANGE_NAME, BuiltinExchangeType.FANOUT);
Map <String, Object> arguments = new HashMap <String, Object> ();
// uniformly set the expiration time of all messages in the queue
arguments.put ("x-message- ttl ", 30000);
// Set the number of milliseconds when no consumers access the queue, delete the queue time
arguments.put (" x-expires ", 20000);
// Set the latest N messages in the queue, if it exceeds N items, the previous message will be removed from the queue
arguments.put ("x-max-length", 4);
// Set the maximum space of the queue content, if the threshold is exceeded, the previous message
arguments.put ( "x-max-length-bytes", 1024);
// Push the deleted message to the specified switch, generally x-dead-letter-exchange and x-dead-letter-routing-key need to set
arguments.put ( "x-dead-letter-exchange", "exchange.dead");
arguments.put ("x-dead-letter-routing-key", "routingkey.dead");
// Set the priority of the message, the priority of the priority is consumed
. 10);
channel.queueDeclare (QUEUE_NAME, false, false, false, arguments);
channel.queueBind (QUEUE_NAME, EXCHANGE_NAME, "");
String message = "Hello RabbitMQ:";
for (int i = 1; i <= 5 ; I ++) {
// expiration: setting an expiration time of a single message
AMQP.BasicProperties.Builder = new new AMQP.BasicProperties Properties () Builder ().
.priority (I) .expiration (I * 1000 + "");
channel.basicPublish (EXCHANGE_NAME, "", properties.build (), (message + i) .getBytes ("UTF-8"));
}
channel.close ();
connection.close ();
15. What are the disadvantages of using message queues?
1. System availability is reduced: you think, as long as other systems are running well, then your system is normal. Now you have to add a message queue, the message queue hangs, your system is not huh. Therefore, system availability is reduced
2. Increased system complexity: Many issues must be considered, such as consistency issues, how to ensure that messages are not repeatedly consumed, and how to ensure reliable transmission of messages. Therefore, there are more things to consider and the complexity of the system increases.