1, RabbitMQ What are the benefits?
1. decoupling, system A and system B directly invoke code in the code system C, D if the future access system, the system also need to modify the code A, too cumbersome!
2. asynchronous, writes messages to the message queue, the non-essential business logic operates in an asynchronous manner, to speed up response
3. The clipping, concurrent amount of time, all requests to the database directly dislike, resulting in connection failure database
2, RabbitMQ in broker What does it mean? cluster it means?
It refers to a broker or more logical grouping erlang node, and applications running on RabbitMQ node. broker cluster is based on the increase of the constraint between the share metadata node.
3, RabbitMQ concept in the channel, exchange and queue is a logical concept, or the corresponding process entity? What is the role each play?
queue has its own erlang process; internal exchange implemented as a lookup table stored binding relationship; channel routing entity actually work, that is responsible in accordance with routing_key the message delivered to the queue. AMQP protocol by the description that, channel real TCP connection on a virtual connection, all AMQP commands are sent through channel, and each channel has a unique ID. A channel can only be used a single OS thread, it is delivered to a message on a particular channel is in order. But a thread on the operating system allows multiple channel.
4. What vhost that? What is the role?
vhost can be understood as a virtual broker, namely mini-RabbitMQ server. Inside contain a separate queue, exchange and binding, etc., but the most important thing is that an independent authority system, the user can do to control vhost range. Of course, from a global perspective of RabbitMQ, vhost as a means of isolating different permissions (a typical example is different applications can run in different vhost).
5, the message transmission based on what?
Since the creation and destruction overhead of TCP connections larger, and the number of concurrent limited by system resources will cause performance bottlenecks. RabbitMQ way channel used to transmit data. Channel virtual connection is established in the real TCP connections, and the number of channels per TCP connection is not limited.
6, how to distribute the message?
If the queue has at least one consumer subscription, the message will be sent in the cycle (round-robin) fashion to consumers. Each message will only be distributed to a consumer subscription (provided that the consumer can properly process the message and confirm).
7, how the message routing?
Conceptually, the message routing must have three parts: switches, routing binding. Producers publish to the message switch; binding from the router determines how to route the message to a particular queue; final message queue, and the consumer receives.
When news release to the switch, the message will have a routing key (routing key), set in the message creation.
Routing keys through the queue, the queue can be bound to the exchanger.
After the message arrives at the switch, RabbitMQ will route the message queue routing keys and key match (with different rules for different routing switch). If the pattern matches to the queue, the message will be delivered to the respective queue; if not matched to any queue, messages will enter the "black hole."
Common exchanger is divided into about three types:
direct: If the routing keys exact match, the message is delivered to the appropriate queue
fanout: If the switch receives a message, it will broadcast to all queues bound
topic: messages from different sources can be made to reach the same queue. When using the topic exchanger can use wildcards, for example: "*" matches any text in a particular location, the routing keys divided into several parts, "#" matches all the rules. "." Special attention: topic sent to switch messages can not be arbitrarily set selection key (routing_key), it must be spaced apart by a series of identifiers consisting of. "."
8. What is metadata? Metadata is divided into what type? What is included? Metadata associated with the cluster What? Metadata is how to save? Metadata cluster in how it is distributed?
In the non-cluster mode, metadata is divided into metadata Queue (queue names and attributes, etc.), Exchange metadata (exchange name, type and attributes), Binding metadata (stored routing table lookup relationship), Vhost metadata (vhost within the range set for the first three names space constraints and security attributes). In the cluster mode, further comprising a node in the cluster node relationship information and the location information. Metadata determination is stored in RAM only, or also stored on the RAM disk, and according to the type of erlang node. Metadata is distributed in the whole node in the cluster.
FIG metadata distribution diagram in FIG queue at a single node cluster and two modes.
9, the statement queue in a single-node and multi-node cluster system consisting of the system, exchange, as well as binding what would be different?
A: When you declare a queue on a single node, as long as the node associated metadata has been changed, you get Queue.Declare-ok response; and declared queue on the cluster, the demand for a full node in the cluster must metadata successfully updated, you will get Queue.Declare-ok response. In addition, if the node type is RAM node is only changed data is stored in memory, but also to change the data stored on the disk if the type of disk node.
& Badmail dead letter queue exchanger: DLX Full (Dead-Letter-Exchange), referred badmail switch, when the message becomes a dead letter queue if the message where the presence of x-dead-letter-exchange parameter, then it is sent to the switch x-dead-letter-exchange corresponding value, the switch switches it is called dead letter, and the dead letter queue is bound exchanger dead letter queue.
10, how to ensure the right message is sent to RabbitMQ?
RabbitMQ confirmation using the transmission mode, to ensure that the message is correctly transmitted RabbitMQ. Transmission confirmation mode: to confirm the channel setting mode (transmission confirmation mode), all messages are posted channel is assigned a unique ID. Once the message is delivered to the destination queue, or after the message is written to disk (persistent message may be), the channel sends an acknowledgment to the producer (containing the unique message ID). If an internal error occurred causing RabbitMQ message is lost, it sends a nack (not acknowledged, unacknowledged) messages. Transmission confirmation pattern is asynchronous, while waiting for the application producer acknowledgment, can continue to send the message. When the confirmation message arrives producer application, the application callback method producers will be triggered to process the confirmation message.
11, how to ensure that the message recipient consume the news?
Mechanism acknowledgment message recipient: the consumer receives each message must be acknowledged (message reception and acknowledgment message are two different operating). Consumers only confirmed the news, RabbitMQ can safely delete the message from the queue. Here and did not use a timeout mechanism, RabbitMQ only to confirm that the need to re-send the message by connecting the Consumer interrupt. In other words, as long as the connection is not interrupted, RabbitMQ to the Consumer sufficient length of time to process the message.
The following lists some special circumstances:
If the consumer receives a message, disconnected or unsubscribe, RabbitMQ think that message was not distributed, then re-distributed to consumers under a subscription before confirming. (There may be a message repeated consumption of risks, need to re according to bizId)
if the consumer has not received the message confirmation message, the connection has not disconnected, the RabbitMQ think the consumer is busy, will not be distributed to the consumer more news.
12, how to avoid repeating the message delivered or repeated consumption?
When message production, internal MQ message generated for each transmission of a producer inner-msg-id, as the basis weight and to idempotent (and retransmits the message delivery failure), to avoid duplicate messages enqueued; message when the consumer required in the message body must have a bizId (for the same business globally unique, such as payment ID, order ID, message ID, etc.) as the basis for deduplication and power, etc., to avoid the same message is repeated consumption.
The problem for the business scenarios to answer the following points:
1. For example, do you get this message insert the database. It would be easy to give this message to make a unique primary key, even if the situation repeated consumption occurs, it will lead to a primary key conflict, to avoid the database appear dirty data.
2. Another example, do you get the news operation set the redis, it would be easy, not solve, because you set several times whether the results are the same, even if the power had set the operation and other operations.
3. If the above two conditions is not enough, the big move. Prepare a third-party media, do consumer records. In redis example, a global id assigned to the message, as long as the message consumer through the <id, message> redis written in the form of KV. Before consumers start spending it, go redis there is no consumption record query can be.
13, how to solve the problem of lost data?
1. producers lost data
The message is not delivered to the MQ producers in how to do? From this point of view the producer lost data, RabbitMQ provide transaction model to ensure and confirm message is not lost producers.
transaction mechanism that is to say, before sending the message, open things (channel.txSelect ()), then sends a message that if anything unusual occurs during sending, things will roll back (channel.txRollback ()), if it is sent successfully submit things (channel.txCommit ()).
However, the disadvantage is that the throughput decreases. Therefore, according to the bloggers experience with production mostly confirm mode. Once entering confirm channel mode, all published messages on the channel sides will be assigned a unique ID (starting at 1), once the message is delivered to all matching queue, sends an Ack to RabbitMQ producer ( the message contains a unique ID), which the manufacturers know the message has correctly reached a destination queue if rabiitMQ could not process the message, will send a Nack message to you, you can perform a retry operation.
2. Message Queue lost data
Where lost data message queue processing, typically is on persistent disks. This configuration can confirm persistence mechanism used in conjunction with, you can give the producer sends a message Ack signal after persistent disk. In this way, if the message before persistent disk, rabbitMQ killed, so producers can not receive Ack signal, the producers will be automatically re-issued.
So how persistent it, by the way here, in fact, very easy, just two steps below
①, the queue is a persistent identity durable set to true, represents a durable queue
②, when sending a message to deliveryMode = 2
After this set, rabbitMQ even hung up, after the restart can recover the data. When the message has not persisted to the hard drive, the service may have been dead, this situation by introducing a mirror that is mirrored-queue queue, but there is no guarantee the message is not lost hundred percent (the entire cluster will hang up)
3. Consumer lost data
Enable manual confirmation mode can solve this problem
① automatic acknowledgment mode, the consumer hang, ack message to be returned to the queue. Consumers exception is thrown, the message will continue to be retransmitted, until the process is successful. Messages are not lost messages, even if the service hang, no process is completed will return to the queue, but will make an exception message continues to try again.
② manual confirmation mode, if the consumer a chance to deal with it is to die, repeatedly sends no response when ack a message to other consumers; if a listener to handle exceptions, and no abnormal capture will repeat the message is received, then He has thrown an exception; if an exception were captured, but no ack in finally, and would send a message repeated (retry mechanism).
③ not acknowledged mode, acknowledge = "none" do not use acknowledgments, as long as the message transmission is completed in the queue will be removed immediately, or whether the client is disconnected abnormal, as long as it is sending the removed not retransmitted.
14, using a delay and dead letter queue queues
dead letter message:
Message is rejected (Basic.Reject or Basic.Nack) and arranged requeue parameter is false
message expired
queue maximum length
expired message:
Only two kinds of messages provided in the present rabbitmq the expiration time by the first set of queues, after such an arrangement, all of the messages in the queue are present in the same expiration time, a second set of the message itself by , then the expiration time of each message are not the same. If you use these two methods, it is subject to the expiration time that a small value. When the message reaches its expiration time has not been consumed, so the news has become a dead letter messages.
Queue settings: using x-message-ttl parameter when the queue stated in milliseconds
Setting a single message: the message is the value of the property parameter expiration milliseconds
Queue Delay: delay queue does not exist in the rabbitmq, but we can simulate the delay provided by the message queue and the expiration time of the dead letter queue. Consumers dead letter queue monitor switch binding, rather than listen queue messages sent.
With these basics, we meet the following requirements:
Demand: the user to create an order in the system, and if over time the user does not pay, then automatically cancel the order.
analysis:
1, the top of this situation, we will use for the delay queue to achieve, then how to create a queue delay
2, the delay may be caused by outdated message queue dead letter queue time +
3, expired messages provided x-message-ttl parameters achieved by the queue
4, dead letter queue by queue stated, is provided to the queue x-dead-letter-exchange parameters, and then additional binding declare a queue x-dead-letter-exchange corresponding to the switches.
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("127.0.0.1");
factory.setPort(AMQP.PROTOCOL.PORT);
factory.setUsername("guest");
factory.setPassword("guest");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
// 声明一个接收被删除的消息的交换机和队列
String EXCHANGE_DEAD_NAME = "exchange.dead";
String QUEUE_DEAD_NAME = "queue_dead";
channel.exchangeDeclare(EXCHANGE_DEAD_NAME, BuiltinExchangeType.DIRECT);
channel.queueDeclare(QUEUE_DEAD_NAME, false, false, false, null);
channel.queueBind(QUEUE_DEAD_NAME, EXCHANGE_DEAD_NAME, "routingkey.dead");
String EXCHANGE_NAME = "exchange.fanout";
Queue_name = String "queue_name";
channel.exchangeDeclare (EXCHANGE_NAME, BuiltinExchangeType.FANOUT);
the Map <String, Object> = new new arguments the HashMap <String, Object> ();
// for all messages in the queue unified set expiration time
arguments. PUT ( "the X--the Message-ttl", 30000);
// set how many milliseconds does not exceed the consumer to access the queue, the queue is deleted time
arguments.put ( "the X--the Expires", 20000);
// set new queue of N messages, if more than the N, the message from the queue in front of the removed off
arguments.put ( "X-max-length",. 4);
// set the maximum contents of a queue space exceeds the threshold value is deleted previous message
arguments.put ( "X-max-length-bytes", 1024);
// delete the message to a designated push switch, typically x-dead-letter-exchange and x-dead-letter-routing- key need to set
arguments.put ( "X-Dead-Letter-Exchange", "exchange.dead");
// delete the message to a designated push switch corresponding routing key
arguments.put ( "x-dead-letter-routing-key", "routingkey.dead");
// set the priority of the message, the priority is preferentially large consumption
arguments.put ( "X-max-priority", 10);
channel.queueDeclare (queue_name, to false, to false, to false, arguments);
channel.queueBind (queue_name , EXCHANGE_NAME, "");
String message = "the Hello RabbitMQ:";
for (int I =. 1; I <=. 5; I ++) {
// expiration: setting an expiration time of a single message
AMQP.BasicProperties.Builder properties = new AMQP .BasicProperties () Builder ().
.priority (I) .expiration (I * 1000 + "");
channel.basicPublish (EXCHANGE_NAME, "", properties.build (), (I + Message) .getBytes ( "UTF 8 "));
}
channel.close ();
Connection.close ();
15, using the message queue have any drawbacks?
1. reduce system availability: Do you think, ah, other systems would have just run properly, then you the system is normal. Now you have to give a message queue in, that message queue hung up, your system is not the Oh. Thus, reducing system availability
2. The system complexity increases: it wants to consider many issues, such as consistency, how to ensure that messages are not repeated consumption, how to ensure the message to ensure reliable transmission. Therefore, something needs to be considered more and system complexity increases.