Article directory
foreword
Basic learning about RabbitMQ1. What is RabbiMQ?
RabbitMQ is a messaging middleware: it accepts and forwards messages. You can think of it as a courier site. When you want to send a package, you put your package in the courier station, and the courier will eventually deliver your courier to the recipient. According to this logic, RabbitMQ is a courier Station, a courier will deliver the express for you. The main difference between RabbitMQ and the express station is that it does not process express mail but receives, stores and forwards message data.
2. What problems can it solve?
- Flow peak elimination
举个例子，如果订单系统最多能处理一万次订单，这个处理能力应付正常时段的下单时绰绰有余，正
常时段我们下单一秒后就能返回结果。但是在高峰期，如果有两万次下单操作系统是处理不了的，只能限
制订单超过一万后不允许用户下单。使用消息队列做缓冲，我们可以取消这个限制，把一秒内下的订单分
散成一段时间来处理，这时有些用户可能在下单十几秒后才能收到下单成功的操作，但是比不能下单的体
验要好
- .Application decoupling
以电商应用为例，应用中有订单系统、库存系统、物流系统、支付系统。用户创建订单后，如果耦合
调用库存系统、物流系统、支付系统，任何一个子系统出了故障，都会造成下单操作异常。当转变成基于
消息队列的方式后，系统间调用的问题会减少很多，比如物流系统因为发生故障，需要几分钟来修复。在
这几分钟的时间里，物流系统要处理的内存被缓存在消息队列中，用户的下单操作可以正常完成。当物流
系统恢复后，继续处理订单信息即可，中单用户感受不到物流系统的故障，提升系统的可用性。
- asynchronous processing
有些服务间调用是异步的，例如 A 调用 B，B 需要花费很长时间执行，但是 A 需要知道 B 什么时候可以执行
完，以前一般有两种方式，A 过一段时间去调用 B 的查询 api 查询。或者 A 提供一个 callback api， B 执行完
之后调用 api 通知 A 服务。这两种方式都不是很优雅，使用消息总线，可以很方便解决这个问题，A 调用 B 服
务后，只需要监听 B 处理完成的消息，当 B 处理完成后，会发送一条消息给 MQ，MQ 会将此消息转发给 A 服
务。这样 A 服务既不用循环调用 B 的查询 api，也不用提供 callback api。同样B 服务也不用做这些操作。A 服
务还能及时的得到异步处理成功的消息。
Three, four core
Producer, Consumer, Exchange, Queue
4. Working principle
noun introduction
- Broker : an application for receiving and distributing messages, RabbitMQ Server is Message Broker
- Virtual host : Designed for multi-tenancy and security factors, the basic components of AMQP are divided into a virtual group, similar to the namespace concept in the network. When multiple different users use the services provided by the same RabbitMQ server, multiple vhosts can be divided, and each user creates an exchange/queue in his own vhost, etc.
- Connection : TCP connection between publisher/consumer and broker
- Channel :: If a Connection is established every time RabbitMQ is accessed, the overhead of establishing a TCPConnection when the message volume is large will be huge and the efficiency will be low. Channel is a logical connection established inside the connection. If the application supports multi-threading, usually each thread creates a separate channel for communication. AMQP method includes the channel id to help the client and message broker identify the channel, so the channels are completely isolated of. Channel as a lightweight
- Exchange : The message arrives at the first stop of the broker, according to the distribution rules, matches the routing key in the query table, and distributes the message to the queue. Commonly used types are: direct (point-to-point), topic (publish-subscribe) and fanout
(multicast) - Queue : The message is finally sent here and waits for the consumer to pick it up
- Binding : A virtual connection between exchange and queue. Binding can contain routing key. Binding information is saved in the query table in exchange for the basis of message distribution.
5. Work Queues
polling
When the same group of consumers consumes messages in the same queue, the polling method is used by default.
unfair distribution
If you need to use unfair distribution, you need to configure it on the consumer side: those who can do more work, and the worker thread with good performance will seize the message.
The consumer side must set the automatic confirmation autoAck to false, and basicQos will be effective.
6. Message response
Consumers may take a while to complete a task, what happens if one of the consumers processes a long task and only completes part of it and suddenly it dies. RabbitMQ marks a message for deletion as soon as it delivers it to a consumer. In this case, suddenly a consumer dies and we lose the messages we were processing. and subsequent messages sent to the consumer because it was unable to receive them.
In order to ensure that the message is not lost during the sending process, rabbitmq introduces a message response mechanism. The message response is: after the consumer receives the message and processes the message, it tells rabbitmq that it has been processed, and rabbitmq can delete the message.
There are two main options for message acknowledgment:
-
Automatic response (unsafe, you can’t handle it when it’s wrong) ------> Fast processing efficiency (the problem is: if there is a connection on the consumer’s side or the channel is closed, then the message will be lost, and too many messages will cause memory loss run out)
-
Answer manually (the problem that needs to be solved is: the message will not be lost)
Channel.basicAck (for positive acknowledgment)
Channel.basicNack (for negative acknowledgment)
Channel.basicReject (for negative acknowledgment) ----> Do not process the message. Discard directlymultiple : true and false , there are multiple messages to be confirmed in one cannel, if true, unanswered messages will be confirmed to receive the message and reply
false is more secure, but the efficiency is not as high as true
Set manual answer:
set whether to answer in batches:
7. RabbitMQ Persistence
queue persistence
D: Identify the queue persistence
code settings:
If there is already a non-persistent queue, an error will be reported if it is declared again
message persistence
(There is no absolute guarantee that the message will not be lost)
8. Release Confirmation
concept
Acts between the producer and mq.
The producer sets the channel to confirm mode. Once the channel enters the confirm mode, all
messages will be assigned a unique ID (starting from 1). Once the message is After posting to all matching queues, the broker will send a confirmation to the producer (including the unique ID of the message), which makes the producer know that the message has arrived at the destination queue correctly. If the message and the queue are durable, then The confirmation message will be sent after the message is written to the disk. The delivery-tag field in the confirmation message returned by the broker to the producer contains the serial number of the confirmation message. In addition, the broker can also set the multiple field of basic.ack to indicate this sequence All messages prior to the number have been processed.
Strategy
Production settings:
single confirmation
Synchronous - safe, but slow
public static void publishMessageSingle() throws Exception {
//耗时136643毫秒
Channel channel = RabbitMqUtils.getChannel();
//队列的声明
String queueName = UUID.randomUUID().toString();
channel.queueDeclare(queueName,true,false,false,null);
//开启发布确认
channel.confirmSelect();
//开始时间
long begin = System.currentTimeMillis();
//批量发消息
for (int i = 0; i < MESSAGE_COUNT; i++) {
String message = i + "";
channel.basicPublish("",queueName,null,message.getBytes());
//单个消息就马上进行确认
boolean flag = channel.waitForConfirms();
if(flag) {
System.out.println("消息发送成功");
}
}
//结束时间
long end = System.currentTimeMillis();
System.out.println("发布" + MESSAGE_COUNT + "个单独确认消息，耗时" + (end-begin) + "毫秒");
}
batch confirmation
The confirmation speed is fast -----> It is impossible to know what message failed to send
public static void publishMessageBatch() throws Exception {
//耗时1711毫秒
Channel channel = RabbitMqUtils.getChannel();
//队列的声明
String queueName = UUID.randomUUID().toString();
channel.queueDeclare(queueName,true,false,false,null);
//开启发布确认
channel.confirmSelect();
//开始时间
long begin = System.currentTimeMillis();
//批量确认消息大小
int batchSize = 100;
//批量发消息，批量发布确认
for (int i = 1; i <= MESSAGE_COUNT; i++) {
String message = i + "";
channel.basicPublish("",queueName,null,message.getBytes());
//判断达到100条消息的时候，批量确认一次
if(i % batchSize == 0){
boolean flag = channel.waitForConfirms();
if(flag) {
System.out.println("消息发送成功");
}
}
}
//结束时间
long end = System.currentTimeMillis();
System.out.println("发布" + MESSAGE_COUNT + "个批量确认消息，耗时" + (end-begin) + "毫秒");
}
Asynchronous confirmation release
Good performance, good control in case of errors
//异步发布确认
public static void publishMessageAsync() throws Exception {
Channel channel = RabbitMqUtils.getChannel();
//队列的声明
String queueName = UUID.randomUUID().toString();
channel.queueDeclare(queueName,true,false,false,null);
//开启发布确认
channel.confirmSelect();
//开始时间
long begin = System.currentTimeMillis();
/**
* 线程安全有序的一个哈希表，适用于高并发的情况下
* 1.轻松的将序号于消息进行关联
* 2.轻松的批量删除条目，只要给到序号
* 3.支持高并发
*/
ConcurrentSkipListMap<Long, String> outstandingConfirms = new ConcurrentSkipListMap<>();
//消息确认成功回调函数
ConfirmCallback ackCallback = (deliveryTag, multiple) -> {
//=====2.删除掉已经确认的消息，剩下的就是未确认的消息=====
if(multiple) {
ConcurrentNavigableMap<Long, String> confirmd =
outstandingConfirms.headMap(deliveryTag);
} else {
outstandingConfirms.remove(deliveryTag);
}
System.out.println("确认的消息：" + deliveryTag);
};
/**
* 消息确认失败回调函数
* 1.消息的标识
* 2.是否为批量确认
*/
ConfirmCallback nackCallback = (deliveryTag, multiple) -> {
//耗时274毫秒
//=====3.打印一下未确认的消息都有哪些=====
String message = outstandingConfirms.get(deliveryTag);
System.out.println("未确认的消息是：" + message + "未确认的消息tag：" + deliveryTag);
};
/**
* 准备消息的监听器，监听哪些消息成功了，哪些消息失败了
* 1.消息发送成功的监听对象
* 2.消息发送失败的监听对象
*/
channel.addConfirmListener(ackCallback,nackCallback);//异步通知
for (int i = 1; i <= MESSAGE_COUNT; i++) {
String message = i + "";
channel.basicPublish("",queueName,null,message.getBytes());
//======1.此处记录下所有要发送的消息，消息的总和=====
outstandingConfirms.put(channel.getNextPublishSeqNo(),message);
}
//结束时间
long end = System.currentTimeMillis();
System.out.println("发布" + MESSAGE_COUNT + "个异步确认消息，耗时" + (end-begin) + "毫秒");
}