Article directory
foreword
The emergence of MQ further reduces the degree of coupling between microservice modules. Compared with synchronous communication, it reduces the waiting time of related services and makes the delivery of messages more changeable and flexible. No matter what, as long as it is integrated by Spring, it will change
. It is very simple, and RabbitMQ is no exception.
Using SpringAMQP to realize message sending and receiving, there is no need to configure connection parameters repeatedly, which solves some "hard-coded" problems. It can be said that it is very similar to the integration of JDBC with MyBatis.
In the past, using native RabbitMQ to send and receive messages is like this:
after using SpringAMQP, sending and receiving messages is like this:
this is a basic queue (Basic-Queue)
as you can see, just introduce dependencies spring-boot-starter-amqp, write yml configuration files, establish connections, create The work of the channel has been done by Spring for us, and all we have to do is to use the tool class to send and monitor messages, which can be said to be quite convenient!
For different scenarios, we need to use different queue models:
1. WorkQueue (work queue)
For a single consumer situation (simple queue), when the producer sends 50 messages per second and the consumer processes 40 messages per second, there will be 10 more messages per second that cannot be processed, resulting in overproduction As a result, messages are accumulated in the queue. Once the upper limit of the queue memory is reached, new incoming messages cannot be processed and are discarded.
In order to improve the speed of message processing and avoid the accumulation of messages in the queue, the queue can be bound to multiple consumers, that is, the WorkQueue is generally
designed in this way for the convenience of observing the console:
Producer:
@Test
public void testSendMessage2WorkQueue() throws InterruptedException {
String queueName = "work.queue";
String message = "hello, MQQ";
for (int i = 1; i <= 50; i++) {
rabbitTemplate.convertAndSend(queueName, message + i);
Thread.sleep(20);
}
}
Consumers:
In order to simulate real scenarios as much as possible (consumers have different capabilities to process messages), set the sleep parameters of the two consumers to two different times
@RabbitListener(queues = "work.queue")
public void listenWorkQueue1(String msg) throws InterruptedException {
System.out.println("消费者1接收到消息——【" + msg + "】" + "At "+LocalTime.now());
Thread.sleep(20);
}
@RabbitListener(queues = "work.queue")
public void listenWorkQueue2(String msg) throws InterruptedException {
System.err.println("消费者2接收到消息——【" + msg + "】" + "At "+ LocalTime.now());
Thread.sleep(200);
}
message prefetch mechanism
Observing the console after running, I found that it took almost five seconds to process all the messages. Obviously, this kind of efficiency is very low: Why is the speed of
binding two consumers to consume messages slow instead of fast?
If you observe the console carefully, you will find that 50 messages are evenly distributed, and two consumers consume messages with even and odd ids respectively. It seems that consumers with poor processing capabilities are holding back (200msX25=5000ms=5s ) , why is this happening?
This is because MQ has a message prefetching mechanism, that is, consumers will get the channel of the message in advance before processing, and then process the message one by one. This process is isolated from the message processing!
If there are still people who don’t understand, just think about how we process messages when using native RabbitMQ:
when the callback function is executed, it is possible that the message will not be processed, and the program will continue to execute downwards, and the processing will start after a while Messages (in fact, I think this is also a place that reflects RabbitMQ's asynchronous )
mechanism is the key to ensure asynchronicity. By artificially setting parameters, the method of message prefetching can also be adjusted to ensure processing efficiency, just like so:
listener:
simple:
prefetch: 1
The prefetch parameter is used to ensure the number of messages consumers get each time, and the next batch of messages can only be obtained after the processing is completed. After the
data prefetch is set, the consumer processes all the messages within one second:
it can be seen that for The consumer settings in WorkQueue need to implement the "allocation according to work" strategy to be more perfect.
After using the work queue WorkQueue, the efficiency of processing messages has been greatly improved, and there will be no message accumulation.
2.Publish&Subscribe (publish-subscribe)
For the simple queue and work queue models, the producer publishes a message, and once the consumer consumes it, the message will be destroyed. In this way, it is impossible to send a message to multiple consumers at the same time.
For a microservice project, in the payment order model, when the payment service is completed, a message will be sent to notify the SMS service and the order service at the same time... This requires high availability of the message, and the message cannot be destroyed after a service consumes it, causing other The service cannot receive the message.
How to send the same message to multiple consumers and let them receive it? Using the publish & subscribe working model,
messages can be routed to different queues through the exchange, and then consumers can consume the messages in their respective subscription queues. There
are different publishing strategies for different types of exchanges:
1. Fanout (broadcast)
SpringAMQPA provides an API for declaring exchanges, queues, and binding relationships.
Therefore, using the implementation class under the Exchange interface can route messages to each bound Queue, and the code will become very simple. Declare the exchange and bind it Queue can:
@Bean
public FanoutExchange fanoutExchange(){
return new FanoutExchange("yu7.fanout");
}
// fanout.queue1
@Bean
public Queue fanoutQueue1(){
return new Queue("fanout.queue1");
}
// 绑定队列1到交换机
@Bean
public Binding fanoutBinding1(Queue fanoutQueue1, FanoutExchange fanoutExchange){
return BindingBuilder
.bind(fanoutQueue1)
.to(fanoutExchange);
}
// fanout.queue2
@Bean
public Queue fanoutQueue2(){
return new Queue("fanout.queue2");
}
// 绑定队列2到交换机
@Bean
public Binding fanoutBinding2(Queue fanoutQueue2, FanoutExchange fanoutExchange){
return BindingBuilder
.bind(fanoutQueue2)
.to(fanoutExchange);
}
Note: The switch can only be used as a forwarding route for messages, not as a storage for messages. Once the route fails, the message will be lost!
2. DirectExchange (routing)
DirectExchange will route the received message to the specified Queue according to the rules. When the producer publishes the message, the RoutingKey of the specified message matches the bindingKey declared by the consumer, so as to achieve "precise guidance"
. To complete the configuration, it can be completed with one click through the @RabbitListener annotation, so there is no need to use configuration classes at all:
Producer:
@Test
public void testSendDirectExchange() {
// 交换机名称
String exchangeName = "yu7.direct";
// 消息
String message = "hello, MQ!";
// 发送消息
rabbitTemplate.convertAndSend(exchangeName, "A", message);
}
consumer:
@RabbitListener(bindings = @QueueBinding(
value = @Queue(name = "direct.queue1"),
exchange = @Exchange(name = "yu7.direct", type = ExchangeTypes.DIRECT),
key = {
"A", "B"}
))
public void listenDirectQueue1(String msg){
System.out.println("消费者接收到direct.queue1的消息：【" + msg + "】");
}
@RabbitListener(bindings = @QueueBinding(
value = @Queue(name = "direct.queue2"),
exchange = @Exchange(name = "yu7.direct", type = ExchangeTypes.DIRECT),
key = {
"A", "C"}
))
public void listenDirectQueue2(String msg){
System.out.println("消费者接收到direct.queue2的消息：【" + msg + "】");
}
When the bindingKey of the queue is the same, it becomes a broadcast model
3.TopicExchange (topic)
Topic is very similar to Direct. It allows RoutingKey-BindingKey to be matched in the form of wildcards, so that more targeted routing and subscription to more messages can be made, and the situation where multiple BindingKeys were used in the past can now be solved with only one:
#: represents 0 or more words
*: refers to a word
Advantages of MQ
1. Low coupling: every time there is a new demand, you only need to add the corresponding subscription.
2. Throughput improvement: each processes the events subscribed by itself, and does not need to wait for the execution to be completed before releasing resources.
3. Fault isolation: because there is no strong Dependence, if there is a problem in a certain link in the middle, it will not affect the entire process
4. Traffic peak shaving: MQ is like a pipeline. When a large number of requests come, you all line up for me and execute them sequentially