Table of contents
1. Message confirmation function
1. Message confirmation function
Guaranteeing the reliability of messages mainly relies on three mechanisms: one is message persistence, the other is transaction mechanism, and the other is message confirmation mechanism.
1) Message persistence
Message persistence is to write messages to local files. If rabbitmq fails and exits, it will read queue data from the local file system when restarting.
2) Transaction mechanism
The transaction mechanism of rabbitmq provides the opening, committing and rolling back of the transaction between the message producer and the message server (broker) (as shown in the figure below). This mechanism can ensure the reliability of messages, but it also has disadvantages: the use of transaction mechanism will increase the number of interactions between message producers and brokers (servers), resulting in waste of performance, and the transaction mechanism is blocked, so you need to wait after sending a message RabbitMQ responds, and the next message can only be sent after getting the response, so the transaction mechanism is not recommended (the performance difference between RabbitMQ transaction mode and non-transaction mode can be as high as hundreds of times, the specific value varies with machine performance and network environment, but the difference will be very obvious)
Transaction submission process:
- The client requests the server to open a transaction (tx.select)
- The server returns a response to receive the start transaction (tx.select-ok)
- forward news
- The client requests to commit the transaction (tx.commit)
- The server commits the transaction and returns a response (tx.commit-ok)
3) Message Confirmation
Message confirmation is divided into: sender confirmation, receiver confirmation. The confirmation of the sender is divided into: the confirmation of the message arriving at the switch, and the confirmation of the message arriving at the queue bound to the switch.
2 Development example
Basic code for sample development:
git clone -b rabbitmqDemo [email protected]:heizifeng/rabbit-mqdemo.git2.1 Producer Confirmation
Because: each RabbitTemplate instance can only register one ConfirmCallback, so if you start the web container and call this method multiple times to send messages, an exception will be reported. (The test case can pass because the container terminates after each test is executed, and the next time it runs is a new container)
Add the configuration class of RabbitTemplate, and specify the message confirmation callback method in the configuration class:
package com.zking.rabbitmqdemo.provied.config;
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.rabbit.connection.ConnectionFactory;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.support.converter.Jackson2JsonMessageConverter;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
/**
*
* @site www.xiaomage.com
* @company xxx公司
* @create 2021-11-14 10:04
*/
@Configuration
@Slf4j
public class RabbitTemplateConfig {
@Bean
public RabbitTemplate createRabbitTemplate(ConnectionFactory connectionFactory) {
RabbitTemplate template = new RabbitTemplate(connectionFactory);
template.setMandatory(true);
template.setMessageConverter(new Jackson2JsonMessageConverter());
template.setEncoding("utf-8");
//实现消息发送到exchange后接收ack回调,publisher-confirms:true
//如果队列是可持久化的，则在消息成功持久化之后生产者收到确认消息
template.setConfirmCallback(((correlationData, ack, cause) -> {
if(ack) {
log.info("消息成功发送到exchange，id:{}", correlationData.getId());
} else {
/*
* 消息未被投放到对应的消费者队列，可能的原因：
* 1）发送时在未找到exchange，例如exchange参数书写错误
* 2）消息队列已达最大长度限制（声明队列时可配置队列的最大限制），此时
* 返回的cause为null。
*/
log.info("******************************************************");
log.info("11消息发送失败: {}", cause);
}
}));
//消息发送失败返回队列，publisher-returns:true
template.setMandatory(true);
//实现消息发送的exchange，但没有相应的队列于交换机绑定时的回调
template.setReturnCallback((message, replyCode, replyText, exchange, routingKey) -> {
String id = message.getMessageProperties().getCorrelationId();
log.info("消息：{} 发送失败, 应答码：{} 原因：{} 交换机: {} 路由键: {}", id, replyCode, replyText, exchange, routingKey);
});
return template;
}
}
Write a delay queue (dead letter) for sending messages
package com.zking.rabbitmqdemo.provied.config;
import org.springframework.amqp.core.*;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.HashMap;
import java.util.Map;
/**
*
* @site www.xiaomage.com
* @company xxx公司
* @create 2021-11-12 10:04
*/
@Configuration
public class RabbitMQConfig {
@Bean(name="directExchange")
public Exchange directExchange() {
return ExchangeBuilder.directExchange("direct_exchange").durable(true).build();
}
@Bean(name="directQueue")
public Queue directQueue() {
Map<String,Object> args = new HashMap<>();
args.put("x-message-ttl", 1000*60*20);
args.put("x-max-length", 3);
args.put("x-overflow","reject-publish");
return QueueBuilder.durable("direct_queue").withArguments(args).build();
}
@Bean
public Binding directBinding(
@Qualifier("directQueue") Queue queue,
@Qualifier("directExchange") Exchange exchange) {
return BindingBuilder
.bind(queue)
.to(exchange)
.with("direct_exchange_routing_key")
.noargs();
}
@Bean(name="topicExchange")
public Exchange topicExchange() {
return ExchangeBuilder
.topicExchange("topic_exchange")
.durable(true)
.build();
}
@Bean(name="topicQueue1")
public Queue topicQueue1() {
return QueueBuilder.durable("topic_queue_q1").build();
}
@Bean(name="topicQueue2")
public Queue topicQueue2() {
return QueueBuilder.durable("topic_queue_q2").build();
}
@Bean
public Binding topicBindingQ1(
@Qualifier("topicQueue1") Queue queue,
@Qualifier("topicExchange") Exchange exchange) {
return BindingBuilder
.bind(queue)
.to(exchange)
.with("topic.queue.#")
.noargs();
}
@Bean
public Binding topicBindingQ2(
@Qualifier("topicQueue2") Queue queue,
@Qualifier("topicExchange") Exchange exchange) {
return BindingBuilder
.bind(queue)
.to(exchange)
.with("topic.queue.#")
.noargs();
}
//死信队列
@Bean(name="dxlExchange")
public Exchange dxlExchange() {
return ExchangeBuilder.topicExchange("dxl_exchange").durable(true).build();
}
@Bean(name="dxlQueue")
public Queue dxlQueue() {
return QueueBuilder.durable("dxl_queue").build();
}
@Bean
public Binding bindingDxl(
@Qualifier("dxlQueue") Queue queue,
@Qualifier("dxlExchange") Exchange exchange) {
return BindingBuilder
.bind(queue)
.to(exchange)
.with("routing_dxl_key")
.noargs();
}
@Bean(name="usualExchange")
public Exchange usualExchange() {
return ExchangeBuilder.directExchange("usual_direct_exchange").durable(true).build();
}
@Bean(name="usualQueue")
public Queue usualQueue() {
return QueueBuilder.durable("usual_queue")
.withArgument("x-message-ttl", 1000*60*2)
.withArgument("x-max-length", 5)
.withArgument("x-dead-letter-exchange", "dxl_exchange")
.withArgument("x-dead-letter-routing-key", "routing_dxl_key")
.build();
}
@Bean
public Binding bindingUsualQueue(
@Qualifier("usualQueue") Queue queue,
@Qualifier("usualExchange") Exchange exchange) {
return BindingBuilder
.bind(queue)
.to(exchange)
.with("routing_usual_key")
.noargs();
}
}
Write a test Service to send messages
package com.zking.rabbitmqdemo.provied.service;
/**
* @author aq
* @site www.xiaomage.com
* @company xxx公司
* @create 2021-11-12 10:11
*/
public interface ISendMsgService {
void sendDirectMsg();
void topicExchangeSend();
void dxlExchangeSend();
void confirmMessage();
}
package com.zking.rabbitmqdemo.provied.service;
import org.springframework.amqp.rabbit.connection.CorrelationData;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import javax.annotation.Resource;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.UUID;
/**
* @author aq
* @site www.xiaomage.com
* @company xxx公司
* @create 2021-11-12 10:08
*/
@Service
public class SendMsgService implements ISendMsgService {
@Autowired
private RabbitTemplate rabbitTemplate;
@Override
public void sendDirectMsg() {
String msg = "rabbitmq direct exchange send msg "
+ LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
rabbitTemplate.convertAndSend("direct_exchange", "direct_exchange_routing_key",msg);
}
@Override
public void topicExchangeSend() {
String msg = "rabbitmq topic exchange send msg "
+ LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
rabbitTemplate.convertAndSend("topic_exchange", "topic.queue.msg", msg);
}
@Override
public void dxlExchangeSend() {
String msg = "rabbitmq usual exchange send msg "
+ LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
rabbitTemplate.convertAndSend("usual_direct_exchange", "routing_usual_key", msg);
}
@Override
public void confirmMessage() {
String msg = "rabbitmq direct exchange send msg and confirm "
+ LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
CorrelationData correlationData = new CorrelationData(UUID.randomUUID().toString());
rabbitTemplate.convertAndSend(
"direct.exchange",
"direct.exchange.routing.key",
msg,
correlationData);
}
}
Write test interface
package com.zking.rabbitmqdemo.provied.service;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;
import static org.junit.Assert.*;
/**
*
* @site www.xiaomage.com
* @company xxx公司
* @create 2021-11-12 10:12
*/
@RunWith(SpringRunner.class)
@SpringBootTest
public class SendMsgServiceTest {
@Autowired
private ISendMsgService sendMsgService;
@Test
public void sendDirectMsg() {
sendMsgService.sendDirectMsg();
}
@Test
public void topicExchangeSend() {
sendMsgService.topicExchangeSend();
}
@Test
public void testDxlExchange() {
sendMsgService.dxlExchangeSend();
}
}
2.2 Consumer Confirmation
1) Directly use @RabbitListener and @RabbitHandler annotations to configure the listening container through the configuration file:
application.properties
server.port=8084
#rabbitMQç¸å³éç½®
spring.rabbitmq.host=192.168.164.128
spring.rabbitmq.port=5672
spring.rabbitmq.username=admin
spring.rabbitmq.password=admin
spring.rabbitmq.virtual-host=my_vhost
#启用消息确认
spring.rabbitmq.listener.simple.acknowledge-mode=manual
spring.rabbitmq.listener.direct.acknowledge-mode=manual
#连接模式为channel
spring.rabbitmq.cache.connection.mode=channel
spring.rabbitmq.cache.channel.size=50
#每个队列的消费者数量
spring.rabbitmq.listener.direct.consumers-per-queue=2
#侦听器调用者线程的最小数量
spring.rabbitmq.listener.simple.concurrency=2
spring.rabbitmq.listener.simple.max-concurrency=100
#每次用队列中取出1个消息，在有多个消息消费者，且消息者处理能力不均时，可以
#起到均衡各消息消费者的处理功能的功能
spring.rabbitmq.listener.direct.prefetch=1
/**
* @author Administrator
* @create 2020-02-2422:30
*/
@Component
@Slf4j
public class ReceiverConfirmDemo implements ChannelAwareMessageListener {
/**
* 指定监听的队列.
* 该注解可以放在方法上，也可以放在类上，如果放在类上则需要
* 在一个方法上设置@RabbitHandler(isDefault=true)，否则会报如下异常：
* “Listener method ‘no match’ threw exception”。
* 因此建议注解始终使用在方法上。
*/
@RabbitListener(queues = "direct.queue")
//指定该方法为消息处理器
@RabbitHandler
@Override
public void onMessage(Message message, Channel channel) throws IOException {
log.info("消息内容: {}",new String(message.getBody()));
/**
* 模拟业务处理方法.
* 对应业务方法中出现的异常需要区分对待，例如以下情况：
* 1）网络异常等可能恢复的异常，可以设置消息重新返回到队列，以便于重新处理
* 2）对应业务数据等不可恢复的异常，则可以进行补偿操作，或放入死信队列进行人工干预
*/
try {
log.info("正在处理 ....");
//延迟5秒
TimeUnit.SECONDS.sleep(5);
long deliveryTag = message.getMessageProperties().getDeliveryTag();
//模拟在业务处理是发生了网络异常，如：在连接数据库保存数据时网络发生了抖动
//此类异常是可以恢复的，需要要消息重新返回队列，以便于下次处理
if(deliveryTag % 2 == 0) {
throw new ConnectException("模拟消息消费者发生网络异常");
}
//模拟发生不可恢复异常，此种情况消息重新入队没有意义
if(deliveryTag % 3 == 0) {
throw new ClassCastException("模拟消息消费者发生不可恢复异常");
}
} catch (SocketException se) {
log.info("SocketException: {}", se.getMessage());
//拒绝deliveryTag对应的消息，第二个参数是否requeue，true则重新入队列，false则不会重新入队
//如果配置了死信队列则消息会被投递到死信队列
channel.basicReject(message.getMessageProperties().getDeliveryTag(), true);
//不确认deliveryTag对应的消息，第二个参数是否应用于多消息，第三个参数是否requeue，
//与basic.reject区别就是同时支持多个消息，可以nack该消费者先前接收未ack的所有消息。
// nack后的消息也会被自己消费到
//channel.basicNack(message.getMessageProperties().getDeliveryTag(), false, true);
//是否恢复消息到队列，参数是是否requeue，true则重新入队列，
// 并且尽可能的将之前recover的消息投递给其他消费者消费，
//而不是自己再次消费。false则消息会重新被投递给自己
//channel.basicRecover(true);
return;
} catch (Exception e) {
//此处处理无法恢复的异常，可记录日志或将消息发送到指定的队列以便于后续的处理
log.info("Exception: {}", e.getMessage());
channel.basicReject(message.getMessageProperties().getDeliveryTag(), false);
return;
}
log.info("处理完毕， 发送ack确认 .... ");
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
}
}
After the message consumer is written, the message will be sent through the message producer to check the consumption of the message consumer.
2) Use SimpleMessageListenerContainer to configure the listening container
/**
* @author Administrator
* @create 2020-03-0119:27
*/
@Configuration
@Slf4j
public class MessageListenerContainer {
@Autowired
private ReceiverConfirmDemo receiverConfirmDemo;
@Bean
public SimpleMessageListenerContainer simpleMessageListenerContainer(ConnectionFactory connectionFactory){
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setConcurrentConsumers(1);
container.setAcknowledgeMode(AcknowledgeMode.MANUAL);
container.setQueueNames("direct.queue");
//后置处理器，接收到的消息都添加了Header请求头
/*container.setAfterReceivePostProcessors(message -> {
message.getMessageProperties().getHeaders().put("desc",10);
return message;
});*/
container.setMessageListener(receiverConfirmDemo);
return container;
}
}
The receiverConfirmDemo injected into this class is the ReceiverConfirmDemo message processor that has been written above.
operation result
If there is another problem, if it is a network abnormality and the customer restores the data, it will return to the original queue. If it is an unhandled exception, it can be saved in the database and manually intervened in the process.