Time-To-Live and Expiration — RabbitMQ
1. Dead letter queue
Dead Letter Exchanges — RabbitMQ
Dead letter queue:
The full name of DLX (Dead-Letter-Exchange) is called a dead letter exchange. When a message becomes a dead letter, if the queue where the message is located has the x-dead-letter-exchange parameter, it will be sent to x On the switch corresponding to the value of -dead-letter-exchange, this switch is called a dead letter switch, and the queue bound to this dead letter switch is a dead letter queue
Dead letter message:
- The message is rejected (Basic.Reject or Basic.Nack) and the value of the requeue parameter is set to false
- Message expiration (message TTL expires. TTL : short for Time To Live, ie expiration time)
- Queue reaches maximum length
Expiration message:
There are 2 methods in rabbitmq to set the expiration time of the message:
- The first is by setting the queue . After this setting, all messages in the queue have the same expiration time
- The second is by setting the message itself , so the expiration time of each message is different
If these two methods are used at the same time, the value with the shorter expiration time shall prevail. When the message reaches the expiration time and has not been consumed, then that message becomes a dead letter message
Queue setting : use the ** x-message-ttl ** parameter when declaring the queue, in milliseconds;
- The setting of this attribute in the queue is valid only when the queue is declared for the first time. If the queue already exists at the beginning and does not have this attribute, the queue must be deleted and declared again.
- The TTL of the queue can only be set to a fixed value, once set it cannot be changed, otherwise an exception will be thrown
Single message setting : it is to set the value of the expiration parameter of the message attribute, and the unit is milliseconds.
illustrate:
For the first method of setting queue properties, once the message expires, it will be erased from the queue; in the second method, even if the message expires, it will not be erased from the queue immediately, because whether each message expires is determined just before delivery to the consumer
1. Producer:
When declaring the queue, use the attribute to specify the name of the dead letter queue switch.
test:
package rabbitmq;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.BuiltinExchangeType;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
public class Producer {
public static ConnectionFactory getConnectionFactory() {
// 创建连接工程，下面给出的是默认的case
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("192.168.99.100");
factory.setPort(5672);
factory.setUsername("guest");
factory.setPassword("guest");
factory.setVirtualHost("/");
return factory;
}
public static void main(String[] args) throws IOException, TimeoutException {
ConnectionFactory connectionFactory = getConnectionFactory();
Connection newConnection = null;
Channel createChannel = null;
try {
newConnection = connectionFactory.newConnection();
createChannel = newConnection.createChannel();
// 声明一个正常的direct类型的交换机
createChannel.exchangeDeclare("order.exchange", BuiltinExchangeType.DIRECT);
// 声明死信交换机为===order.dead.exchange
String dlxName = "order.dead.exchange";
createChannel.exchangeDeclare(dlxName, BuiltinExchangeType.DIRECT);
// 声明队列并指定死信交换机为上面死信交换机
Map<String, Object> arg = new HashMap<String, Object>();
arg.put("x-dead-letter-exchange", dlxName);
createChannel.queueDeclare("myQueue", true, false, false, arg);
String message = "测试消息";
createChannel.basicPublish("order.exchange", "routing_key_myQueue", null, message.getBytes());
System.out.println("消息发送成功");
} catch (Exception e) {
e.printStackTrace();
} finally {
if (createChannel != null) {
createChannel.close();
}
if (newConnection != null) {
newConnection.close();
}
}
}
}result:
(1) Generate two Exchanges
(2) The dead letter queue of queue myQueue has attributes
2. Consumers:
One consumer listens to the normal queue, and one consumer listens to the dead letter queue. (only the bound switches are different)
Consumer 1: Listen to the normal queue
package rabbitmq;
import java.io.IOException;
import java.util.Date;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.DefaultConsumer;
import com.rabbitmq.client.Envelope;
import com.rabbitmq.client.AMQP.BasicProperties;
public class Consumer {
public static ConnectionFactory getConnectionFactory() {
// 创建连接工程，下面给出的是默认的case
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("192.168.99.100");
factory.setPort(5672);
factory.setUsername("guest");
factory.setPassword("guest");
factory.setVirtualHost("/");
return factory;
}
public static void main(String[] args) throws IOException, TimeoutException {
ConnectionFactory connectionFactory = getConnectionFactory();
Connection newConnection = null;
Channel createChannel = null;
try {
newConnection = connectionFactory.newConnection();
createChannel = newConnection.createChannel();
// 队列绑定交换机-channel.queueBind(队列名, 交换机名, 路由key[广播消息设置为空串])
createChannel.queueBind("myQueue", "order.exchange", "routing_key_myQueue");
createChannel.basicConsume("myQueue", false, "", new DefaultConsumer(createChannel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, BasicProperties properties,
byte[] body) throws IOException {
System.out.println("consumerTag: " + consumerTag);
System.out.println("envelope: " + envelope);
System.out.println("properties: " + properties);
String string = new String(body, "UTF-8");
System.out.println("接收到消息： -》 " + string);
long deliveryTag = envelope.getDeliveryTag();
Channel channel = this.getChannel();
System.out.println("拒绝消息， 使之进入死信队列");
System.out.println("时间： " + new Date());
try {
TimeUnit.SECONDS.sleep(3);
} catch (InterruptedException e) {
}
// basicReject第二个参数为false进入死信队列或丢弃
channel.basicReject(deliveryTag, false);
}
});
} catch (Exception e) {
e.printStackTrace();
} finally {
}
}
}Consumer 2: Listen to the dead letter queue
package rabbitmq;
import java.io.IOException;
import java.util.Date;
import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.DefaultConsumer;
import com.rabbitmq.client.Envelope;
import com.rabbitmq.client.AMQP.BasicProperties;
public class Consumer2 {
public static ConnectionFactory getConnectionFactory() {
// 创建连接工程，下面给出的是默认的case
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("192.168.99.100");
factory.setPort(5672);
factory.setUsername("guest");
factory.setPassword("guest");
factory.setVirtualHost("/");
return factory;
}
public static void main(String[] args) throws IOException, TimeoutException {
ConnectionFactory connectionFactory = getConnectionFactory();
Connection newConnection = null;
Channel createChannel = null;
try {
newConnection = connectionFactory.newConnection();
createChannel = newConnection.createChannel();
// 队列绑定交换机-channel.queueBind(队列名, 交换机名, 路由key[广播消息设置为空串])
createChannel.queueBind("myQueue", "order.dead.exchange", "routing_key_myQueue");
createChannel.basicConsume("myQueue", false, "", new DefaultConsumer(createChannel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, BasicProperties properties,
byte[] body) throws IOException {
System.out.println("时间： " + new Date());
System.out.println("consumerTag: " + consumerTag);
System.out.println("envelope: " + envelope);
System.out.println("properties: " + properties);
String string = new String(body, "UTF-8");
System.out.println("接收到消息： -》 " + string);
long deliveryTag = envelope.getDeliveryTag();
Channel channel = this.getChannel();
channel.basicAck(deliveryTag, true);
System.out.println("死信队列中处理完消息息");
}
});
} catch (Exception e) {
e.printStackTrace();
} finally {
}
}
}Result: Consumer 1 listens normally, basicReject is false and enters the dead letter queue after rejecting; consumer 2 listens to the dead letter queue to receive the message.
The log output by the consumer is as follows:
consumerTag: amq.ctag-0noHs24F0FsGe-dfwwqWNw
envelope: Envelope(deliveryTag=1, redeliver=false, exchange=order.exchange, routingKey=routing_key_myQueue)
properties: #contentHeader<basic>(content-type=null, content-encoding=null, headers=null, delivery-mode=null, priority=null, correlation-id=null, reply-to=null, expiration=null, message-id=null, timestamp=null, type=null, user-id=null, app-id=null, cluster-id=null)
接收到消息： -》 测试消息
拒绝消息， 使之进入死信队列
时间： Sat Nov 07 12:18:44 CST 2020The log output by consumer 2 is as follows:
时间： Sat Nov 07 12:18:47 CST 2020
consumerTag: amq.ctag-ajYMpMFkXHDiYWkD3XFJ7Q
envelope: Envelope(deliveryTag=1, redeliver=false, exchange=order.dead.exchange, routingKey=routing_key_myQueue)
properties: #contentHeader<basic>(content-type=null, content-encoding=null, headers={x-death=[{reason=rejected, count=1, exchange=order.exchange, time=Sat Nov 07 01:52:19 CST 2020, routing-keys=[routing_key_myQueue], queue=myQueue}]}, delivery-mode=null, priority=null, correlation-id=null, reply-to=null, expiration=null, message-id=null, timestamp=null, type=null, user-id=null, app-id=null, cluster-id=null)
接收到消息： -》 测试消息
死信队列中处理完消息息Notice:
After entering the dead letter queue, the headers add some dead letter related information, including the original queue and the reason for entering the dead letter.
Supplement: Before the queue enters the dead letter queue, its routingKey can also be modified, and the following attributes can only be modified under the premise of specifying x-dead-letter-exchange, otherwise an error will be reported
(1) Modify the way the producer declares the queue, as follows:
// 声明一个正常的direct类型的交换机
createChannel.exchangeDeclare("order.exchange", BuiltinExchangeType.DIRECT);
// 声明死信交换机为===order.dead.exchange
String dlxName = "order.dead.exchange";
createChannel.exchangeDeclare(dlxName, BuiltinExchangeType.DIRECT);
// 声明队列并指定死信交换机为上面死信交换机
Map<String, Object> arg = new HashMap<String, Object>();
arg.put("x-dead-letter-exchange", dlxName);
// 修改进入死信队列的routingkey，如果不修改会使用默认的routingKey
arg.put("x-dead-letter-routing-key", "routing_key_myQueue_dead");
createChannel.queueDeclare("myQueue", true, false, false, arg);(2) Modify consumer 2 listening to the dead letter queue:
// 队列绑定交换机-channel.queueBind(队列名, 交换机名, 路由key[广播消息设置为空串])
createChannel.queueBind("myQueue", "order.dead.exchange", "routing_key_myQueue_dead");As a result, the information received by consumer 2 is as follows:
时间： Sat Nov 07 12:27:08 CST 2020
consumerTag: amq.ctag-THqpEdYH_-iNeCIccgpuaw
envelope: Envelope(deliveryTag=1, redeliver=false, exchange=order.dead.exchange, routingKey=routing_key_myQueue_dead)
properties: #contentHeader<basic>(content-type=null, content-encoding=null, headers={x-death=[{reason=rejected, count=1, exchange=order.exchange, time=Sat Nov 07 02:00:41 CST 2020, routing-keys=[routing_key_myQueue], queue=myQueue}]}, delivery-mode=null, priority=null, correlation-id=null, reply-to=null, expiration=null, message-id=null, timestamp=null, type=null, user-id=null, app-id=null, cluster-id=null)
接收到消息： -》 测试消息
死信队列中处理完消息Two, delay queue
Delay queue, that is, the message will not be consumed immediately after entering the queue, but will be consumed only after reaching the specified time
RabbitMQ itself does not provide a delay queue, we can use the survival time of the message and the dead letter queue to achieve delay
A typical application scenario is to close the order without payment within 30 minutes. There is also a scenario where the bill is not confirmed within 24 hours, and a reminder message is sent
Delay queue plugin installation
2.1.1, yml configuration
spring:
rabbitmq:
host: 192.168.99.12
port: 5672
username: guest
password: guest
# 发送确认
publisher-confirms: true
# 路由失败回调
publisher-returns: true
template:
# 必须设置成true 消息路由失败通知监听者，false 将消息丢弃
mandatory: true
#消费端
listener:
simple:
# 每次从RabbitMQ获取的消息数量
prefetch: 1
default-requeue-rejected: false
# 每个队列启动的消费者数量
concurrency: 1
# 每个队列最大的消费者数量
max-concurrency: 1
# 签收模式为手动签收-那么需要在代码中手动ACK
acknowledge-mode: manual
#邮件队列
email:
queue:
name: demo.email
#邮件交换器名称
exchange:
name: demoTopicExchange
#死信队列
dead:
letter:
queue:
name: demo.dead.letter
exchange:
name: demoDeadLetterTopicExchange
#延时队列
delay:
queue:
name: demo.delay
exchange:
name: demoDelayTopicExchange
2.1.2. Delay queue configuration
/**
* rabbitmq 配置
*
* @author DUCHONG
* @since 2020-08-23 14:05
**/
@Configuration
@Slf4j
public class RabbitmqConfig {
@Value("${email.queue.name}")
private String emailQueue;
@Value("${exchange.name}")
private String topicExchange;
@Value("${dead.letter.queue.name}")
private String deadLetterQueue;
@Value("${dead.letter.exchange.name}")
private String deadLetterExchange;
@Value("${delay.queue.name}")
private String delayQueue;
@Value("${delay.exchange.name}")
private String delayExchange;
@Bean
public Queue emailQueue() {
Map<String, Object> arguments = new HashMap<>(2);
// 绑定死信交换机
arguments.put("x-dead-letter-exchange", deadLetterExchange);
// 绑定死信的路由key
arguments.put("x-dead-letter-routing-key", deadLetterQueue+".#");
return new Queue(emailQueue,true,false,false,arguments);
}
@Bean
TopicExchange emailExchange() {
return new TopicExchange(topicExchange);
}
@Bean
Binding bindingEmailQueue() {
return BindingBuilder.bind(emailQueue()).to(emailExchange()).with(emailQueue+".#");
}
//私信队列和交换器
@Bean
public Queue deadLetterQueue() {
return new Queue(deadLetterQueue);
}
@Bean
TopicExchange deadLetterExchange() {
return new TopicExchange(deadLetterExchange);
}
@Bean
Binding bindingDeadLetterQueue() {
return BindingBuilder.bind(deadLetterQueue()).to(deadLetterExchange()).with(deadLetterQueue+".#");
}
//延时队列
@Bean
public Queue delayQueue() {
return new Queue(delayQueue);
}
@Bean
CustomExchange delayExchange() {
Map<String, Object> args = new HashMap<>();
args.put("x-delayed-type", "topic");
//参数二为类型：必须是x-delayed-message
return new CustomExchange(delayExchange, "x-delayed-message", true, false, args);
}
@Bean
Binding bindingDelayQueue() {
return BindingBuilder.bind(delayQueue()).to(delayExchange()).with(delayQueue+".#").noargs();
}
}
2.2. Message sender
30 minutes is too long, here is a delay of 2 minutes to see the effect
@Configuration
@EnableScheduling
@Slf4j
public class ScheduleController {
@Autowired
RabbitTemplate rabbitTemplate;
@Value("${exchange.name}")
private String topicExchange;
@Value("${delay.exchange.name}")
private String delayTopicExchange;
@Scheduled(cron = "0 0/1 * * * ?")
public void sendEmailMessage() {
String msg = RandomStringUtils.randomAlphanumeric(8);
JSONObject email=new JSONObject();
email.put("content",msg);
email.put("to","[email protected]");
CorrelationData correlationData=new CorrelationData(UUID.randomUUID().toString());
rabbitTemplate.convertAndSend(topicExchange,"demo.email.x",email.toJSONString(),correlationData);
log.info("---发送 email 消息---{}---messageId---{}",email,correlationData.getId());
}
@Scheduled(cron = "0 0/1 * * * ?")
public void sendDelayOrderMessage() throws Exception{
//订单号 id实际是保存订单后返回的，这里用uuid代替
String orderId = UUID.randomUUID().toString();
// 模拟订单信息
JSONObject order=new JSONObject();
order.put("orderId",orderId);
order.put("goodsName","vip充值");
order.put("orderAmount","99.00");
CorrelationData correlationData=new CorrelationData(orderId);
MessageProperties messageProperties = new MessageProperties();
messageProperties.setMessageId(orderId);
//30分钟时间太长，这里延时120s消费
messageProperties.setHeader("x-delay", 120000);
Message message = new Message(order.toJSONString().getBytes(CharEncoding.UTF_8), messageProperties);
rabbitTemplate.convertAndSend(delayTopicExchange,"demo.delay.x",message,correlationData);
log.info("---发送 order 消息---{}---orderId---{}",order,correlationData.getId());
//睡一会，为了看延迟效果
TimeUnit.MINUTES.sleep(10);
}
}
2.3. Message consumer
@Component
@Slf4j
public class MessageHandler {
/**
* 邮件发送
* @param message
* @param channel
* @param headers
* @throws IOException
*/
@RabbitListener(queues ="demo.email")
@RabbitHandler
public void handleEmailMessage(Message message, Channel channel, @Headers Map<String,Object> headers) throws IOException {
try {
String msg=new String(message.getBody(), CharEncoding.UTF_8);
JSONObject jsonObject = JSON.parseObject(msg);
jsonObject.put("messageId",headers.get("spring_returned_message_correlation"));
log.info("---接受到消息---{}",jsonObject);
//主动异常
int m=1/0;
//手动签收
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}
catch (Exception e) {
log.info("handleEmailMessage捕获到异常,拒绝重新入队---消息ID---{}", headers.get("spring_returned_message_correlation"));
//异常，ture 重新入队,或者false,进入死信队列
channel.basicNack(message.getMessageProperties().getDeliveryTag(),false,false);
}
}
/**
* 死信消费者，自动签收开启状态下，超过重试次数，或者手动签收，reject或者Nack
* @param message
*/
@RabbitListener(queues = "demo.dead.letter")
public void handleDeadLetterMessage(Message message, Channel channel,@Headers Map<String,Object> headers) throws IOException {
//可以考虑数据库记录，每次进来查数量，达到一定的数量，进行预警，人工介入处理
log.info("接收到死信消息:---{}---消息ID---{}", new String(message.getBody()),headers.get("spring_returned_message_correlation"));
//回复ack
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}
/**
* 延时队列消费
* @param message
* @param channel
* @param headers
* @throws IOException
*/
@RabbitListener(queues ="demo.delay")
@RabbitHandler
public void handleOrderDelayMessage(Message message, Channel channel, @Headers Map<String,Object> headers) throws IOException {
try {
String msg=new String(message.getBody(), CharEncoding.UTF_8);
JSONObject jsonObject = JSON.parseObject(msg);
log.info("---接受到订单消息---orderId---{}",message.getMessageProperties().getMessageId());
log.info("---订单信息---order---{}",jsonObject);
//业务逻辑，根据订单id获取订单信息，如果还未支付，设置关闭状态，如果已支付，不做任何处理
//手动签收
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}
catch (Exception e) {
log.info("handleOrderDelayMessage捕获到异常,重新入队---orderId---{}", headers.get("spring_returned_message_correlation"));
//异常，ture 重新入队,或者false,进入死信队列
channel.basicNack(message.getMessageProperties().getDeliveryTag(),false,true);
}
}
}
2.4. Results
The running results show that the message with the same order number is received by consumers only 2 minutes after sending, which is in line with expectations
https://www.cnblogs.com/geekdc/p/13550620.html
Message Queue RabbitMQ (5): Dead Letter Queue and Delay Queue
RabbitMQ's Dead Letter Queue and Delay Queue - Programmer Sought
RabbitMQ Dead Letter Queue and Delay Queue_51CTO Blog_rabbitmq Delay Queue