Summary of two implementation methods of rabbitmq delay queue

ps: Delay queue in the article = Delay queue

What is a delay queue

The object stored in the delay queue must be the corresponding delayed message. The so-called "delayed message" means that when the message is sent, the consumer does not want to get the message immediately, but waits for the specified time before the consumer gets this The message is consumed.

Scenario 1: In the order system, a user usually has 30 minutes to pay after placing an order. If the payment is not successful within 30 minutes, the order will be processed. This is to use the delayed queue to send order information to the delayed queue.

Scenario 2: The user wants to remotely control the smart device at home through a mobile phone to work at a specified time. At this time, the user instruction can be sent to the delay queue, and the instruction will be pushed to the smart device when the time set by the instruction is up.

---

How RabbitMQ implements late queue

method one

The AMQP protocol and RabbitMQ queue do not directly support the delay queue function, but the function of the delay queue can be simulated by the following features.
But we can use the two characteristics of RabbitMQ to curve the delay queue:

• 特性1、Time To Live(TTL)

RabbitMQ can set x-expires for Queue or x-message-ttl for Message to control the lifetime of the message. If it times out (the two are set at the same time, the first expiration time shall prevail), the message becomes a dead letter ( Dead letter)

RabbitMQ has two methods to set the expiration time of messages in the queue.

• A: Through the queue property setting, all messages in the queue have the same expiration time.
• B: Set the message individually, the TTL of each message can be different.

If used at the same time, the expiration time of the message is based on the smaller TTL between the two. Once the lifetime of a message in the queue exceeds the set TTL value, it becomes a dead letter

• 特性2、Dead Letter Exchanges（DLX）

RabbitMQ's Queue can be configured with x-dead-letter-exchange and x-dead-letter-routing-key (optional) two parameters. If a dead letter appears in the queue, it will be rerouted and forwarded to the specified one according to these two parameters. queue.

• x-dead-letter-exchange: resend the dead letter to the designated exchange after the dead letter appears
• x-dead-letter-routing-key: After the dead letter appears, the dead letter will be sent again according to the specified routing-key

The dead letter in the queue includes:

• The TTL of the message or queue expired

• The queue reaches the maximum length

• The message was rejected by the consumer (basic.reject or basic.nack) and requeue=false

Combining the above two features, after setting the TTL rule, when a message becomes a dead letter in a queue, it can be re-forwarded to another Exchange or Routing Key using the DLX feature, and the message can be consumed again.

Setting method:

Step 1: Set TTL to generate dead letters. There are two ways: Per-Message TTL and Queue TTL. The first one can set an expiration time for each message, which can be used in most scenarios. The second is to set the expiration time for the queue and apply. For one-time delayed tasks

There are other ways to generate dead letters, such as consumers who refuse to consume basic.reject or basic.nack (provided that the consumer's attribute requeue=false)

• Per-Message TTL (set an expiration time for each message) ( official document )

The java client sends a message to the queue that can only stay for 60 seconds:

byte[] messageBodyBytes = "Hello, world!".getBytes();
AMQP.BasicProperties properties = new AMQP.BasicProperties();
properties.setExpiration("60000");//设置消息的过期时间为60秒
channel.basicPublish("my-exchange", "routing-key", properties, messageBodyBytes);
//这条消息发送到相应的队列之后，如果60秒内没有被消费，则变为死信

• Queue TTL (set an expiration time for the entire queue)

Create a queue, the message expiration time of the queue is 30 minutes (the queue will be deleted if there is no consumer consumption message within 30 minutes, and the message in the queue will become a dead letter after deletion)

java client方式：

Map<String, Object> args = new HashMap<String, Object>();
args.put("x-expires", 1800000);
channel.queueDeclare("myqueue", false, false, false, args);

rabbitmqctl命令方式（.* 为所有队列， 可以替换为指定队列）：
rabbitmqctl set_policy expiry ".*" '{"expires":1800000}' --apply-to queues

rabbitmqctl (Windows):
rabbitmqctl set_policy expiry ".*" "{""expires"":1800000}" --apply-to queues

Step 2: Set the forwarding rules for dead letters (if there are no rules, then directly discard the dead letters)

• Dead Letter Exchanges setting method ( official document )

Java Client方式：
//声明一个直连模式的exchange
channel.exchangeDeclare("some.exchange.name", "direct");
//声明一个队列，当myqueue队列中有死信产生时，会转发到交换器some.exchange.name
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-dead-letter-exchange", "some.exchange.name");

//如果设置死信会以路由键some-routing-key转发到some.exchange.name，如果没设默认为消息发送到本队列时用的routing key
//args.put("x-dead-letter-routing-key", "some-routing-key");
channel.queueDeclare("myqueue", false, false, false, args);

命令行方式（.* 为所有队列， 可以替换为指定队列）：
设置 "dead-letter-exchange"
rabbitmqctl：
rabbitmqctl set_policy DLX ".*" '{"dead-letter-exchange":"my-dlx"}' --apply-to queues
rabbitmqctl (Windows)：
rabbitmqctl set_policy DLX ".*" "{""dead-letter-exchange"":""my-dlx""}" --apply-to queues
设置 "dead-letter-routing-key"
rabbitmqctl：
rabbitmqctl set_policy DLX ".*" '{ "dead-letter-routing-key":"my-routing-key"}' --apply-to queues
rabbitmqctl (Windows)：
rabbitmqctl set_policy DLX ".*" "{""dead-letter-routing-key"":""my-routing-key""}" --apply-to queues

Method Two

Rabbitmq 3.5.7 and above provides a plug-in (rabbitmq-delayed-message-exchange) to implement the delayed queue function. At the same time, the plugin depends on Erlang/OPT 18.0 and above.

Plug-in source code address:
https://github.com/rabbitmq/rabbitmq-delayed-message-exchange

Plug-in download link:
https://bintray.com/rabbitmq/community-plugins/rabbitmq_delayed_message_exchange

installation:

Enter the plug-in installation directory
{rabbitmq-server}/plugins/ (you can check the existing plug-ins) to
download the plug-in
rabbitmq_delayed_message_exchange

wget https://bintray.com/rabbitmq/community-plugins/download_file?file_path=rabbitmq_delayed_message_exchange-0.0.1.ez

• 1

(If the downloaded file name is irregular, rename it manually, such as
rabbitmq_delayed_message_exchange-0.0.1.ez)

Enable plugin

rabbitmq-plugins enable rabbitmq_delayed_message_exchange

(关闭插件)
rabbitmq-plugins disable rabbitmq_delayed_message_exchange

 Plug-in use

Use the delayed-messaging feature by declaring an exchange of type
x-delayed-message. x-delayed-message is a type provided by the plugin, not rabbitmq itself

// ... elided code ...
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-delayed-type", "direct");
channel.exchangeDeclare("my-exchange", "x-delayed-message", true, false, args);
// ... more code ...

When sending a message, add the "x-delay" parameter to the header to control the delay time of the message

// ... elided code ...
byte[] messageBodyBytes = "delayed payload".getBytes("UTF-8");
Map<String, Object> headers = new HashMap<String, Object>();
headers.put("x-delay", 5000);
AMQP.BasicProperties.Builder props = new AMQP.BasicProperties.Builder().headers(headers);
channel.basicPublish("my-exchange", "", props.build(), messageBodyBytes);
// ... more code ...

Use example:

Message sender:

import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

public class Send {
	// 队列名称
	private final static String EXCHANGE_NAME="delay_exchange";
	private final static String ROUTING_KEY="key_delay";

	@SuppressWarnings("deprecation")
	public static void main(String[] argv) throws Exception {
		/**
		 * 创建连接连接到MabbitMQ
		 */
		ConnectionFactory factory = new ConnectionFactory();
		factory.setHost("192.168.12.190");
		factory.setUsername("admin");
		factory.setPassword("admin");
		Connection connection = factory.newConnection();
		Channel channel = connection.createChannel();

		SimpleDateFormat sf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");

		// 声明x-delayed-type类型的exchange
		Map<String, Object> args = new HashMap<String, Object>();
		args.put("x-delayed-type", "direct");
		channel.exchangeDeclare(EXCHANGE_NAME, "x-delayed-message", true,
				false, args);


		Map<String, Object> headers = new HashMap<String, Object>();
		//设置在2016/11/04,16:45:12向消费端推送本条消息
		Date now = new Date();
		Date timeToPublish = new Date("2016/11/04,16:45:12");

		String readyToPushContent = "publish at " + sf.format(now)
				+ " \t deliver at " + sf.format(timeToPublish);

		headers.put("x-delay", timeToPublish.getTime() - now.getTime());

		AMQP.BasicProperties.Builder props = new AMQP.BasicProperties.Builder()
				.headers(headers);
		channel.basicPublish(EXCHANGE_NAME, ROUTING_KEY, props.build(),
				readyToPushContent.getBytes());

		// 关闭频道和连接
		channel.close();
		connection.close();
	}
}

Message receiver:

import java.text.SimpleDateFormat;
import java.util.Date;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.QueueingConsumer;

public class Recv {

	// 队列名称
	private final static String QUEUE_NAME = "delay_queue";
	private final static String EXCHANGE_NAME="delay_exchange";

	public static void main(String[] argv) throws Exception,
			java.lang.InterruptedException {
		ConnectionFactory factory = new ConnectionFactory();
		factory.setHost("192.168.12.190");
		factory.setUsername("admin");
		factory.setPassword("admin");
		Connection connection = factory.newConnection();
		Channel channel = connection.createChannel();

		QueueingConsumer queueingConsumer = new QueueingConsumer(channel);

		channel.queueDeclare(QUEUE_NAME, true,false,false,null);
		channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "");
		channel.basicConsume(QUEUE_NAME, true, queueingConsumer);
		SimpleDateFormat sf=new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");
		try {
			System.out.println("****************WAIT***************");
			while(true){
				QueueingConsumer.Delivery delivery = queueingConsumer
						.nextDelivery(); //

				String message = (new String(delivery.getBody()));
				System.out.println("message:"+message);
				System.out.println("now:\t"+sf.format(new Date()));
			}

		} catch (Exception exception) {
			exception.printStackTrace();

		}

	}
}

Start the receiving end and start the sending end.
Operation result:

****************WAIT***************
message:publish at 2016-11-04 16:44:16.887 	 deliver at 2016-11-04 16:45:12.000
now:	2016-11-04 16:45:12.023

 The result shows that we received the message on 2016-11-04 16:45:12.023, and there is a 23 millisecond delay from the time we set 2016-11-04 16:45:12.023

Note: The number of messages sent to the queue when using the rabbitmq-delayed-message-exchange plug-in may not be visible in the web management interface and does not affect normal function usage

Note: During use, it was discovered that when a RAM node with the rabbitmq-delayed-message-exchange plug-in is enabled, it cannot be started when restarted. Checking the log, a Timeout exception was found. The developer explained that this is the node during the startup process. Synchronizing cluster-related data causes startup timeout, and it is recommended not to use Ram nodes

插件开发者:
RAM nodes start blank and need a disk node to sync tables from. In this case it times out.

More importantly, you don’t need RAM nodes. If you’re not sure if you do, you certainly don’t, as don’t 99% of users.