Messaging middleware --RabbitMQ (eight) advanced features all here! (At)

Foreword

Previous messaging middleware --RabbitMQ (seven) advanced features all here! (On) we introduced the 消息如何保障100%的投递成功？, 幂等性概念详解, 在海量订单产生的业务高峰期，如何避免消息的重复消费的问题？, Confirm确认消息、Return返回消息. This we have to introduce the following content.

• Custom consumers
• The message flow restrictor (to prevent too much memory, node goes down)
• ACK message queue and return
• TTL news
• Dead Letter Queue

1. Custom consumers

1.1 Consumer-side custom monitor

We usually just write code while loop, a consumer.nextDelivery method to obtain the next message, and then processed for consumption!

But this certainly is not good in rotation the way, the code is relatively low.

• We use a custom Consumer more convenient, more strong decoupling, it is the most common use in practical work!

1.2 code shows

1.2.1 Producers

/**
 * 
* @ClassName: Producer 
* @Description: 生产者
* @author Coder编程
* @date2019年7月30日 下午23:15:51 
*
 */
public class Producer {

	
	public static void main(String[] args) throws Exception {
		
		//1 创建ConnectionFactory
		Connection connection = ConnectionUtils.getConnection();
		Channel channel = connection.createChannel();
		
		String exchange = "test_consumer_exchange";
		String routingKey = "consumer.save";
		
		String msg = "Hello RabbitMQ Consumer Message";
		
		for(int i =0; i<5; i ++){
			channel.basicPublish(exchange, routingKey, true, null, msg.getBytes());
		}
		
	}
}


复制代码

1.2.2 Consumers

/**
 * 
* @ClassName: Consumer 
* @Description: 消费者
* @author Coder编程
* @date2019年7月30日 下午23:13:51 
*
 */
public class Consumer {

	
	public static void main(String[] args) throws Exception {
		
		
		// 创建ConnectionFactory
		Connection connection = ConnectionUtils.getConnection();
		Channel channel = connection.createChannel();
		
		
		String exchangeName = "test_consumer_exchange";
		String routingKey = "consumer.#";
		String queueName = "test_consumer_queue";
		
		channel.exchangeDeclare(exchangeName, "topic", true, false, null);
		channel.queueDeclare(queueName, true, false, false, null);
		channel.queueBind(queueName, exchangeName, routingKey);
		
		//实现自己的MyConsumer()
		channel.basicConsume(queueName, true, new MyConsumer(channel));
	}
}


复制代码

1.2.3 custom class: MyConsumer

/**
 * 
* @ClassName: MyConsumer 
* @Description: TODO
* @author Coder编程
* @date 2019年7月30日 下午23:11:55 
*
 */
public class MyConsumer extends DefaultConsumer {


	public MyConsumer(Channel channel) {
		super(channel);
	}

	//根据需求，重写自己需要的方法。
	@Override
	public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
		System.err.println("-----------consume message----------");
		//消费标签
		System.err.println("consumerTag: " + consumerTag);
		//这个对象包含许多关键信息
		System.err.println("envelope: " + envelope);
		System.err.println("properties: " + properties);
		System.err.println("body: " + new String(body));
	}


}


复制代码

1.3 Printing results

2. limiting the consumer side

2.1 What is limiting consumer side?

• Suppose a scene, first of all, we have Rabbitmq server unprocessed messages over ten thousand, we just opened a consumer clients, will appear the following conditions:
• A huge amount of instant messages all push over, but we have a single client can not handle so much data at the same time! This time it is easy to cause the server to crash, fail.

Why not be limiting the production side of it?

Because in the case of high concurrency, customer volume is very large, it is difficult to do at the end of production restrictions. So we can do with the MQ limiting the consumer end.

• RabbitMQ provides a qos (quality of service) function, that is in a non-automatic acknowledgment messages on the premise that if a certain number of messages not acknowledged before, is not new news consumption (by value-based channel set or consume the Qos) . In the case of current limiting, do not set up automatic receipt, to be set to manual sign .
• void BasicQos(uint prfetchSize,ushort prefetchCount,bool global);

Parameter Description: prefetchSize: 0 prefetchCount: Do not tell RabbitMQ to push a consumer more than N messages, where once there are N news has not ack, then the consumer will block off until a message ack. global: true \ false if the above settings to the channel, simply put, is above the limit is channel level or consumer level. prefetchSize and global two, rabbitmq not achieved, for the time being not in the case of research prefetch_count no_ask = false entry into force, that is, in the case of these two values automatically answer is not in force.

The first argument: size limit, the message is the message the number of megabytes. Generally do not limit, the second parameter is set to 0: a maximum of how many treatment, the actual work is set to 1 like the third argument: what is limiting the application of policy. In general there are two applications RabbitMQ levels: level 1 2.Consumer channel. Usually set to false, true representation channel level, false representation in consumer level

2.2 code shows

2.2.1 Producers

/**
 * 
* @ClassName: Producer 
* @Description: 生产者
* @author Coder编程
* @date2019年7月30日 下午23:15:51 
*
 */
public class Producer {

	
	public static void main(String[] args) throws Exception {
		
		//1 创建ConnectionFactory
		Connection connection = ConnectionUtils.getConnection();
		Channel channel = connection.createChannel();
		
		String exchange = "test_qos_exchange";
		String routingKey = "qos.save";
		
		String msg = "Hello RabbitMQ QOS Message";
		
		for(int i =0; i<5; i ++){
			channel.basicPublish(exchange, routingKey, true, null, msg.getBytes());
		}
		
	}
}


复制代码

2.2.2 Consumers

/**
 * 
* @ClassName: Consumer 
* @Description: 消费者
* @author Coder编程
* @date2019年7月30日 下午23:13:51 
*
 */
public class Consumer {

	
	public static void main(String[] args) throws Exception {
		
		
		//1 创建ConnectionFactory
		Connection connection = ConnectionUtils.getConnection();
		Channel channel = connection.createChannel();
		
		
		String exchangeName = "test_qos_exchange";
		String queueName = "test_qos_queue";
		String routingKey = "qos.#";
		
		channel.exchangeDeclare(exchangeName, "topic", true, false, null);
		channel.queueDeclare(queueName, true, false, false, null);
		channel.queueBind(queueName, exchangeName, routingKey);
		
		//1 限流方式  第一件事就是 autoAck设置为 false
		//设置为1，表示一条一条数据处理
		channel.basicQos(0, 1, false);
		
		channel.basicConsume(queueName, false, new MyConsumer(channel));
		
		
	}
}


复制代码

2.2.3 custom class: MyConsumer

/**
 * 
* @ClassName: MyConsumer 
* @Description: TODO
* @author Coder编程
* @date 2019年7月30日 下午23:11:55 
*
 */
public class MyConsumer extends DefaultConsumer {


	private Channel channel ;
	
	public MyConsumer(Channel channel) {
		super(channel);
		this.channel = channel;
	}

	@Override
	public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
		System.err.println("-----------consume message----------");
		System.err.println("consumerTag: " + consumerTag);
		System.err.println("envelope: " + envelope);
		System.err.println("properties: " + properties);
		System.err.println("body: " + new String(body));
		
		//需要做签收,false表示不支持批量签收
		channel.basicAck(envelope.getDeliveryTag(), false);
		
	}


}


复制代码

2.2.4 Test Results

Let commented: channel.basicAck (envelope.getDeliveryTag (), false); then start the Consumer. View Exchange

View Queues

Then start Producer. View print the results:

We will find the consumer side, just received a message. Why is this?

The first point because we are in the consumer

channel.basicConsume(queueName, false, new MyConsumer(channel));
复制代码

The second parameter is set to false to manually sign.

The second point is disposed qos only accept a message. If this message is not a response to the Broker Ack, then the Broker will think you have not done this message consumer, it will not continue to send the message.

channel.basicQos(0, 1, false);
复制代码

Control station can look, unack = 1, Ready = 4, total = 5.

---

Next we open comment channel.basicAck (envelope.getDeliveryTag (), false); receipt of messages. Restart the service.

3.1 Printing results

We can see the result of normal print five

4. The consumer side and back ACK queue

4.1 consumer side Hand ACK and NACK

When the consumer side consume, and if we can make an exception due to the traffic log, and then compensate!

If serious problems due to server failure, then we need to manually ACK to protect the consumer end consumer success!

4.2 consumer side back to the queue

Return to the consumer end of the queue for the message is not processed successfully, the message is redelivered to the Broker!

Generally, we in the practical application, will be closed to return to the queue, which is set to False.

4.3 code shows

4.3.1 Producers

/**
 * 
* @ClassName: Producer 
* @Description: 生产者
* @author Coder编程
* @date2019年7月30日 下午23:15:51 
*
 */
public class Producer {

	
	public static void main(String[] args) throws Exception {
		
		//1创建ConnectionFactory
		Connection connection = ConnectionUtils.getConnection();
		Channel channel = connection.createChannel();
		
		String exchange = "test_ack_exchange";
		String routingKey = "ack.save";
		
		
		
		for(int i =0; i<5; i ++){
			
			Map<String, Object> headers = new HashMap<String, Object>();
			headers.put("num", i);
			
			//添加属性，后续会使用到
			AMQP.BasicProperties properties = new AMQP.BasicProperties.Builder()
					.deliveryMode(2) //投递模式，持久化
					.contentEncoding("UTF-8")
					.headers(headers)
					.build();
			String msg = "Hello RabbitMQ ACK Message " + i;
			channel.basicPublish(exchange, routingKey, true, properties, msg.getBytes());
		}
		
	}
}


复制代码

4.3.2 Consumers

/**
 * 
* @ClassName: Consumer 
* @Description: 消费者
* @author Coder编程
* @date2019年7月30日 下午23:13:51 
*
 */
public class Consumer {

	
	public static void main(String[] args) throws Exception {
		
		
		//1创建ConnectionFactory
		Connection connection = ConnectionUtils.getConnection();
		Channel channel = connection.createChannel();
		
		
		String exchangeName = "test_ack_exchange";
		String queueName = "test_ack_queue";
		String routingKey = "ack.#";
		
		channel.exchangeDeclare(exchangeName, "topic", true, false, null);
		channel.queueDeclare(queueName, true, false, false, null);
		channel.queueBind(queueName, exchangeName, routingKey);
		
		// 手工签收 必须要关闭 autoAck = false
		channel.basicConsume(queueName, false, new MyConsumer(channel));
		
		
	}
}


复制代码

4.3.3 custom class: MyConsumer

/**
 * 
* @ClassName: MyConsumer 
* @Description: TODO
* @author Coder编程
* @date 2019年7月30日 下午23:11:55 
*
 */
public class MyConsumer extends DefaultConsumer {


	private Channel channel ;
	
	public MyConsumer(Channel channel) {
		super(channel);
		this.channel = channel;
	}

	@Override
	public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
		System.err.println("-----------consume message----------");
		System.err.println("body: " + new String(body));
		try {
			Thread.sleep(2000);
		} catch (InterruptedException e) {
			e.printStackTrace();
		}
		if((Integer)properties.getHeaders().get("num") == 0) {
			//Nack三个参数  第二个参数：是否是批量，第三个参数：是否重回队列（需要注意可能会发生重复消费，造成死循环）
			channel.basicNack(envelope.getDeliveryTag(), false, true);
		} else {
			channel.basicAck(envelope.getDeliveryTag(), false);
		}
		
	}


}


复制代码

5.1 Print Results:

Note: You can see the return queue will cause problems with repeated consumption cycle of death, this time the best set number of retries, for example, after more than three times, the message or the consumer fails, the message will be discarded.

6. TTL queue / message

6.1 TTL

• Time To Live TTL is the abbreviation, that is, the survival time
• RabbitMQ supports message expiration time, when the message is sent can be specified
• RabbitMQ supports the expiration time of the queue, the message from the queue counted, as long as more than the timeout queue configuration, the message will automatically clear

6.2 code shows

6.2.1 Direct presentation by control units

Created by a queue control units

x-max-length of the maximum queue size x-message-ttl set for 10 seconds, if the message has not been consumed, then, will be cleared.

Add exchange

Queue and Exchange bind

Click test_ttl_exchange bind

To see if the binding is successful

By sending a message control table

Untreated message is automatically cleared

Production side set an expiration time

AMQP.BasicProperties properties = new AMQP.BasicProperties.Builder()
				.deliveryMode(2)
				.contentEncoding("UTF-8")
				.expiration("10000")
				.headers(headers)
				.build();

复制代码

These two properties are not the same, it corresponds to a message body, a queue corresponding to the expired.

7. dead letter queue

7.1 conceptual understanding

Dead Letter Queue: DLX, the team with the dead-letter Exchange Dead-Letter-Exchange RabbitMQ is closely related to

• DLX use, when the message becomes a dead letter (dead message) in a queue, it can be re-publish Exchange to another, this is DLX Exchange

News become dead letter following situations

• Message is rejected (basic.reject / basic.nack) and requeue = false
• TTL expired messages
• The maximum length of the queue

DLX is a normal Exchange, the Exchange and the general is no different, it can be specified on any queue, a queue is actually a set of attributes

When this has dead letter queue, RabbitMQ will automatically republish this message to the Exchange set up, and then be routed to another queue.

Can listen to the messages in the queue to do the appropriate treatment, this feature can make immediate function parameters RabbitMQ3.0 previously supported.

7.2 code shows

• Dead letter queue settings:
• First, we need to set the exchange dead letter queue and queue, then bind: Exchange: dlx.exchange Queue: dlx.queue RoutingKey: #
• Then we declare the normal switches, queue, bind, but we need to add a parameter to the queue: arguments.put ( "x-dead-letter-exchange", "dlx.exchange");
• Thus the expiration message, to requeue, when the maximum length of the queue, the message can be routed directly to the dead letter queue!

7.2.1 Producers

/**
 * 
* @ClassName: Producer 
* @Description: 生产者
* @author Coder编程
* @date2019年7月30日 下午23:15:51 
*
 */
public class Producer {

	
	public static void main(String[] args) throws Exception {
		
		//创建ConnectionFactory
		Connection connection = ConnectionUtils.getConnection();
		Channel channel = connection.createChannel();
		
		String exchange = "test_dlx_exchange";
		String routingKey = "dlx.save";
		
		String msg = "Hello RabbitMQ DLX Message";
		
		for(int i =0; i<1; i ++){
			
			AMQP.BasicProperties properties = new AMQP.BasicProperties.Builder()
					.deliveryMode(2)
					.contentEncoding("UTF-8")
					.expiration("10000")
					.build();
			channel.basicPublish(exchange, routingKey, true, properties, msg.getBytes());
		}
		
	}
}


复制代码

7.2.2 Consumers

/**
 * 
* @ClassName: Consumer 
* @Description: 消费者
* @author Coder编程
* @date2019年7月30日 下午23:13:51 
*
 */
public class Consumer {

	
	public static void main(String[] args) throws Exception {
		
		
		//创建ConnectionFactory
		Connection connection = ConnectionUtils.getConnection();
		Channel channel = connection.createChannel();
		
		// 这就是一个普通的交换机 和 队列 以及路由
		String exchangeName = "test_dlx_exchange";
		String routingKey = "dlx.#";
		String queueName = "test_dlx_queue";
		
		channel.exchangeDeclare(exchangeName, "topic", true, false, null);
		
		Map<String, Object> agruments = new HashMap<String, Object>();
		agruments.put("x-dead-letter-exchange", "dlx.exchange");
		//这个agruments属性，要设置到声明队列上
		channel.queueDeclare(queueName, true, false, false, agruments);
		channel.queueBind(queueName, exchangeName, routingKey);
		
		//要进行死信队列的声明:
		channel.exchangeDeclare("dlx.exchange", "topic", true, false, null);
		channel.queueDeclare("dlx.queue", true, false, false, null);
		channel.queueBind("dlx.queue", "dlx.exchange", "#");
		
		channel.basicConsume(queueName, true, new MyConsumer(channel));
		
		
	}
}


复制代码

7.2.3 custom class: MyConsumer

/**
 * 
* @ClassName: MyConsumer 
* @Description: TODO
* @author Coder编程
* @date 2019年7月30日 下午23:11:55 
*
 */
public class MyConsumer extends DefaultConsumer {


	public MyConsumer(Channel channel) {
		super(channel);
	}

	@Override
	public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
		System.err.println("-----------consume message----------");
		System.err.println("consumerTag: " + consumerTag);
		System.err.println("envelope: " + envelope);
		System.err.println("properties: " + properties);
		System.err.println("body: " + new String(body));
	}
}

复制代码

7.2.4 Test Results

Run Consumer, view the control station View Exchanges

View queue

It can be seen more test_dlx_queue DLX identification indicating when the dead letter queue occurs, sends a message to the dead letter queue dlx_queue

Close Consumer, only running Producer

After more than 10 seconds, the message expiration

In our work, the dead letter queue is very important for consumers not news, in a dead-letter state. We can only use the compensation mechanism.

summary

This introduces the advanced features of RabbitMQ, first introduced the Internet giant in actual use, how to protect 100% message delivery success and idempotency, and a confirmation message to RabbitMQ, the return message, ACK and return to the queue, limiting the message, and the use of time-out, the dead letter queue

The end of the sentence

Welcome attention to personal micro-channel public number: Coder program for the latest original technical articles and free learning materials, a lot more boutique mind maps, interview data, PMP preparation materials waiting for you to lead, allowing you to learn technical knowledge anytime, anywhere! Create a qq group: 315 211 365, we welcome into the group exchange study together. Thank you! Can also be introduced to the side of a friend in need.

Articles included to Github: github.com/CoderMerlin... Gitee: gitee.com/573059382/c... welcome attention and star ~

Reference article:

"RabbitMQ messaging middleware succinctly"

recommended article:

Messaging middleware --RabbitMQ (five) Quick Start producers and consumers, SpringBoot integrate RabbitMQ!

Messaging middleware --RabbitMQ (f) understanding of the core concepts of Exchange Switch!

Messaging middleware --RabbitMQ (seven) advanced features all here! (On)