Current limiting, return mechanism, dead letter queue in RabbitMQ

Table of contents

advantage

shortcoming

1. Current limiting

2. return mechanism

3. Dead letter queue

advantage

High availability: RabbitMQ supports multiple methods such as clusters and mirrored queues to achieve high availability and ensure the stable operation of the system.
Strong reliability: RabbitMQ uses the AMQP protocol as the standard for message delivery, which can ensure the reliability and orderliness of message delivery.
Strong flexibility: RabbitMQ supports multiple message modes, including point-to-point, publish-subscribe, routing, RPC, etc., and you can choose the appropriate mode according to your business needs.
Excellent performance: RabbitMQ is developed based on the Erlang language. It has the characteristics of high concurrency and low latency, and supports fast processing of large amounts of data and messages.
Easy to deploy and manage: RabbitMQ provides a wealth of management tools and API interfaces for easy configuration, monitoring and management. At the same time, it also provides a visual interface to make the operation easier and easier to understand.
Open source and free: RabbitMQ is an open source software and is completely free to use. It can be easily deployed and used in any environment.

shortcoming

High learning cost: RabbitMQ involves many concepts and technologies, which may be difficult for beginners to learn.
Complex configuration: The configuration of RabbitMQ is relatively complicated, and corresponding settings need to be made according to actual needs. If configured incorrectly, system performance and stability may be affected.
High resource usage: RabbitMQ is developed using Erlang language, which requires a lot of memory and CPU resources. If it is deployed on a low-configuration server, it may cause system delays and other problems.
Single point of failure: Although RabbitMQ supports cluster mode, there are still single points of failure in some cases.
Message accumulation: If consumers process messages too slowly or abnormal conditions occur, RabbitMQ may cause problems such as message accumulation and memory overflow.

1. Current limiting

The so-called current limit is to limit how many messages the consumer can consume at one time

// 每次只消费1000条消息，RabbitMQ放过来的这1000条消息没有处理完，就不会再放进来新的消息
$channel->basic_qos(null, 1000, null);

This setting is not usually used, but it will be used when the message backlog is serious. If this value is not set when the message backlog is serious, it is likely to cause the consumer to hang up.

2. return mechanism

The return Listener is used to handle some non-routable messages!

The producer sends the message to a certain queue by specifying an exchange and routingkey, and then the consumer listens to the queue for consumption processing. However, in some cases, if the current exchange does not exist or the specified routingkey cannot be routed when we send a message, at this time, if we want to listen to such unreachable messages, we must use the return listener.

# 虽然定义了交换机，但是指定了一个没有绑定队列的路由键
<?php
declare(strict_types = 1);

use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;
use PhpAmqpLib\Exchange\AMQPExchangeType;

require __DIR__ . '/../vendor/autoload.php';

try {
    $connection = new AMQPStreamConnection('192.168.78.200', 5672, 'root', '123456');
    $channel = $connection->channel();
    // 定义一个交换机
    $channel->exchange_declare('test_return_exchange', AMQPExchangeType::DIRECT);
    $message = new AMQPMessage("Hello World!");

    // 这里我们指定了一个没有绑定队列的路由键，会导致rabbitmq找到不队列
    $channel->basic_publish($message, 'test_return_exchange', 'xxxxxx');

    while (true) {
        $channel->wait();
    }

    $channel->close();
    $connection->close();
} catch (Exception $e) {
    print_r($e->getMessage());
}

Although the code was successfully executed and the switch we declared was created, but because the routing key we specified could not find the queue bound to it, the message would not be pushed into rabbitmq, but because rabbitmq did not report an error, we It will be mistaken for the push to be successful.

Even if we use the confirm mechanism to monitor the push status, it is useless, because the confirm cannot be monitored because the switch is not found or the route cannot reach the queue. Here we can only use another advanced feature of rabbitmq, the return mechanism. The following code introduces the return mechanism:

<?php
declare(strict_types = 1);

use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;
use PhpAmqpLib\Exchange\AMQPExchangeType;

require __DIR__ . '/../vendor/autoload.php';

try {
    $connection = new AMQPStreamConnection('192.168.78.200', 5672, 'root', '123456');
    $channel = $connection->channel();
    // 定义一个交换机
    $channel->exchange_declare('test_return_exchange', AMQPExchangeType::DIRECT);
    $message = new AMQPMessage("Hello World!");

    // 指定第三个参数mandatory为true，表示当rabbitmq无法找到路由或队列时，将消息返回给生产者
    // 这里我们指定了一个没有绑定队列的路由键，会导致rabbitmq找到不队列，进而触发return
    $channel->basic_publish($message, 'test_return_exchange', 'xxxxxx', true);

    // 监听消息未找到交换机或者未找到路由键对应的路由
    $channel->set_return_listener(function ($replyCode, $replyText, $exchange, $routingKey, $message) {
        $msg  = 'oh hoo！发生错误了'.PHP_EOL;
        $msg .= '错误码：'.$replyCode.PHP_EOL;
        $msg .= '错误信息：'.$replyText.PHP_EOL;
        $msg .= '指定的交换机：'.$exchange.PHP_EOL;
        $msg .= '指定的路由键：'.$routingKey.PHP_EOL;
        $msg .= '投递的消息：'.$message->body.PHP_EOL;
        print_r($msg);
    });

    while (true) {
        $channel->wait();
    }

    $channel->close();
    $connection->close();
} catch (Exception $e) {
    print_r($e->getMessage());
}

At this time, if you run the program again, an error will be reported. We have output an error here. In actual production, the error should be recorded in the specified log data table.

# 程序返回
root@204d5054860c:/data/test-yii/web# php testReturn.php 
oh hoo！发生错误了
错误码：312
错误信息：NO_ROUTE
指定的交换机：test_return_exchange
指定的路由键：xxxxxx
投递的消息：Hello World!

3. Dead letter queue

The dead letter queue is actually just an ordinary queue bound to the dead letter switch, and the dead letter switch is just an ordinary switch, but it is a switch specially used to process dead letters.

The dead letter message is a layer of guarantee made by RabbitMQ for us. In fact, we can also not use the dead letter queue, but when the message consumption is abnormal, the message is actively delivered to another switch or recorded in the log table for subsequent special processing. .

A message will be judged as a dead letter by rabbitmq in the following scenarios, and then put into the dead letter queue.

message rejected
message expired
queue overload

The following simulation shows that a message is put into a normal queue. The expiration time of this message is set to 10 seconds. There is no consumer to process it within 10 seconds, so this message expires and becomes a dead letter. At this time, RabbitMQ It will be put into the dead letter queue, which is the dead letter queue we declared in the code.

Note: There can be not only one dead letter queue in rabbitmq, but multiple dead letter queues. If we do not specify a dead letter queue, expired messages will be discarded by rabbitmq.

<?php
declare(strict_types = 1);

use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;
use PhpAmqpLib\Exchange\AMQPExchangeType;
use PhpAmqpLib\Wire\AMQPTable;

require __DIR__ . '/../vendor/autoload.php';

try {
    $connection = new AMQPStreamConnection('192.168.78.200', 5672, 'root', '123456');

    // 设置交换机名称
    $deadLetterExchangeNameName = 'dead_letter_exchange';
    $normalExchangeName = 'exchange_6';
    // 设置队列名称
    $deadLetterQueueName = 'dead_letter_queue';
    $normalQueueName = 'queue_6';
    // 设置路由键
    $deadLetterRoutingKey = 'dead_letter_key';
    $normalRoutingKey = 'route_6';

    // 创建通道
    $channel = $connection->channel();
    // 声明交换器，作为死信交换器
    $channel->exchange_declare($deadLetterExchangeNameName, AMQPExchangeType::DIRECT, false, true);
    // 声明交换器，作为普通交换器
    $channel->exchange_declare($normalExchangeName, AMQPExchangeType::DIRECT, false, true);

    $args = new AMQPTable();
    // 设置消息过期时间为25s，25秒后消息会进入死信队列
    $args->set('x-message-ttl', 25000);
    // 设置死信交换器
    $args->set('x-dead-letter-exchange', $deadLetterExchangeNameName);
    // 设置死信路由键
    $args->set('x-dead-letter-routing-key', $deadLetterRoutingKey);

    // 声明队列，作为普通队列，同时将死信队列相关参数传入
    $channel->queue_declare($normalQueueName, false, true, false, false, false, $args);
    // 声明死信队列
    $channel->queue_declare($deadLetterQueueName, false, true, false, false);

    // 将普通队列与普通交换机绑定
    $channel->queue_bind($normalQueueName, $normalExchangeName, $normalRoutingKey);
    // 将死信队列与死信交换机绑定，同时指定死信路由键
    $channel->queue_bind($deadLetterQueueName, $deadLetterExchangeNameName, $deadLetterRoutingKey);
    // 设置传递的消息
    $message = new AMQPMessage('Hello DLX Message queue_6');

    // confirm监听消息投递【与死信队列无关，仅用于排错】
    $channel->confirm_select();
    $channel->set_ack_handler(function ($message) {
        print_r('投递成功啦,消息是'.$message->body);
    }) ;
    $channel->set_nack_handler(function ($message) {
        print_r('投递失败啦,消息是'.$message->body);
    }) ;

    // return监听未找到交换机或路由问题【与死信队列无关，仅用于排错】
    $channel->set_return_listener(function ($replyCode, $replyText, $exchange, $routingKey, $message) {
        $msg  = 'oh hoo！发生错误了'.PHP_EOL;
        $msg .= '错误码：'.$replyCode.PHP_EOL;
        $msg .= '错误信息：'.$replyText.PHP_EOL;
        $msg .= '指定的交换机：'.$exchange.PHP_EOL;
        $msg .= '指定的路由键：'.$routingKey.PHP_EOL;
        $msg .= '投递的消息：'.$message->body.PHP_EOL;
        print_r($msg);
    });

    // 发送消息到RabbitMQ
    $channel->basic_publish($message, $normalExchangeName, $normalRoutingKey, true);

    while (true) {
        $channel->wait();
    }

    // 关闭通道和连接
    $channel->close();
    $connection->close();

} catch (Exception $e) {
    print_r($e->getMessage());
}