深入详解高性能消息队列中间件 RabbitMQ

1、引言

2、什么是 RabbitMQ ？

3、RabbitMQ 优势

VC++常用功能开发汇总（专栏文章列表，欢迎订阅，持续更新...）https://blog.csdn.net/chenlycly/article/details/124272585 C++软件异常排查从入门到精通系列教程（专栏文章列表，欢迎订阅，持续更新...）https://blog.csdn.net/chenlycly/article/details/125529931 C++软件分析工具从入门到精通案例集锦（专栏文章正在更新中...）https://blog.csdn.net/chenlycly/article/details/131405795 C/C++基础与进阶（专栏文章，持续更新中...）https://blog.csdn.net/chenlycly/category_11931267.html 开源组件及数据库技术（专栏文章，持续更新中...）https://blog.csdn.net/chenlycly/category_12458859.html 网络编程与网络问题分享（专栏文章，持续更新中...）https://blog.csdn.net/chenlycly/category_2276111.html

1、引言

在进行系统设计的时候，各个模块、服务器之间为了实现数据的交互，通常是建立连接通过发送消息来进行。如果将他们一一建立连接，就会出现链路太多，每一条链路都必须感知对端等问题。此场景下消息将非常混乱，后期维护也将非常痛苦。为了解决这个问题，精简系统，引入RabbitMq。各相关模块不在相互发送消息，而将消息都发送给RabbitMQ，由RabbitMQ负责将消息传递出去。

那么，什么是RabbitMQ？RabbitMQ又是如何实现这些功能的呢？

在这里，给大家重点推荐一下我的两个热门畅销专栏：

专栏1：（该专栏订阅量接近350个，有很强的实战参考价值，广受好评！）

C++软件异常排查从入门到精通系列教程（专栏文章列表，欢迎订阅，持续更新...）https://blog.csdn.net/chenlycly/article/details/125529931

本专栏根据近几年C++软件异常排查的项目实践，系统地总结了引发C++软件异常的常见原因以及排查C++软件异常的常用思路与方法，详细讲述了C++软件的调试方法与手段，以图文并茂的方式给出具体的实战问题分析实例，带领大家逐步掌握C++软件调试与异常排查的相关技术，适合基础进阶和想做技术提升的相关C++开发人员！

专栏中的文章都是通过项目实战总结出来的（通过项目实战积累了大量的异常排查素材和案例），有很强的实战参考价值！专栏文章还在持续更新中，预计文章篇数能更新到200篇以上！

专栏2：

C/C++基础与进阶https://blog.csdn.net/chenlycly/category_11931267.html

以多年的开发实战为基础，总结并讲解一些的C/C++基础与进阶内容，以图文并茂的方式对相关知识点进行详细地展开与阐述！专栏涉及了C/C++领域的多个方面的内容，同时给出C/C++及网络方面的常见笔试面试题，并详细讲述Visual Studio常用调试手段与技巧！

2、什么是 RabbitMQ ？

在讲RabbitMQ之前，需要先了解一下AMQP的概念。

AMQP，即Advanced Message Queuing Protocol（高级消息队列协议），是一个提供统一消息服务的应用层标准高级消息队列协议。AMQP是应用层协议的一个开放标准，为面向消息的中间件设计，基于此协议的客户端与消息中间件传递消息，不受客户端/中间件不同产品、不同开发语言等条件的限制。该协议是一种二进制协议，提供客户端应用于消息中间件之间异步、安全、高效的交互。相对于我们常见的REST API，AMQP更容易实现，可以降低开销，同时灵活性高，可以轻松的添加负载平衡和高可用性的功能，并保证消息传递，在性能上AMQP协议也相对更好一些。

RabbitMQ是AMQP的一个开源实现，服务器端用Erlang语言编写，用于在分布式系统中存储转发消息，支持多种客户端，如：Python、Ruby、.NET、Java、JMS、C、PHP、 ActionScript、XMPP、STOMP等，支持AJAX。 MQ（Messages Queue）是一种应用程序与应用程的通信方法。RabbitMQ相当于生产者与消费者的模式，消息发送端（生产者）将消息写入消息队列，消息接收端（消费者）从消息队列中取出消息、消费消息；而消息的发送端无需知道消息接受端的存在，反之亦然。

3、RabbitMQ 优势

RabbitMQ主要有以下几个优势：

可靠性(Reliablity)：使用了一些机制来保证可靠性，比如持久化、传输确认、发布确认。

灵活的路由(Flexible Routing)：在消息进入队列之前，通过Exchange来路由消息。对于典型的路由功能，Rabbit已经提供了一些内置的Exchange来实现。针对更复杂的路由功能，可以将多个Exchange绑定在一起，也通过插件机制实现自己的Exchange。

消息集群(Clustering)：多个RabbitMQ服务器可以组成一个集群，形成一个逻辑Broker。

高可用(Highly Avaliable Queues)：队列可以在集群中的机器上进行镜像，使得在部分节点出问题的情况下队列仍然可用。

多种协议(Multi-protocol)：支持多种消息队列协议，如STOMP、MQTT等。

多种语言客户端(Many Clients)：几乎支持所有常用语言，比如Java、.NET、Ruby等。

管理界面(Management UI)：提供了易用的用户界面，使得用户可以监控和管理消息Broker的许多方面。

Tracing mechanism (Tracing): If the message is abnormal, RabbitMQ provides a message tracking mechanism so that users can find out what happened.

Plugin System: Provides many plug-ins to expand in many aspects, and you can also edit your own plug-ins.

4. Analysis of the overall architecture of RabbitMQ

Before introducing RabbitMQ in detail, let’s introduce several important concepts:

Queue：Message queue

Exchange: switch, which will deliver messages according to routing rules

Routing key: Routing key, exchange will deliver messages based on it

Bind: Binds queue and exchange, and delivers the message to the corresponding message queue according to routing rules.

Producer：Message producer

Consumer: The consumer of the message

The overall architecture diagram of RabbitMQ is as follows:

P (Producer, message producer) is responsible for sending, and C (Consumer, message consumer) is responsible for consuming messages. Among them, the definition of switch exchange, queue, and the binding of exchange and Queue can be placed on either the sending end or the consumer end. However, no matter where they are defined, they must be defined before use, otherwise an error will occur. This article uniformly defines exchange on the producer side, and puts the definition of queue and the binding of queue and exchange on the consumer side. In addition, in order to prevent the first use of exchange on the consumer side, exchange can be defined on the consumer side at the same time. This article does not consider this situation. It is already defined by default when using exchange on the consumer side.

4.1. Sending message process

The basic process of sending messages on the P side is:

1) Connect to the server;
2) Declare exchange and set its related properties;
3) Send the message to exchange.

Among them, there are three types of exchange: fanout, routing, topic:

1) fanout does not process the routing key, just leave it empty. As long as the queue is simply bound to the switch, the messages sent to the switch will be forwarded to all queues bound to the switch.
2) Routing processes routing keys, which requires binding a queue to the switch and requiring messages to exactly match a specific routing key.
3) Topic matches the routing key with a certain pattern. At this time, the queue needs to be bound to a pattern. The matching specifications are "#" matching one or more words, and "*" matching one word.

4.2. Consumption message process

The basic process of C consuming messages is:

1) Connect to the server;
2) Declare the queue and its attributes (persistence, whether to automatically delete the queue when there are no consumers, etc.);
3) Set routing key, and bind queue and exchange together through routing key;
4) Wait for messages and consume messages.

Among them, the attributes that queue can set are: Exclusive, auto_delete, and durable.

1) Exclusive: Exclusive queue. If a queue is declared as an exclusive queue, the queue is only visible to the connection that declares it for the first time, and is automatically deleted when the connection is disconnected.
2) Auto_delete: Automatically delete. If the queue does not have any subscribed consumers, the queue will be automatically deleted. This queue is suitable for temporary queues.
3) Durable: The queue will not be lost after the server is restarted.

An example of the above exchange, queue and binding:

Mq.queue_bind(“QueueTest”, “ExchangeTest”, “Test”)

What this binding means is: any message sent to the exchange ExchangeTest with the routing key Test will be routed to the queue named QueueTest.

5. RabbitMQ application

General platform messages are roughly divided into two types: notif and req-ack-notif. There are exactly two models corresponding to rabbitmq: publish/subscribe and rpc. The following explains these two models based on practical applications.

5.1. Broadcast

Suppose the application server receives a message A and needs to broadcast it to multiple other business servers. According to the basic structure of rabbitmq in Figure 1, we should be able to think of two ways:

Method1

Method2

Which of the above two methods can achieve our purpose? The answer is Method1. If Method2 is used, the queue will distribute the messages to the two consumers in sequence. For example, client C1 receives messages 1,3,5..., and client C2 receives messages 2,4,6....

Although this method cannot achieve our purpose, but to insert a point here, the processing volume of each message may and almost certainly is different, so sometimes client C1 has processed N messages, but client C2 has processed only one message. It has not been processed yet. In order to solve this problem, rabbitmq provides the concept of fair dispatch, that is, Fair dispatch: Rabbitmq will not assign multiple tasks to workers at the same time. Only after the worker completes the task, will it receive the task again.

Back to the place we just discussed, we have established the use of Method1 to complete this function, and now perform some simple coding verification based on this method (note: the verification language is python). The P client code of the publish/subscribe model is as follows:

import pika

#建立连接
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()

#声明交换机
channel.exchange_declare(exchange='exchangeTest', type='fanout')

#发送消息
channel.basic_publish(exchange='exchangeTest', routing_key='', body='Hello World!')
connection.close()

C client code of publish/subscribe model:

import pika

#建立连接
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()

#创建queue
channel.queue_declare(queue=’QueueTest’)

#绑定
channel.queue_bind(exchange=’exchangeTest’, queue=’QueueTest’)
def callback(ch, method, properties, body):    
print “ [x] Received %r” %(body, )    
channel.basic_consume(callback, queue =’QueueTest’, no_ack=True)
channel.start_consuming()

AMQP supports enabling multiple MQ communication channels on a TCP connection. Each channel can be applied as a communication stream and is assigned an integer identifier, which is automatically maintained by the .channel() method of the Connection() class. Every AMQP program must have at least one connection and one channel.

5.2、RPC

For most messages, we not only notify you, but also require the other party to reply to us after receiving the message. At this time,
we need the RPC model provided by rabbitmq, as shown in the following figure:

The biggest difference between the RPC model and the broadcast model is that when the consumer client receives a message, it needs to reply to the sender P. Similarly, the message producer P is not only the sender, he also needs to receive the message reply from the consumer C.

From P to C, we know that it is OK to directly bind Queue1 to exchange. So what does C use to reply to P when it replies to the message? For this reason, rabbitmq provides the callback queue and associated ID when P sends a message. When C replies to P, it can use the callback queue. The purpose of providing the correlation ID is to verify whether the Correlation_Id matches even if the P end receives the message from Queue2. If it does not match, it will be ignored directly.

Use the following code for verification (note: the verification language is python). The code for the P side of the RPC model is as follows:

import pika
class Center(object):
    def __init__(self):
        self.connection = pika.BlockingConnection(pika.ConnectionParameters( 
host='localhost'))
        self.channel = self.connection.channel()      
        #定义接收返回消息的队列，此处为一随机生成的队列
        result = self.channel.queue_declare(exclusive=True)
        self.callback_queue = result.method.queue
        #等待接收消息
        self.channel.basic_consume(self.on_response, no_ack=True,queue=self.callback_queue)

    #定义接收到返回消息的处理方法
    def on_response(self, ch, method, props, body):
        self.response = body
    def request(self, n):
        self.response = None

#发送计算请求        
self.channel.basic_publish(exchange='',
 routing_key='compute_queue', properties=pika.BasicProperties
(reply_to = self.callback_queue,), body=str(n))
        #接收返回的数据
        while self.response is None:
            self.connection.process_data_events()
        return int(self.response)
center = Center()
response = center.request(30)
print " [.] Got %r" % (response,)

C-side code of RPC model:

import pika

class Center(object):
    def __init__(self):
        self.connection = pika.BlockingConnection(pika.ConnectionParameters( 
host='localhost'))
        self.channel = self.connection.channel()      
        #定义接收返回消息的队列，此处为一随机生成的队列
        result = self.channel.queue_declare(exclusive=True)
        self.callback_queue = result.method.queue

        #等待接收消息
        self.channel.basic_consume(self.on_response, no_ack=True,queue=self.callback_queue)
    #定义接收到返回消息的处理方法
    def on_response(self, ch, method, props, body):
        self.response = body
    def request(self, n):
        self.response = None

#发送计算请求        
self.channel.basic_publish(exchange='',
 routing_key='compute_queue', properties=pika.BasicProperties
(reply_to = self.callback_queue,),body=str(n))
        #接收返回的数据
        while self.response is None:
            self.connection.process_data_events()
        return int(self.response)
center = Center()
response = center.request(30)
print " [.] Got %r" % (response,)