RabbitMQ cluster configuration

Before you configure a cluster

hostname resolution

RabbitMQ nodes use domain names to address each other, so the hostnames of all cluster members must be resolvable from all cluster nodes, you can modify the hosts file or use DNS resolution

If you want to use the full hostname of the node name (RabbitMQ defaults to the short name), and the full hostname can be resolved using DNS, you may want to investigate setting the environment variable RABBITMQ_USE_LONGNAME=true

There are several ways to create a cluster

　　via config file

　　rabbitmqctl manual configuration

　　Via plugins (eg: AWS (EC2) instance discovery, Kubernetes discovery, Consul-based discovery, etcd-based discovery)

The composition of a cluster can be changed dynamically, all RabbitMQs start running as a single node, these nodes can join the cluster, and then can leave the cluster again and go back to a single node

A RabbitMQ cluster can tolerate the failure of a single node. Nodes can start and notify at will, as long as they can contact the cluster member nodes known at the time of shutdown

Clustering is meant to be used on a local area network, it is not recommended to run a cluster across a wide area network

Nodes can be disk nodes or RAM nodes

RAM nodes store internal database tables in RAM. This excludes messages, message store indexes, queue indexes, and other
cases where node state is above 90% and you want all nodes to be disk nodes; RAM nodes are a special case that can be used to improve high queuing, swapping, or binding Churning performance clusters. RAM nodes do not provide meaningfully higher message rates. When in doubt, use only disk nodes.
Since RAM nodes only store internal database tables in RAM, they must synchronize them from peer nodes at startup. This means that the cluster must contain at least one disk node. Therefore it is not possible to manually delete the last disk node remaining in the cluster

 

rabbitmqctl configure cluster

hostname	ip	system	RabbitMQ
rabbit1	192.168.88.1	CentOS7.2.1511	3.7.0
rabbit2	192.168.88.2
rabbit3	192.168.88.3

Bind hosts file

192.168.88.1 rabbit1
192.168.88.2 rabbit2
192.168.88.3 rabbit3

 

Install RabbitMQ on three machines

RabbitMQ installation tutorial

 

Set nodes to authenticate each other: Erlang cookies

RabbitMQ nodes and CLI tools (such as rabbitmqctl) use cookies to determine if they are allowed to communicate with each other, for two nodes to be able to communicate they must have the same shared secret, called an Erlang cookie.
A cookie is just a string that can be up to There are 255 characters. It is usually stored in a local file. The file must be accessible only by the owner (400 permissions). Each cluster node must have the same cookie, the file location is /var/lib/rabbitmq/.erlang.cookie, set rabbit2 and rabbit3 to be the same as rabbit2, and the permission is 400

 

Start all nodes as normal

rabbitmq-server -detached
rabbitmq-server -detached
rabbitmq-server -detached

 

Now that three independent RabbitMQs are started, we use the cluster_status command to view the cluster status

[root@rabbit1 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit1 ...
[{nodes,[{disc,[rabbit@rabbit1]}]},
 {running_nodes,[rabbit@rabbit1]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit1,[]}]}]


[root@rabbit2 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit2 ...
[{nodes,[{disc,[rabbit@rabbit2]}]},
 {running_nodes,[rabbit@rabbit2]},
 {cluster_name,<<"rabbit@rabbit2">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit2,[]}]}]


[root@rabbit3 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit3 ...
[{nodes,[{disc,[rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit3]},
 {cluster_name,<<"rabbit@rabbit3">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit3,[]}]}]

 

To connect the three nodes in the cluster, we add the rabbit@c2 and rabbit@c3 nodes to the rabbit@c1 node cluster

First, add rabbit@c2 to the cluster of rabbit@c1

　　1. Stop the rabbitmq application of rabbit@c2,

　　2. Join the rabbit@c1 cluster

　　3. Then start the RabbitMQ program

Note: Joining a cluster implicitly resets the node, removing all resources and data that previously existed on this node

[root@rabbit2 ~]# rabbitmqctl stop_app
Stopping rabbit application on node rabbit@rabbit2 ...

[root@rabbit2 ~]# rabbitmqctl join_cluster rabbit@rabbit1
Clustering node rabbit@rabbit2 with rabbit@rabbit1

[root@rabbit2 ~]# rabbitmqctl start_app
Starting node rabbit@rabbit2 ...
 completed with 0 plugins.

Now we check the cluster status on any node of rabbit1 and rabbit2, we can see that these two nodes have joined a cluster

[root@rabbit1 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit1 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2]}]},
 {running_nodes,[rabbit@rabbit2,rabbit@rabbit1]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit2,[]},{rabbit@rabbit1,[]}]}]

 

Let's add the rabbit3 node to the cluster again

[root@rabbit3 ~]# rabbitmqctl stop_app
Stopping rabbit application on node rabbit@rabbit3 ...

[root@rabbit3 ~]# rabbitmqctl join_cluster rabbit@rabbit1
Clustering node rabbit@rabbit3 with rabbit@rabbit1

[root@rabbit3 ~]# rabbitmqctl start_app
Starting node rabbit@rabbit3 ...
 completed with 0 plugins.

Through the cluster_status command on any node, we can see that the three nodes joined a cluster

[root@rabbit1 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit1 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit3,rabbit@rabbit2,rabbit@rabbit1]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit3,[]},{rabbit@rabbit2,[]},{rabbit@rabbit1,[]}]}]

[root@rabbit2 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit2 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit3,rabbit@rabbit1,rabbit@rabbit2]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit3,[]},{rabbit@rabbit1,[]},{rabbit@rabbit2,[]}]}]

[root@rabbit3 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit3 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit1,[]},{rabbit@rabbit2,[]},{rabbit@rabbit3,[]}]}]

By following the above steps, we can add new nodes to the cluster at any time while the cluster is running

已加入群集的节点可随时停止。他们也可以崩溃。在这两种情况下，群集的其余部分都会继续运行，并且节点在再次启动时会自动“跟上”（同步）其他群集节点。

 

我们关闭rabbit@rabbit1和rabbit@rabbit3，并检查每一步中的集群状态

[root@rabbit1 ~]# rabbitmqctl stop
Stopping and halting node rabbit@rabbit1 ...

[root@rabbit2 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit2 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit3,rabbit@rabbit2]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit3,[]},{rabbit@rabbit2,[]}]}]

[root@rabbit3 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit3 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit2,rabbit@rabbit3]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit2,[]},{rabbit@rabbit3,[]}]}]

 [root@rabbit3 ~]# rabbitmqctl stop
Stopping and halting node rabbit@rabbit3 ...

[root@rabbit2 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit2 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit2]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit2,[]}]}]

 

 现在我们再次启动节点，在我们继续检查集群状态时

[root@rabbit3 ~]# rabbitmq-server -detached
[root@rabbit3 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit3 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit2,rabbit@rabbit3]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit2,[]},{rabbit@rabbit3,[]}]}]

[root@rabbit2 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit2 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit3,rabbit@rabbit2]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit3,[]},{rabbit@rabbit2,[]}]}]

[root@rabbit1 ~]# rabbitmq-server -detached
[root@rabbit1 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit1 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit2,rabbit@rabbit3,rabbit@rabbit1]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit2,[]},{rabbit@rabbit3,[]},{rabbit@rabbit1,[]}]}]

[root@rabbit2 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit2 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit1,rabbit@rabbit3,rabbit@rabbit2]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit1,[]},{rabbit@rabbit3,[]},{rabbit@rabbit2,[]}]}]

[root@rabbit3 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit3 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit1,rabbit@rabbit2,rabbit@rabbit3]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit1,[]},{rabbit@rabbit2,[]},{rabbit@rabbit3,[]}]}]

一些重要的警告：
当整个集群关闭时，最后一个关闭的节点必须是第一个要联机的节点。
如果要脱机的最后一个节点无法恢复，可以使用forget_cluster_node命令将其从群集中删除
如果所有集群节点同时停止并且不受控制（例如断电），则可能会留下所有节点都认为其他节点在其后停止的情况。在这种情况下，您可以在一个节点上使用force_boot命令使其再次可引导

 

集群移除节点

当节点不再是节点的一部分时，需要从集群中明确地删除节点。我们首先从集群中删除rabbit@rabbit3，并将其返回到独立操作

在rabbit@rabbit3上：

　　1、我们停止RabbitMQ应用程序，

　　2、重置节点

　　3、重新启动RabbitMQ应用程序

[root@rabbit3 ~]# rabbitmqctl stop_app
Stopping rabbit application on node rabbit@rabbit3 ...

[root@rabbit3 ~]# rabbitmqctl reset
Resetting node rabbit@rabbit3 ...

[root@rabbit3 ~]# rabbitmqctl start_app
Starting node rabbit@rabbit3 ...
 completed with 0 plugins.

 

在节点上 运行cluster_status命令确认rabbit@rabbit3现在不再是集群的一部分并独立运行

[root@rabbit3 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit3 ...
[{nodes,[{disc,[rabbit@rabbit3]}]},
 {running_nodes,[rabbit@rabbit3]},
 {cluster_name,<<"rabbit@rabbit3">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit3,[]}]}]

[root@rabbit1 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit1 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2]}]},
 {running_nodes,[rabbit@rabbit2,rabbit@rabbit1]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit2,[]},{rabbit@rabbit1,[]}]}]

[root@rabbit2 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit2 ...
[{nodes,[{disc,[rabbit@rabbit1,rabbit@rabbit2]}]},
 {running_nodes,[rabbit@rabbit1,rabbit@rabbit2]},
 {cluster_name,<<"rabbit@rabbit1">>},
 {partitions,[]},
 {alarms,[{rabbit@rabbit1,[]},{rabbit@rabbit2,[]}]}]

 

我们也可以远程删除节点，例如，在处理无响应的节点时，这很有用
比如：我们在节点rabbit@rabbit2上把rabbit@rabbit1从集群中移除

[root@rabbit1 ~]# rabbitmqctl stop_app
Stopping rabbit application on node rabbit@rabbit1 ...

[root@rabbit2 ~]# rabbitmqctl forget_cluster_node rabbit@rabbit1
Removing node rabbit@rabbit1 from the cluster

请注意，rabbit1仍然认为它与rabbit2集群 ，并试图启动它将导致错误。我们需要重新设置才能重新启动。

[root@rabbit1 ~]# rabbitmqctl reset     #必须要重置
Resetting node rabbit@rabbit1 ...

[root@rabbit1 ~]# rabbitmqctl start_app
Starting node rabbit@rabbit1 ...
 completed with 0 plugins.

 

现在查看集群状态，三个节点都时作为独立的节点

请注意，rabbit@rabbit2保留了簇的剩余状态，而rabbit@rabbit1 和rabbit@rabbit3是刚刚初始化的RabbitMQ。如果我们想重新初始化rabbit@rabbit2，我们按照与其他节点相同的步骤进行：

[root@rabbit2 ~]# rabbitmqctl stop_app
Stopping rabbit application on node rabbit@rabbit2 ...

[root@rabbit2 ~]# rabbitmqctl reset
Resetting node rabbit@rabbit2 ...

[root@rabbit2 ~]# rabbitmqctl start_app
Starting node rabbit@rabbit2 ...
 completed with 0 plugins.

 

主机名更改

RabbitMQ节点使用主机名相互通信。因此，所有节点名称必须能够解析所有集群对等的名称。像rabbitmqctl这样的工具也是如此
除此之外，默认情况下RabbitMQ使用系统的当前主机名来命名数据库目录。如果主机名更改，则会创建一个新的空数据库。为了避免数据丢失，建立一个固定和可解析的主机名至关重要。每当主机名更改时，您应该重新启动RabbitMQ
如果要使用节点名称的完整主机名（RabbitMQ默认为短名称），并且可以使用DNS解析完整的主机名，则可能需要调查设置环境变量 RABBITMQ_USE_LONGNAME = true

 

从客户端连接到群集

客户端可以正常连接到群集中的任何节点。如果该节点出现故障，并且集群的其余部分仍然存在，那么客户端应该注意到已关闭的连接，并且应该能够重新连接到群集的一些幸存的成员。通常，将节点主机名或IP地址烧入客户端应用程序是不可取的：这会引入不灵活性，并且如果集群配置发生更改或集群中节点数发生更改，则需要编辑，重新编译和重新部署客户端应用程序。相反，我们推荐一个更抽象的方法：这可能是一个动态的DNS服务，它具有非常短的TTL配置，或者一个普通的TCP负载均衡器，或者用起搏器或类似技术实现的某种移动IP。一般来说

 

具有RAM节点的集群

RAM节点只将其元数据保存在内存中。由于RAM节点不必像光盘节点那样写入光盘，它们可以更好地执行。但是请注意，由于永久队列数据总是存储在磁盘上，因此性能改进将仅影响资源管理（例如添加/删除队列，交换或虚拟主机），但不会影响发布速度或消耗速度
RAM节点是高级用例; 设置你的第一个群集时，你应该不使用它们。您应该有足够的光盘节点来处理您的冗余要求，然后在需要时添加额外的RAM节点进行缩放
只包含RAM节点的集群是脆弱的; 如果群集停止，您将无法再次启动， 并将丢失所有数据。RabbitMQ将阻止在许多情况下创建RAM节点的群集，但是它不能完全阻止它

这里的例子仅仅为了简单起见，显示了具有一个光盘和一个RAM节点的集群; 这样的集群是一个糟糕的设计选择

创建RAM节点
我们可以在首次加入集群时将节点声明为RAM节点。像之前一样，我们使用rabbitmqctl join_cluster来完成此 操作，但传递 --ram标志

rabbit2$ rabbitmqctl stop_app
Stopping node rabbit@rabbit2 ...done
.
rabbit2$ rabbitmqctl join_cluster --ram rabbit@rabbit1
Clustering node rabbit@rabbit2 with [rabbit@rabbit1] ...done.
rabbit2$ rabbitmqctl start_app
Starting node rabbit@rabbit2 ...done.

RAM节点在集群状态中显示为：
rabbit1$ rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit1 ...
[{nodes,[{disc,[rabbit@rabbit1]},{ram,[rabbit@rabbit2]}]},
 {running_nodes,[rabbit@rabbit2,rabbit@rabbit1]}]
...done.

rabbit2$ rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit2 ...
[{nodes,[{disc,[rabbit@rabbit1]},{ram,[rabbit@rabbit2]}]},
 {running_nodes,[rabbit@rabbit1,rabbit@rabbit2]}]
...done.

更改节点类型
我们可以将节点的类型从ram更改为disc，反之亦然。假设我们想要颠倒rabbit @ rabbit2和rabbit @ rabbit1的类型 ，将前者从ram节点转换为disc节点，将后者从disc节点转换为ram节点。要做到这一点，我们可以使用 change_cluster_node_type命令。该节点必须先停止

rabbit2$ rabbitmqctl stop_app
Stopping node rabbit@rabbit2 ...done.

rabbit2$ rabbitmqctl change_cluster_node_type disc
Turning rabbit@rabbit2 into a disc node ...
...done.
Starting node rabbit@rabbit2 ...done.

rabbit1$ rabbitmqctl stop_app
Stopping node rabbit@rabbit1 ...done.

rabbit1$ rabbitmqctl change_cluster_node_type ram
Turning rabbit@rabbit1 into a ram node ...

rabbit1$ rabbitmqctl start_app
Starting node rabbit@rabbit1 ...done.

　　

 

每天进步一点，加油!