Daily progress a little bit: Interpretation of messaging middleware -RabbitMQ (cluster theory and build articles)

Abstract: The actual production applications will adopt cluster program message queue, so if you choose to have it RabbitMQ cluster scheme is necessary to understand the principles under

In general, if only to learn or to verify the correctness of RabbitMQ business works so use that single instance deployment on the local environment or test environment can be, but for MQ middleware itself reliability, concurrency, throughput, and consider the problem of message accumulation capacity in the production environment will generally consider the use RabbitMQ cluster program. For RabbitMQ message queue so mature products, it is not difficult to set up and there are a lot of children's shoes how to build a blog written RabbitMQ message queue of the cluster, but may still have a children's shoes do not understand the principle behind it, which can lead to its encounter clusters can not be further tuning when performance problems. This part describes principles of RabbitMQ cluster programs, how to build with load balancing capabilities of small and medium scale RabbitMQ cluster, and finally gives the building a production environment can have high availability, high reliability and small and medium-scale high-throughput RabbitMQ cluster design.

First, the principle RabbitMQ cluster scheme

The RabbitMQ message queue itself is based middleware products written in Erlang, Erlang language born with the distributed nature (achieved by magic cookie each node of the cluster synchronization Erlang). Therefore, RabbitMQ natural support Clustering. This makes the image itself does not require RabbitMQ ActiveMQ, Kafka are implemented as programs and stored metadata HA cluster by ZooKeeper. A cluster is a way to ensure the reliability, while horizontal expansion in order to achieve by increasing the message throughput capability. The following first look at the overall program RabbitMQ cluster:

FIG employed above three nodes of a cluster RabbitMQ, the Exchange A (switch, for RabbitMQ shoes do not understand the basic concept underlying concepts may look) metadata information on all nodes is the same, the Queue ( queue) stored message data integrity will only exist on that node it creates. Other nodes only know this metadata information queue pointer and a pointer to the queue of the owner node.

Synchronization (1) RabbitMQ cluster metadata

RabbitMQ cluster always synchronize internal metadata (index similar) of four types: a metadata queues: queue name and its properties; B exchanger metadata: switch name, type, and attributes; C binding element. data: a simple table shows how to route messages to the queue; d.vhost metadata: namespace and provide security for the queue attributes, and binding within the exchanger Vhost; Accordingly, when a user accesses any node RabbitMQ when, by rabbitmqctl inquiry into the queue / user / exchange / vhost and other information are the same.

(2) Why RabbitMQ cluster only by way of synchronized metadata

I'm sure many students will ask, want to implement HA solutions that will complete all Queue data RabbitMQ cluster on all nodes do not keep a can begin? ( May be similar to main mode MySQL main thing ) like this, any node fails or is unavailable down, so long as the user of the client can be connected to any other node can complete the usual thing publish and subscribe message. I would like RabbitMQ author of such a design is mainly based on the cluster itself up to consider the performance and storage space. First, storage space, if each cluster node has a copy of all data completely Queue, then the storage space of each node will be very large, the message backlog ability of the cluster will be very weak ( can not increase the backlog of messages through capacity expansion cluster node ); second, performance, news publishers need to copy the message to each cluster node for persistent messaging, network and disk overhead of synchronous replication will be significantly increased.

The basic principle (3) RabbitMQ cluster send / subscribe messaging

It works RabbitMQ cluster is as follows:

1, the client node scenario where direct connection queue

If there is a message producer or message consumers via client connections amqp-client node to publish a message or subscribe to 1, then the time of cluster messaging only associated with the node 1, this is no problem; if the customer end is connected to node 2 or node 3 (data not on the queue node 1), and then the situation like now?

Non-2 data queue, connected to the client node where the scene

If the message producer connected to a node 2 or node 3, data queue 1 at this time is not complete on the two nodes, then the process of sending a message from the two main nodes of a routing forwarding action, these two the metadata node (i.e. mentioned above: points to the owner node of the queue pointer ) forwarded to node 1, the message will be finally transmitted or stored in a queue of a node. Similarly, if the message consumer node 2 or node 3 is connected, and that these two nodes will also play a role as a forwarding node routing, will consume from a queue node 1 pull message.

Two, RabbitMQ cluster structures

(1) Set up the cluster RabbitMQ installed components required

There RabbitMQ cluster structures necessary before mounting the following components in the package on each virtual machine, are as follows:. A the Jdk 1.8 . B the Erlang runtime environment ., Here is otp_src_19.3.tar.gz (200MB +) C RabbitMQ the Server component , here's rabbitmq-server-generic-unix- 3.6.10.tar.gz specific steps on how to install these three components, there have been numerous Bowen conducted a very detailed description, then this article not repeat them. Students may need specific reference to these steps to complete the installation.

(2) Set up 10 nodes of the cluster RabbitMQ

The main section of the show is to build a cluster, you need to make sure that each machine on the three components are installed correctly, and examples of RabbitMQ on each virtual machine to start up properly. a. RabbitMQ edit each cookie file, the cookie file to ensure that each node uses the same value, which can scp cookie on one machine to the other each node, the default path for the cookie / var / lib / rabbitmq / .erlang.cookie or $ HOME / .erlang.cookie, determining whether another node may communicate via cookie. b. Configure hosts file for each node (vim / etc / hosts)

xxx.xxx.xxx.xxx rmq-broker-test-1
xxx.xxx.xxx.xxx rmq-broker-test-2
xxx.xxx.xxx.xxx rmq-broker-test-3
......
xxx.xxx.xxx.xxx rmq-broker-test-10
复制代码

c. Start from node to node RabbitMQ service

rabbitmq-server -detached
复制代码

d. View the work of running and each node of the cluster

rabbitmqctl status, rabbitmqctl cluster_status
复制代码

. E In rmq-broker-test-1 master node, rmq-broker-test-2 on:

rabbitmqctl stop_app 
rabbitmqctl reset 
rabbitmqctl join_cluster rabbit@rmq-broker-test-2 
rabbitmqctl start_app 
复制代码

Procedure on the remaining nodes on the same rmq-broker-test-2 virtual machine. . D RabbitMQ cluster nodes are only two types: memory node / disk node, single-node systems run only disk-type nodes. In a cluster, some nodes may be selected to configure the memory of the node. Memory node all the queues, switches, binding relationship, users, permissions, and metadata information vhost is stored in memory. The disk node information is stored on disk, but higher performance memory node, in order to ensure high availability cluster, the cluster must ensure that there are more than two disk nodes to ensure that when there is a disk node crashes, the cluster also We can provide external access to services. In the above operation, by the following manner, provided the new node is a node or a disk memory node:

#加入时候设置节点为内存节点（默认加入的为磁盘节点）
[root@mq-testvm1 ~]# rabbitmqctl join_cluster rabbit@rmq-broker-test-1 --ram
复制代码

#也通过下面方式修改的节点的类型
[root@mq-testvm1 ~]# rabbitmqctl changeclusternode_type disc | ram
复制代码

. E can be viewed by the last "rabbitmqctl cluster_status" mode state of the cluster, the cluster state RabbitMQ built above the node 10 (i.e. the disk node 3, but eh, seven nodes of the node memory) as follows:

Cluster status of node 'rabbit@rmq-broker-test-1'
[{nodes,[{disc,['rabbit@rmq-broker-test-1','rabbit@rmq-broker-test-2',
                'rabbit@rmq-broker-test-3']},
         {ram,['rabbit@rmq-broker-test-9','rabbit@rmq-broker-test-8',
               'rabbit@rmq-broker-test-7','rabbit@rmq-broker-test-6',
               'rabbit@rmq-broker-test-5','rabbit@rmq-broker-test-4',
               'rabbit@rmq-broker-test-10']}]},
 {running_nodes,['rabbit@rmq-broker-test-10','rabbit@rmq-broker-test-5',
                 'rabbit@rmq-broker-test-9','rabbit@rmq-broker-test-2',
                 'rabbit@rmq-broker-test-8','rabbit@rmq-broker-test-7',
                 'rabbit@rmq-broker-test-6','rabbit@rmq-broker-test-3',
                 'rabbit@rmq-broker-test-4','rabbit@rmq-broker-test-1']},
 {cluster_name,<<"rabbit@mq-testvm1">>},
 {partitions,[]},
 {alarms,[{'rabbit@rmq-broker-test-10',[]},
          {'rabbit@rmq-broker-test-5',[]},
          {'rabbit@rmq-broker-test-9',[]},
          {'rabbit@rmq-broker-test-2',[]},
          {'rabbit@rmq-broker-test-8',[]},
          {'rabbit@rmq-broker-test-7',[]},
          {'rabbit@rmq-broker-test-6',[]},
          {'rabbit@rmq-broker-test-3',[]},
          {'rabbit@rmq-broker-test-4',[]},
          {'rabbit@rmq-broker-test-1',[]}]}]
复制代码

(3) Configuration HAProxy

HAProxy provide high availability, load balancing, and proxy TCP and HTTP-based applications, support for virtual hosts, it's free, fast and reliable a solution. According to official data, the maximum limit of concurrent 10G. 4 HAProxy support from the network layer to the switching layer 7, i.e., to cover all the TCP protocol. That is, Haproxy even supports Mysql balanced load. In order to achieve soft RabbitMQ cluster load balancing, where you can choose HAProxy. Before article on how to install HAProxy there are a lot of students wrote, not repeat them here, needy students can refer to the online approach. Here mainly said particular configuration is finished after the component mounting HAProxy. HAProxy using a single configuration file for all properties, including IP from the front end to the back-end server. The following shows a load balancing configuration for seven RabbitMQ cluster nodes (a node for another 3 disk configuration and metadata preservation cluster, not load). Meanwhile, HAProxy running on another machine. HAProxy specific configuration is as follows:

#全局配置
global
        #日志输出配置，所有日志都记录在本机，通过local0输出
        log 127.0.0.1 local0 info
        #最大连接数
        maxconn 4096
        #改变当前的工作目录
        chroot /apps/svr/haproxy
        #以指定的UID运行haproxy进程
        uid 99
        #以指定的GID运行haproxy进程
        gid 99
        #以守护进程方式运行haproxy #debug #quiet
        daemon
        #debug
        #当前进程pid文件
        pidfile /apps/svr/haproxy/haproxy.pid

#默认配置
defaults
        #应用全局的日志配置
        log global
        #默认的模式mode{tcp|http|health}
        #tcp是4层，http是7层，health只返回OK
        mode tcp
        #日志类别tcplog
        option tcplog
        #不记录健康检查日志信息
        option dontlognull
        #3次失败则认为服务不可用
        retries 3
        #每个进程可用的最大连接数
        maxconn 2000
        #连接超时
        timeout connect 5s
        #客户端超时
        timeout client 120s
        #服务端超时
        timeout server 120s

        maxconn 2000
        #连接超时
        timeout connect 5s
        #客户端超时
        timeout client 120s
        #服务端超时
        timeout server 120s

#绑定配置
listen rabbitmq_cluster
        bind 0.0.0.0:5672
        #配置TCP模式
        mode tcp
        #加权轮询
        balance roundrobin
        #RabbitMQ集群节点配置,其中ip1~ip7为RabbitMQ集群节点ip地址
        server rmq_node1 ip1:5672 check inter 5000 rise 2 fall 3 weight 1
        server rmq_node2 ip2:5672 check inter 5000 rise 2 fall 3 weight 1
        server rmq_node3 ip3:5672 check inter 5000 rise 2 fall 3 weight 1
        server rmq_node4 ip4:5672 check inter 5000 rise 2 fall 3 weight 1
        server rmq_node5 ip5:5672 check inter 5000 rise 2 fall 3 weight 1
        server rmq_node6 ip6:5672 check inter 5000 rise 2 fall 3 weight 1
        server rmq_node7 ip7:5672 check inter 5000 rise 2 fall 3 weight 1

#haproxy监控页面地址
listen monitor
        bind 0.0.0.0:8100
        mode http
        option httplog
        stats enable
        stats uri /stats
        stats refresh 5s
复制代码

In the above configuration "listen rabbitmq_cluster bind 0.0.0.0:5671" as defined herein, connected to the client IP address and port number. Load balancing algorithm is here arranged roundrobin- WRR. Most relevant configuration RabbitMQ cluster load balancing is a "server rmq_node1 ip1: 5672 check inter 5000 rise 2 fall 3 weight 1" of this, it identifies and defines the rear end of the RabbitMQ service. Mainly the following meanings: (a) "server" section: identifies the definition HAProxy RabbitMQ services; (b) "ip1: 5672" section: identifies the backend service address of RabbitMQ; (c) "check inter" section: each represents every how many milliseconds to check RabbitMQ service is available; (d) "rise" part: that RabbitMQ service after a failure, how many health checks before need to be confirmed again available; (e) "fall" section: expressed the need for how many times he failed after the health check, HAProxy RabbitMQ will stop using the service.

#启用HAProxy服务
[root@mq-testvm12 conf]# haproxy -f haproxy.cfg
复制代码

After starting, you can see HAproxy interface map as follows:

(4) RabbitMQ cluster architecture design

After the above RabbitMQ10 node in the cluster to build and HAProxy soft elastic load balancing can set up a small-scale RabbitMQ cluster configuration after, however, to be able to use actual production environment also requires each instance in the cluster according to the actual business needs Some monitor performance parameters indicators of performance, throughput capacity, and bulk messages viewpoint, Kafka may be selected as used to monitor RabbitMQ cluster queue. Thus, there is first given a small scale RabbitMQ cluster architecture design:

For the production and message consumers can request a soft load will be distributed to HAProxy RabbitMQ cluster node Node1 ~ Node7, wherein three nodes Node8 ~ Node10 as disk storage node cluster configuration information and metadata. In view of space reasons there is not the monitoring part described in detail, the statistical data will be monitored in detail how to use HTTP API interface RabbitMQ of the subsequent length.

Third, the summary

This paper describes in detail the working principle of RabbitMQ and how to build a cluster with load balancing capabilities of small and medium size RabbitMQ cluster approach, and finally gives the architecture design RabbitMQ cluster. Limited to the author's talent and less learning, content herein may also be understood not in place, if any elaborate unreasonable message also hope to explore.

Reprinted Author: Man mania

Original link: www.jianshu.com/p/637693684...

Written in the last:

No. I welcome everyone's attention the public opening of the new [ calm as code ], mass Java-related articles, learning materials will be updated on the inside, finishing materials will be on the inside. (Smoke powder difficult, a lot of support)

I feel good to write on a point of praise, plus followers chant! Point of attention, do not get lost, continuously updated! ! !