EMQ cluster deployment and use Nginx to achieve load balancing

1. Preparation

Three servers or virtual machines (141, 142, 143).

Two, stand-alone deployment

Three virtual machines are deployed separately:

$ cd /usr/local
$ unzip emqx-centos7-v3.1.0.zip

Start EMQX

$ cd /usr/local/emqx
$ ./bin/emqx start
emqx 3.1.0 is started successfully!
 
$ ./bin/emqx_ctl status
Node '[email protected]' is started
emqx v3.1.0 is running

Visit http://xxx.xxx.xxx.xxx:18083, default user name: admin, password: public, as shown in the figure:
Insert picture description here

Three, EMQ cluster construction

First we first engage two machines, and then we enable the following ports on the two machines

1883 MQTT 协议端口
8883 MQTT/SSL 端口
8083 MQTT/WebSocket 端口
8080 HTTP API 端口
18083 Dashboard 管理控制台端口

Next, modify the EMQX configuration file

vim ./etc/emqx.conf

Modify the following content and change to the respective local ip

node.name = [email protected]
node.name = [email protected]
node.name = [email protected]

In addition, the cluster name in the unified configuration file, where the cluster name must be the same, otherwise it will not be able to join the cluster and display no response

 cluster.name = emqx

After the modification is completed, after starting the two nodes, execute on 192.168.72.142 and 192.168.72.143 respectively

./bin/emqx_ctl cluster join [email protected]
./bin/emqx_ctl cluster join [email protected]

Query the cluster status on any node:

./bin/emqx_ctl cluster status

Then we visit http://192.168.72.141:18083, we will find that three nodes have been successfully run, as shown in the figure:

Fourth, use nginx to configure load balancing

1), install nginx dependent packages

# 解决依赖包openssl安装
sudo apt-get install openssl libssl-dev

# 解决依赖包pcre安装
sudo apt-get install libpcre3 libpcre3-dev

# 解决依赖包zlib安装
sudo apt-get install zlib1g-de

2), download nginx

# 下载nginx(到官网查看对应版本)
wget http://nginx.org/download/nginx-1.13.1.tar.gz
# 解压文件到/opt目录下
tar -xzvf nginx-1.13.1.tar.gz -C /opt

3), install nginx

# 到安装目录下，编译过程中开启 --with-stream,tcp转发必须模块，因为emqx转发的是tcp连接
./configure --with-http_stub_status_module --with-http_ssl_module --with-stream --prefix=/opt/nginx
make && make install

4) Edit the nginx configuration file, use nginx 8000 port to proxy emq1883 port

#user  nobody;
worker_processes  1;
#worker_rlimit_nofile 90000;
#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    
    
    worker_connections  1024;
}

stream{
    
    
    upstream emqx_cluster {
    
    
    	#轮询
        server 192.168.72.141:1883;
        server 192.168.72.142:1883;
        server 192.168.72.143:1883;
    }

    server{
    
    
       listen  18084 so_keepalive=on;
       proxy_connect_timeout 10s;
       proxy_timeout 20s;
       proxy_pass emqx_cluster;

      # ssl_handshake_timeout 15s;
      #ssl_certificate     /export/softwares/emqx/etc/certs/cert.pem;
      #ssl_certificate_key /export/softwares/emqx/etc/certs/key.pem;
    }
}

5), start nginx

./nginx

Five, load balancing test

./emqtt_bench sub -h 192.168.72.141 -p 8000 -c 300 -i 1 -t bench /％i -q 1

Insert picture description here

Six, system tuning and stress testing

Refer to official documents:
1. EMQ system tuning
2. EMQ performance test