Use Consul, docker-compose, Consul-template, and registrator to build a Consul cluster in docker (graphic details!)

Use Consul, docker-compose, Consul-template, and registrator to build a Consul cluster in docker

Environmental preparation:

Server: 192.168.2.8 Docker-ce, Compose, Consul, Consul-template
Server: 192.168.2.9 Docker-ce, registrator
Later when Consul multi-node is added, a node will be added
Server: 192.168.2.7 Docker-ce, Compose, Consul

Experimental requirements:

Realize the intercommunication between containers and containers in a single-machine network
Build Consul services to realize automatic discovery and update of services Realize
high availability of Consul services

One, deploy Consul server

Consul server: 192.168.2.8

//把consul_0.9.2_linux_amd64.zip拖进来
mkdir /root/consul
cp consul_0.9.2_linux_amd64.zip /root/consul
cd /root/consul
unzip consul_0.9.2_linux_amd64.zip
mv consul /usr/bin
cd /usr/bin
//把docker-compose包拖进来
chmod +x docker-compose

consul agent \
-server \
-bootstrap \
-ui \
-data-dir=/var/lib/consul-data \
-bind=192.168.2.8 \
-client=0.0.0.0 \
-node=consul-server01 &> /var/log/consul.log &

执行完可以用jobs查看，正在运行中

//查看集群信息
consul members 
consul info | grep leader

//通过http api获取集群信息
curl 127.0.0.1:8500/v1/status/peers //查看集群server成员
curl 127.0.0.1:8500/v1/status/leader //集群 Raf leader
curl 127.00.1:8500/v1/catalog/services //注册的所有服务
curl 127.0.0.1:8500/v1/catalog/nginx //查看 nginx 服务信息
curl 127.0.0.1:8500/v1/catalog/nodes //集群节点详细信息

2. Deploy the container service to automatically join the nginx cluster

Server: 192.168.2.9

1. Underwear Gliderlabs / Registrator Gliderlabs / Registrator

You can check the running status of the container and register automatically, and you can also cancel the service of the docker container to the service configuration center.
Currently, it supports Consul, Etcd (also distributed storage) and SkyDNS2. It is
on the 192.168.2.9 node. Do the following:

docker run -d \
--name=registrator \
--net=host \
-v /var/run/docker.sock:/tmp/docker.sock \
--restart=always \
gliderlabs/registrator:latest \
-ip=192.168.2.9 \
consul://192.168.2.8:8500

docker images //查看生成的镜像
docker ps -a //查看开启的容器

2. Test the service to find out whether the function is normal

Server: 192.168.2.9

docker run -itd -p:83:80 --name test-01 -h test01 nginx
docker run -itd -p:84:80 --name test-02 -h test02 nginx
docker run -itd -p:88:80 --name test-03 -h test03 httpd
docker run -itd -p:89:80 --name test-04 -h test04 httpd

docker ps -a

3. Verify that http and nginx services are registered to consul

Browser enter http://192.168.2.8:8500, "click NODES",
and then click "consurl-server01", there will be 5 services

Consul server: 192.168.2.8
View service on consul server

curl 127.0.0.1:8500/v1/catalog/services

4. Install consul-template

Consul-Template is a daemon process used to query Consul cluster information in real time,
update any number of specified templates on the file system, and generate configuration files. After the update is complete,
you can choose to run the shell command to perform the update operation and reload Nginx. Consul-Template
can query the service catalog, Key, Key-values, etc. in Consul.
This powerful abstraction function and query language template can make Consul-Template particularly suitable for dynamic
creation of configuration files. For example: create Apache/Nginx Proxy Balancers, Haproxy Backends

5. Prepare template nginx template file

Consul server: 192.168.2.8
//Operate on consul

vim /root/consul/nginx.ctmpl
upstream http_backend {
    
    
 {
    
    {
    
    range service "nginx"}}
 server {
    
    {
    
    .Address}}:{
    
    {
    
    .Port}};
 {
    
    {
    
    end}}
}
server {
    
    
 listen 83;
 server_name localhost 192.168.2.8;
 access_log /var/log/nginx/mhh.cn-access.log;
 index index.html index.php;
 location / {
    
    
 proxy_set_header HOST $host;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header Client-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_pass http://http_backend;
 }
}

Note: Finally, the configuration file will be generated according to this template

6. Compile and install nginx

Consul server: 192.168.2.8

yum install gcc pcre-devel zlib-devel -y
tar zxvf nginx-1.12.0.tar.gz -C /opt
cd /opt
./configure --prefix=/usr/local/nginx
make && make install

7. Configure nginx

Consul server: 192.168.2.8

vim /usr/local/nginx/conf/nginx.conf
http {
    
    
	include	mime.types;
	include vhost/*.conf; //添加虚拟主机目录
	default_type application/octet-stream;

//创建虚拟主机目录
mkdir /usr/local/nginx/conf/vhost
//创建日志文件目录
mkdir /var/log/nginx  
//启动nginx
/usr/local/nginx/sbin/nginx
netstat -natp | grep nginx

8. Configure and start the template

Consul server: 192.168.2.8
Upload the consul-template_0.19.3_linux_amd64.zip package to the /root/consul directory

cp consul-template_0.19.3_linux_amd64.zip /root/consul
cd /root/consul
unzip consul-template_0.19.3_linux_amd64.zip
mv consul-template /usr/bin/
consul-template -consul-addr 192.168.2.8:8500 \
-template "/root/consul/nginx.ctmpl:/usr/local/nginx/conf/vhost/kgc.conf:/usr/local/nginx/sbin/nginx -s reload" \
--log-level=info

//Open another terminal to view the generated configuration file
Consul server: 192.168.2.8
It is mhh.conf in the /usr/local/nginx/conf/vhost directory

cd /usr/local/nginx/conf/vhost
ls
vim mhh.conf

9. Add a nginx container node

Server: 192.168.2.9
Add a nginx container node. Test service discovery and configuration update function
//register on the registrator server

docker run -itd -p:85:80 --name test-05 -h test05 nginx

Browser enter http://192.168.2.8:8500, "click NODES",
and then click "consurl-server01", you will find that a server has been added

Consul server: 192.168.2.8
In the consul server monitoring and loading, there will be prompts to automatically update

2021/03/26 12:03:16.909869 [INFO] (runner) initiating run
2021/03/26 12:03:16.910975 [INFO] (runner) rendered "/root/consul/nginx.ctmpl" => "/usr/local/nginx/conf/vhost/mhh.conf"
2021/03/26 12:03:16.910993 [INFO] (runner) executing command "/usr/local/nginx/sbin/nginx -s reload" from "/root/consul/nginx.ctmpl" => "/usr/local/nginx/conf/vhost/mhh.conf"
2021/03/26 12:03:16.911091 [INFO] (child) spawning: /usr/local/nginx/sbin/nginx -s reload

Check the mhh.conf configuration file under vhost again.

Even if any nginx container is stopped, the configuration file will be changed accordingly. I will not demonstrate it here.
l Server: 192.168.2.9
View the logs of three nginx containers
It is found that all nginx nodes have been load balanced by nginx. The load balancing accesses the nginx container, and the request is normally polled to each container node. The addition or deletion of the container will directly modify the configuration file under the vhost directory.

docker logs -f test-01
docker logs -f test-02
docker logs -f test-05

Visit 192.168.2.8:83, which is the port where nginx provides services.
You can refresh the web page repeatedly and observe the logs of three nginx containers.

View the logs. The access IP is 192.168.2.8

Three, deploy consul multi-node

Newly added Consul server: 192.168.2.7

//Add an existing docker environment server 192.168.2.7/24 to join the existing cluster

consul agent \
-server \
-bootstrap \
-ui \
-data-dir=/var/lib/consul-data \
-bind=192.168.2.7 \
-client=0.0.0.0 \
-node=consul-server02 \
-enable-script-checks=true \
-datacenter=dc1 \
-join 192.168.2.8 &> /var/log/consul.log &

-enable-script-checks=true:设置检查服务为可用
-datacenter:数据中心名称
-join:加入到已有的集群中

Log in to 192.168.2.8:8500 in the browser again or refresh the interface directly