Registrator+Consul+Consul-template+HaProxy to dynamically modify Haproty configuration files

Implementation requirements:

Using Haproxy for load balancing, it is troublesome to manually add or delete node server information in the configuration file.

Collect the information that needs to be registered to Consul as a Haproxy node server through Registrator, and then register to Consul key/value .

Consul-template reads information from Consul key/value, then automatically modifies the Haproxy configuration file and reloads Haproxy. No need to modify haproxy.cfg .

Cluster environment:

Postil: The Mesos cluster building process is omitted here

Turn off selinux and firewall

setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
systemctl stop firewalld && systemctl disable firewalld

ZK-server

Container to do consul:
Get the Consul- server image
[root@zk-server ~]# docker pull docker.io/gliderlabs/consul-server
启动Consul-server
[root@zk-server ~]# docker run -d --name=consul --net=host docker.io/gliderlabs/consul-server -bootstrap -bind=192.168.200.8

Host to do consul

Download the consul package and unzip it

[root@zk-server ~]# wget https://releases.hashicorp.com/consul/0.6.4/consul_0.6.4_linux_386.zip
[root@zk-server ~]# unzip consul_0.6.4_linux_386.zip
[root@zk-server ~]# mv consul /usr/bin/

Download the consul-template package and unzip it

[root@zk-server ~]# wget https://releases.hashicorp.com/consul-template/0.15.0/consul-template_0.15.0_linux_386.zip
[root@zk-server ~]# unzip consul-template_0.15.0_linux_386.zip
[root@zk-server ~]# mv consul-template /usr/bin/

Install HaProxy and start

[root@zk-server ~]# yum -y install haproxy
[root@zk-server ~]# systemctl start haproxy

Create the consul server configuration directory

[root@zk-server ~]# mkdir /config

Write agent and server json files

[root @ zk-server ~] # vi /config/agent.json
Add the following content:
{
        "client_addr": "0.0.0.0",
        "data_dir": "/data",
        "leave_on_terminate": true,
        "dns_config": {
                "allow_stale": true,
                "max_stale": "1s"
        }
}
[root@zk-server ~]# vi /config/server.json
添加内容如下：
{
        "ui": true,
        "dns_config": {
                "allow_stale": false
        }
}

启动 consul 单节点服务器，当然，你 consul 服务器节点多的话也可以做 consul 集群。

[root@zk-server ~]# consul agent -server -config-dir=/config -bootstrap -bind=192.168.200.8 &

Postil ：可用 consul members 查看 consul 集群节点

Slave1-server

获取 Registrator 镜像

[root@slave1 ~]# docker pull gliderlabs/registrator:latest

启动 Registrator

Postil ：这种启动方式是注册到 Consul 的 key/value

[root@slave1 ~]# docker run -d --restart=always --name=registrator --net=host --volume=/var/run/docker.sock:/tmp/docker.sock docker.io/gliderlabs/registrator -ip 192.168.200.10 consulkv://192.168.200.8:8500/hello

Postil ： -ip 后面跟 registration 所属的主机 IP, 一定要设置此属性 , 否则服务 IP 会显示为 127.0.0.1

测试 Registrator 是否把本机容器注册到 Consul key/value

启动个容器

[root@slave1 ~]# docker run -d -P --name=test --net=bridge image/nginx

进入Consul UI界面查看

http://192.168.200.8:8500/ui/#/dc1/kv/

ZK-server

创建 consul 配置目录

[root@zk-server ~]# mkdir -p /data/cfg/consul
[root@zk-server ~]# vi /data/cfg/consul/tmpl.json
添加内容如下：
consul = "127.0.0.1:8500"
 
template {
  source = "/etc/haproxy/haproxy.ctmpl"
  destination = "/etc/haproxy/haproxy.cfg"
  command = "systemctl reload haproxy"
}

编写 haproxy 模版

[root@zk-server ~]# vi /etc/haproxy/haproxy.ctmpl
添加内容如下：
global
    log         127.0.0.1 local2
 
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon
 
    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000
 
frontend  main *:80
    acl url_static       path_beg       -i /static /images /javascript /stylesheets
    acl url_static       path_end       -i .jpg .gif .png .css .js
 
    use_backend static          if url_static
    default_backend             app
 
backend static
    balance     roundrobin
    server      static 127.0.0.1:4331 check
 
backend app
    balance     roundrobin
     {{range $key, $pairs := tree "hello/" | byKey}}{{range $serverid, $pair := $pairs}}
     server app {{.Value}} check inter 2000 fall 3 weight 1 {{end}}{{end}}

启动 consul-template

[root@zk-server ~]#consul-template -config /data/cfg/consul/tmpl.json > consul-template.out 2>&1 &

用 marathon 启动一个 nginx 容器，看 registrator 是否注册到 consul, 然后看 consul-template 是否自动添加了这个后端服务器到 /etc/haproxy/haproxy.cfg

访问 HaProxy_IP

 

 

http://www.tuicool.com/articles/IniY7vi