I do not know what time the profile was last modified, changed what? But also to change the profile or republish the project triggered manually restart the service? No reason was found to affect the wrong configuration file line normal deployment? Whether you are troubled because of these problems? 50+ online project, hundreds + configuration file, we often abuse these configuration files no love, it's time to make a change! This article will take you to solve these problems, drinking coffee and easy operation and maintenance

Configuration Center Selection

Selection principles: Simple, easy to fall, do not pick the platform, do not pick the language, rely as little as possible.

Compared Disconf, Apollo and other programs, the final choice Etcd + Confd program, in line with the principle of upper and Etcd we have had to use in the deployment of Kubernetes, be hundreds of times.

Chart Configuration Center

Center integrated configuration mode using the C / S, as a server used to store data Etcd, Confd as a client to fetch data updating ETCD
In order to facilitate the management wrote WebUI, is actually a Etcd WebUI services, primarily to interact with Etcd services, access to data Etcd
Confd pull fixed position based on the profile data to Etcd clusters, padding data and the format of the template file is set according to the configuration file to generate the final
After the profile generation may be blended check_cmdand reload_cmdcommand checks the configuration file and reload

Configuration Center deployment

Etcd cluster

System Environment
- System：Debian 8
- Etcd: v3.3.9
server address
- 192.168.107.101
- 192.168.107.102
- 192.168.107.103

All servers need to execute the following command to create a directory and install etcd

1. Download the installation package and extract etcd

# wget https://github.com/coreos/etcd/releases/download/v3.3.9/etcd-v3.3.9-linux-amd64.tar.gz
# tar -zxvf etcd-v3.3.9-linux-amd64.tar.gz

2. Copy the program to / usr / bin directory to facilitate the implementation, etcd to go write, direct run, there are two documents etcd and ectdctl,

# mv etcd-v3.3.9-linux-amd64/etcd* /usr/bin/

3. Create etcd profile directories /etc/etcdand data storage directory/home/data/etcd

# mkdir /etc/etcd /home/data/etcd

Node three nodes etcd profile are as follows

Configure node1

# cat /etc/etcd/etcd.conf 
name: 'node1'
data-dir: /home/data/etcd

listen-peer-urls: http://192.168.107.101:2380
listen-client-urls: http://192.168.107.101:2379,http://127.0.0.1:2379

initial-cluster-state: 'new'
initial-cluster-token: 'etcd-cluster-conf'
advertise-client-urls: http://192.168.107.101:2379
initial-advertise-peer-urls: http://192.168.107.101:2380
initial-cluster: node1=http://192.168.107.101:2380,node2=http://192.168.107.102:2380,node3=http://192.168.107.103:2380

node2 configuration

# cat /etc/etcd/etcd.conf 
name: 'node2'
data-dir: /home/data/etcd

listen-peer-urls: http://192.168.107.102:2380
listen-client-urls: http://192.168.107.102:2379,http://127.0.0.1:2379

initial-cluster-state: 'new'
initial-cluster-token: 'etcd-cluster-conf'
advertise-client-urls: http://192.168.107.102:2379
initial-advertise-peer-urls: http://192.168.107.102:2380
initial-cluster: node1=http://192.168.107.101:2380,node2=http://192.168.107.102:2380,node3=http://192.168.107.103:2380

node3 Configuration

# cat /etc/etcd/etcd.conf 
name: 'node3'
data-dir: /home/data/etcd

listen-peer-urls: http://192.168.107.103:2380
listen-client-urls: http://192.168.107.103:2379,http://127.0.0.1:2379

initial-cluster-state: 'new'
initial-cluster-token: 'etcd-cluster-conf'
advertise-client-urls: http://192.168.107.103:2379
initial-advertise-peer-urls: http://192.168.107.103:2380
initial-cluster: node1=http://192.168.107.101:2380,node2=http://192.168.107.102:2380,node3=http://192.168.107.103:2380

After the configuration of each node are boot

Need to run in the background, it is recommended to use screentools

# /usr/bin/etcd --config-file /etc/etcd/etcd.conf

After starting all three nodes completed, can etcdctl member listview the list of cluster command, confirmed the cluster status

# etcdctl member list
732ca490026f580d: name=node3 peerURLs=http://192.168.107.103:2380 clientURLs=http://192.168.107.103:2379 isLeader=false
bc16d35c3ad1c5ee: name=node2 peerURLs=http://192.168.107.102:2380 clientURLs=http://192.168.107.102:2379 isLeader=true
f7a043d3b65cd4a4: name=node1 peerURLs=http://192.168.107.101:2380 clientURLs=http://192.168.107.101:2379 isLeader=false

Confd

1. Download confd and put /usr/bin/the directory easy to use

# wget https://github.com/kelseyhightower/confd/releases/download/v0.16.0/confd-0.16.0-linux-amd64
# mv confd-0.16.0-linux-amd64 /usr/bin/confd
# chmod +x /usr/bin/confd

2. Create a new profile directory confd

# mkdir /etc/confd/{conf.d,templates}

3. Creating a new resource file, .tomlend of file has become a fixed format

# cat /etc/confd/conf.d/nginx.conf.toml 
[template]
src = "nginx.conf.tmpl"
dest = "/tmp/nginx.conf"

keys = [
   "/conf/project/env/nginx/nginx.conf",
]

check_cmd = "/usr/sbin/nginx -t -c {{.src}}"
reload_cmd = "/usr/sbin/service nginx reload"

Here we have a new resource file nginx configuration of parameters:

src : Specifies the location of the template file, which is nginx configuration file template tmpl location
dest : Specifies the absolute path to the configuration file finally generated or updated, in order to test us here to specify to / tmp / under
Keys : template file inside to use the key, it is the key etcd inside corresponding project profile
check_cmd : the Check command is executed after updating the configuration file is complete, whether we here at check nginx configuration file has a syntax error
reload_cmd : here you can execute commands through configuration after check, the check is no problem on the step, it will execute the reload command to reload the configuration file
prefix : Configure the prefix key, for example, are based on our key beginning with / conf, you can add a configuration prefix="/conf", it can be omitted and in the lower keys in the / conf up
owner : configured to generate user profiles
the MODE : configure permissions to create Profiles

4. Create a new template file

# cat /etc/confd/templates/nginx.conf.tmpl 
{{getv "/conf/project/env/nginx/nginx.conf"}}

confd template syntax There are many, not repeat them here, specifically check the official website
We are the entire contents of the configuration file as a value exists etcd inside, so there need only a getv instruction to get the value of value is filled to the destination file on it

FBI test

Well etcd cluster deployment and confd services, then we will test whether they can work a normal

1. Create a server KV value Etcd

# etcdctl set /conf/project/env/nginx/nginx.conf 'user  www-data;
> worker_processes 4;
> 
> pid        /var/run/nginx.pid;
> error_log  /home/logs/nginx/error.log  warn;
> 
> events  {
>     use epoll;
>     worker_connections 51200;
> }
> 
> http {
>     default_type  application/octet-stream;
> 
>     server {
>         listen       80;
>         server_name  domain.com;
> 
>         root /home/project/webroot;
>         index index.shtml index.html;
>     }
> }'

# 查看设置key的内容
# etcdctl get /conf/project/env/nginx/nginx.conf
user  www-data;
worker_processes 4;

pid        /var/run/nginx.pid;
error_log  /home/logs/nginx/error.log  warn;

events  {
    use epoll;
    worker_connections 51200;
}

http {
    default_type  application/octet-stream;

    server {
        listen       80;
        server_name  domain.com;

        root /home/project/webroot;
        index index.shtml index.html;
    }
}

Etcd API v2 and sub-version v3, large differences in the two versions, v3 optimize a lot, but considering the compatibility issues that we're using version v2
The default is v2 release, the environment variable can export ETCDCTL_API=3be switched to v3 version, v2 by etcdctl -vcan see the api version v3 by etcdctl versionviewing api version

2. Start confd

# confd -watch -backend etcd -node=http://192.168.107.101:2379 -node=http://192.168.107.102:2379 -node=http://192.168.107.103:2379
2018-08-23T13:46:13+08:00 onlinegame.i.nease.net confd[17084]: INFO Backend set to etcd
2018-08-23T13:46:13+08:00 onlinegame.i.nease.net confd[17084]: INFO Starting confd
2018-08-23T13:46:13+08:00 onlinegame.i.nease.net confd[17084]: INFO Backend source(s) set to http://192.168.107.101:2379, http://192.168.107.102:2379, http://192.168.107.103:2379
2018-08-23T13:46:13+08:00 onlinegame.i.nease.net confd[17084]: INFO Target config /tmp/nginx.conf out of sync
2018-08-23T13:46:13+08:00 onlinegame.i.nease.net confd[17084]: INFO Target config /tmp/nginx.conf has been updated

Configuration Parameter Description

-watch : open watch mode, monitor file changes etcd distribution center, once there is a change here Update Now, this option does not modify the client does not update the configuration center
-backend : the back-end types, currently supports many types etcd, zookeeper, consul, vault, redis, file, rancher , etc., confd there are some individually configurable for the type of back-end barrier, concrete can confd --helpview the command
-node : etcd node addresses, multiple nodes, then write more -node so good, we etcd a three-node cluster here three times so write '-node'
-onetime : can be used to replace the top of the -watchparameter indicates the operating once quit, if you do not want the configuration file updated in real time, just wanted to update once, you can use this parameter
-interval : it can be used to replace the top of the -watchparameter that indicates every how many seconds to take a backend data, if you want to reduce the pressure etcd server, but also want the client configuration file can be automatically updated, can be controlled by this parameter

3. You can see /tmp/nginx.conf documents have been properly synchronized and updated via the top log for /tmp/nginx.conf determine the content correctly

WebUI Kerrigan

You can not all configuration files are updated by the command line, right? In order to facilitate the management, I spent three days (three days really) wrote a WebUI, named Kerrigan, enable the directory tree, view the configuration online, modify the configuration, review the configuration update history and other useful features

Configuration page, this page can be configured by connecting the information etcd

First, the left side of the list of items (item information synchronization CMDB)

Click the project list, according to the corresponding rules go out the directory structure inside etcd presented by tree

Click Profile on the right will show the current configuration file contents

Click the "Edit" button to edit the configuration, the new page the same, but editing is not allowed to modify the path

Click on "History" button, go to the profile page of history, this page shows the configuration file all the changes history

Written in the last

This is not to say ugly interface burst! No way, the rear end of the front line test plus I have a man do, no cell design, so see, and not the most important functions easy to use it
Why not configmap K8S of? We originally wanted to do with configmap distribution center K8S of, but not all projects are run in K8S in and modify configmap also need to restart the container to take effect, so there is no use of
etcd anyone can modify it, feeling insecure ah? In fact, we are using the account password authentication, and only the internal network, limit IP, security, you know, another solution is etcd go ssl, but the client-side to put the certificate does not use too much trouble, this article details recorded etcd user authentication methods: Basic Auth authentication security configuration of Etcd
How to confirm Client-side configuration file update is successful? If you are a one-time activation can determine whether to activate the command was executed normally after the start command, if you are a watch mode or interval, so. . Only human flesh check it, I do not have a good way

Long View original article by more public attention No.

If you think the article help you, please forward to share so that more friends to see. If you feel that reading is not fun, it is recommended to read the following articles:

Easy to configure and center floor plan of operation and maintenance friendly