SaltStack入门篇（七）之架构部署实战

模块：https://docs.saltstack.com/en/2016.11/ref/states/all/index.html 
实战架构图：

实验环境设置：

主机名    　　　　　　　　　　　　　　IP地址    　　　　　　　　　　　　角色
linux-node1.example.com    192.168.56.11    Master、Minion、Haproxy+Keepalived、Nginx+PHP
linux-node2.example.com    192.168.56.12    Minion、Memcached、Haproxy+Keepalived、Nginx+PHP

SaltStack环境设置： 
base环境用于存放初始化的功能，prod环境用于放置生产的配置管理功能

[root@linux-node1 ~]# vim /etc/salt/master
file_roots:
  base:
    - /srv/salt/base
  dev:
    - /srv/salt/dev
  test:
    - /srv/salt/test
  prod:
    - /srv/salt/prod

pillar_roots:
  base:
    - /srv/pillar/base
  prod:
    - /srv/pillar/prod

1、系统初始化

当我们的服务器上架并安装好操作系统后，都会有一些基础的操作，所以生产环境中使用SaltStack，建议将所有服务器都会涉及的基础配置或者软件部署归类放在base环境下。此处，在base环境下创建一个init目录，将系统初始化配置的sls均放置到init目录下，称为“初始化模块”。

（1）需求分析和模块识别

初始化内容	模块使用	文件
关闭SElinux	file.managed	/etc/selinux/config
关闭默认firewalld	service.disabled
时间同步	pkg.installed
文件描述符	file.managed	/etc/security/limits.conf
内核优化	sysctl.present
SSH服务优化	file.managed、service.running
精简开机系统服务	service.dead
DNS解析	file.managed	/etc/resolv.conf
历史记录优化history	file.append	/etc/profile
设置终端超时时间	file.append	/etc/profile
配置yum源	file.managed	/etc/yum.repo.d/epel.repo
安装各种agent	pkg.installed 、file.managed、service.running
基础用户	user.present、group.present
常用基础命令	pkg.installed、pkgs
用户登录提示、PS1的修改	file.append	/etc/profile

（2）需求实现

  1 [root@linux-node1 base]# pwd
  2 /srv/salt/base
  3 [root@linux-node1 base]# mkdir init/files -p
  4 
  5 1、关闭selinux
  6 #使用了file模块的managed方法
  7 [root@linux-node1 init]# vim selinux.sls 
  8 selinux-config:
  9   file.managed:
 10     - name: /etc/selinux/config
 11     - source: salt://salt/init/files/selinux-config
 12     - user: root
 13     - group: root
 14     - mode: 0644
 15 [root@linux-node1 init]# cp /etc/selinux/config files/selinux-config
 16 
 17 2、关闭firewalld
 18 #使用service模块的dead方法，直接关闭firewalld，并禁止开机启动
 19 [root@linux-node1 init]# vim firewalld.sls 
 20 firewall-stop:
 21   service.dead:
 22     - name: firewalld.service
 23     - enable: False
 24 
 25 3、时间同步
 26 #先使用pkg模块安装ntp服务，再使用cron模块加入计划任务
 27 [root@linux-node1 init]# vim ntp.sls 
 28 ntp-install:
 29   pkg.installed:
 30     - name: ntpdate
 31 
 32 cron-ntpdate:
 33   cron.present:
 34     - name: ntpdate time1.aliyun.com
 35     - user: root
 36     - minute: 5
 37 
 38 4、修改文件描述符
 39 #使用file模块的managed方法
 40 [root@linux-node1 init]# vim limit.sls 
 41 limit-config:
 42   file.managed:
 43     - name: /etc/security/limits.conf
 44     - source: salt://init/files/limits.conf
 45     - user: root
 46     - group: root
 47     - mode: 0644
 48 [root@linux-node1 init]# cp /etc/security/limits.conf files/
 49 [root@linux-node1 init]# echo "*               -       nofile          65535
 50 " >> files/limits.conf 
 51 
 52 5、内核优化
 53 #使用sysctl模块的present方法,此处演示一部分，这里没有使用name参数，所以id就相当于是name
 54 [root@linux-node1 init]# vim sysctl.sls 
 55 net.ipv4.tcp_fin_timeout:
 56   sysctl.present:
 57     - value: 2
 58 
 59 net.ipv4.tcp_tw_reuse:
 60   sysctl.present:
 61     - value: 1
 62 
 63 net.ipv4.tcp_tw_recycle:
 64   sysctl.present:
 65     - value: 1
 66 
 67 net.ipv4.tcp_syncookies:
 68   sysctl.present:
 69     - value: 1
 70 
 71 net.ipv4.tcp_keepalive_time:
 72   sysctl.present:
 73     - value: 600
 74 
 75 6、SSH服务优化
 76 #使用file.managed和service.running以及watch，对ssh服务进行优化配置
 77 [root@linux-node1 init]# vim sshd.sls
 78 sshd-config:
 79   file.managed:
 80     - name: /etc/ssh/sshd_config
 81     - source: salt://init/files/sshd_config
 82     - user: root
 83     - gourp: root
 84     - mode: 0600
 85   service.running:
 86     - name: sshd
 87     - enable: True
 88     - reload: True
 89     - watch:
 90       - file: sshd-config
 91 [root@linux-node1 init]# cp /etc/ssh/sshd_config files/
 92 [root@linux-node1 init]# vim files/sshd_config 
 93 Port 8022
 94 UseDNS no
 95 PermitRootLogin no
 96 PermitEmptyPasswords no
 97 GSSAPIAuthentication no
 98 
 99 7、精简开机启动的系统服务
100 #举例关闭postfix开机自启动
101 [root@linux-node1 init]# vim thin.sls 
102 postfix:
103   service.dead:
104     - enable: False
105 
106 8、DNS解析
107 [root@linux-node1 init]# vim dns.sls 
108 dns-config:
109   file.managed:
110     - name: /etc/resolv.conf
111     - source: salt://init/files/resolv.conf
112     - user: root
113     - group: root
114     - mode: 644
115 [root@linux-node1 init]# cp /etc/resolv.conf files/
116 
117 9、历史记录优化history
118 #使用file.append扩展修改HISTTIMEFORMAT的值
119 [root@linux-node1 init]# vim history.sls 
120 history-config:
121   file.append:
122     - name: /etc/profile
123     - text:
124       - export HISTTIMEFORMAT="%F %T `whoami` "
125       - export HISTSIZE=5
126       - export HISTFILESIZE=5
127 
128 10、设置终端超时时间
129 #使用file.append扩展修改TMOUT环境变量的值
130 [root@linux-node1 init]# vim tty-timeout.sls 
131 ty-timeout:
132   file.append:
133     - name: /etc/profile
134     - text:
135       - export TMOUT=300
136 
137 11、配置yum源
138 #拷贝yum源
139 [root@linux-node1 init]# vim yum-repo.sls 
140 /etc/yum.repos.d/epel.repo:
141   file.managed:
142     - source: salt://init/files/epel.repo
143     - user: root
144     - group: root
145     - mode: 0644
146 
147 12、安装各种agent（如安装zabbix-agent）
148 #相当于一个软件的安装、配置、启动，此处也使用了jinja模板和pillar
149 [root@linux-node1 base]# mkdir zabbix
150 [root@linux-node1 base]# vim zabbix/zabbix-agent.sls 
151 zabbix-agent:
152   pkg.installed:
153     - name: zabbix22-agent
154   file.managed:
155     - name: /etc/zabbix_agentd.conf
156     - source: salt://zabbix/files/zabbix_agentd.conf
157     - template: jinja
158     - defaults:
159       ZABBIX-SERVER: {{ pillar['zabbix-agent']['Zabbix_Server'] }}
160     - require:
161       - pkg: zabbix-agent
162   service.running:
163     - enable: True
164     - watch:
165       - pkg: zabbix-agent
166       - file: zabbix-agent
167 zabbix_agent.conf.d:
168   file.directory:
169     - name: /etc/zabbix_agentd.conf.d
170     - watch_in:
171       - service: zabbix-agent
172     - require:
173       - pkg: zabbix-agent
174       - file: zabbix-agent
175 [root@linux-node1 srv]# vim pillar/base/zabbix.sls 
176 zabbix-agent:
177   Zabbix_Server: 192.168.56.11
178 
179 13、基础用户
180 #增加基础管理用户www，使用user.present和group.present
181 [root@linux-node1 init]# vim user-www.sls 
182 www-user-group:
183   group.present:
184     - name: www
185     - gid: 1000
186 
187   user.present:
188     - name: www
189     - fullname: www
190     - shell: /sbin/bash
191     - uid: 1000
192     - gid: 1000
193 
194 14、常用基础命令
195 #这里因为各软件包会依赖源，所以使用include讲yum源包含进来，并在pkg.installed最后增加require依赖
196 [root@linux-node1 init]# vim pkg-base.sls 
197 include:
198   - init.yum-repo
199 base-install:
200   pkg.installed:
201     - pkgs:
202       - screen
203       - lrzsz
204       - tree
205       - openssl
206       - telnet
207       - iftop
208       - iotop
209       - sysstat
210       - wget
211       - dos2unix
212       - lsof
213       - net-tools
214       - mtr
215       - unzip
216       - zip
217       - vim
218       - bind-utils
219     - require:
220       - file: /etc/yum.repos.d/epel.repo
221 
222 15、用户登录提示、PS1的修改    
223 [root@linux-node1 init]# vim tty-ps1.sls 
224 /etc/bashrc:
225   file.append:
226     - text:
227       - export PS1=' [\u@\h \w]\$ '
228 
229 16、编写一个总的状态，并写入top file中
230 #将所有初始化所需要的功能编写完成，每个小功能都是一个sls文件，统一放在init目录下。此时再使用include把这些初始化的功能都包含进来。
231 [root@linux-node1 init]# vim init-all.sls 
232 include:
233   - init.dns
234   - init.yum-repo
235   - init.firewalld
236   - init.history
237   - init.limit
238   - init.ntp
239   - init.pkg-base
240   - init.selinux
241   - init.sshd
242   - init.sysctl
243   - init.thin
244   - init.tty-timeout
245   - init.tty-ps1
246   - init.user-www
247 
248 #在top.sls里面给Minion指定状态并执行，强烈建议先测试，确定SaltStack会执行哪些操作然后再应用状态到服务器上
249 [root@linux-node1 base]# vim top.sls 
250 base:
251   '*':
252     - init.init-all
253 [root@linux-node1 base]# salt '*' state.highstate test=True
254 [root@linux-node1 base]# salt '*' state.highstate 

View Code

2、MySQL主从

1.需求分析： 
配置MySQL主从的有以下步骤： 
（1）MySQL安装初始化—->mysql-install.sls 
（2）MySQL的主配置文件my.cnf配置不同的server_id–>mariadb-server-master.cnf、mariadb-server-slave.cnf 
（3）创建主从同步用户–>master.sls 
（4）master获取bin-log和post值–>通过脚本实现 
（5）slave上，change master && start slave–>slave.sls

2.需求实现：

(1)在prod环境下载创建modules和mysql目录
[root@linux-node1 prod]# pwd
/srv/salt/prod
[root@linux-node1 prod]# mkdir modules/mysql

（2）配置安装和配置状态文件install.sls
[root@linux-node1 mysql]# cat install.sls 
mysql-install:
  pkg.installed:
    - pkgs:
      - mariadb
      - mariadb-server

mysql-config:
  file.managed:
    - name: /etc/my.cnf
    - source: salt://modules/mysql/files/my.cnf
    - user: root
    - gourp: root
    - mode: 644
[root@linux-node1 mysql]# cp /etc/my.cnf files/

（3）在主上配置mariadb-server.cnf,并更改server_id，以及创建主从用户
[root@linux-node1 mysql]# cat master.sls 
include:
  - modules.mysql.install

master-config:
  file.managed:
    - name: /etc/my.cnf.d/mariadb-server.cnf
    - source: salt://modules/mysql/files/mariadb-server-master.cnf
    - user: root
    - group: root
    - mode: 0644

master-grant:
  cmd.run:
    - name: mysql -e "grant replication slave on *.* to repl@'192.168.56.0/255.255.255.0' identified by '123456';flush privileges;"
[root@linux-node1 mysql]# cp /etc/my.cnf.d/mariadb-server.cnf files/mariadb-server-master.cnf 
[root@linux-node1 mysql]# cp /etc/my.cnf.d/mariadb-server.cnf files/mariadb-server-slave.cnf 

#修改主从的配置文件的server_id和开启主上的log-bin功能
[root@linux-node1 mysql]# vim files/mariadb-server-master.cnf 
[mysqld]
server_id=1111
log-bin=mysql-bin
[root@linux-node1 mysql]# vim files/mariadb-server-slave.cnf 
[mysqld]
server_id=2222

（4）编写shell脚本获取bin-log值和pos值
[root@linux-node1 mysql]# cat files/start-slave.sh 
#!/bin/bash
for i in `seq 1 10`
do
    mysql -h 192.168.56.11 -urepl -p123456 -e "exit"
    if [ $? -eq 0 ];then
        Bin_log=`mysql -h 192.168.56.11 -urepl -p123456 -e "show master status;"|awk  'NR==2{print $1}'`
        POS=`mysql -h 192.168.56.11 -urepl -p123456 -e "show master status;"|awk  'NR==2{print $2}'`
    mysql -e "change master to master_host='192.168.56.11', master_user='repl', master_password='123456', master_log_file='$Bin_log', master_log_pos=$POS;start slave;"
    exit;
    else
        sleep 60;
    fi
done

（5）从库上配置slave，并启动
[root@linux-node1 mysql]# cat slave.sls 
include:
  - modules.mysql.install

slave-config:
  file.managed:
    - name: /etc/my.cnf.d/mariadb-server.cnf
    - source: salt://modules/mysql/files/mariadb-server-slave.cnf
    - user: root
    - group: root
    - mode: 0644

start-slave:
  file.managed:
    - name: /tmp/start-slave.sh
    - source: salt://modules/mysql/files/start-slave.sh
    - user: root
    - group: root
    - mode: 755
  cmd.run:
    - name: /bin/bash /tmp/start-slave.sh

View Code

3、HAproxy+Keepalived

（1）pkg配置管理

[root@linux-node1 modules]# mkdir pkg
[root@linux-node1 pkg]# vim pkg-init.sls 
pkg-init:
  pkg.installed:
    - names:
      - gcc
      - gcc-c++
      - glibc
      - make
      - autoconf
      - openssl
      - openssl-devel
[root@linux-node1 pkg]# salt 'linux-node1*' state.sls modules.pkg.pkg-init saltenv=prod test=True

（2）haproxy配置管理

[root@linux-node1 modules]# mkdir haproxy/files -p
[root@linux-node1 haproxy]# cat haproxy.sls 
include:
  - pkg.pkg-init

haproxy-install:
  file.managed:
    - name: /usr/local/src/haproxy-1.5.3.tar.gz
    - source: salt://modules/haproxy/files/haproxy-1.5.3.tar.gz
    - user: root
    - group: root
    - mode: 755
  cmd.run:
    - name: cd /usr/local/src && tar -zxvf haproxy-1.5.3.tar.gz && cd haproxy-1.5.3 && make TARGET=linux26 PREFIX=/usr/local/haproxy && make install PREFIX=/usr/local/haproxy
    - unless: test -d /usr/local/haproxy
    - require:
      - pkg: pkg-init
      - file: haproxy-install

/etc/init.d/haproxy:
  file.managed:
    - source: salt://modules/haproxy/files/haproxy.init
    - user: root
    - group: root
    - mode: 755
    - require:
      - cmd: haproxy-install

net.ipv4.ip_nonlocal_bind:
  sysctl.present:
    - value: 1

haproxy-config-dir:
  file.directory:
    - name: /etc/haproxy
    - mode: 755
    - user: root
    - group: root

haproxy-init:
  cmd.run:
    - name: chkconfig --add haproxy
    - unless: chkconfig --list | grep haproxy
    - require:
      - file: /etc/init.d/haproxy
[root@linux-node1 haproxy]# cp /usr/local/src/haproxy-1.5.3.tar.gz files/
[root@linux-node1 haproxy]# cp /usr/local/src/haproxy-1.5.3/examples/haproxy.init files/
[root@linux-node1 haproxy]# tree 
.
├── files
│   ├── haproxy-1.5.3.tar.gz
│   └── haproxy.init
└── install.sls

View Code

（3）Keepalived配置管理

[root@linux-node1 keepalived]# vim install.sls 
include:
  - pkg.pkg-init

keepalived-install:
  file.managed:
    - name: /usr/local/src/keepalived-1.2.17.tar.gz
    - source: salt://modules/keepalived/files/keepalived-1.2.17.tar.gz
    - user: root
    - gourp: root
    - mode: 755
  cmd.run:
    - name: cd /usr/locall/src && tar -zxf keepalived-1.2.17.tar.gz && cd keepalived-1.2.17 && ./configure --prefix=/usr/local/keepalived --disable-fwmark && make && make install
    - unless: test -d /usr/local/keepalived
    - require:
      - pkg: pkg-init
      - file: keepalived-install

/etc/sysconfig/keeplived:
  file.managed:
    - source: salt://modules/keepalived/files/keepalived-sysconfig
    - user: root
    - gourp: root
    - mode: 644

/etc/init.d/keepalived:
  file.managed:
    - sourcd: salt://modules/keepalived/files/keepalived.init
    - user: root
    - group: root
    - mode: 755

keepalive-init:
  cmd.run:
    - name: chkconfig --add keepalived
    - unless: chkconfig --list | grep keepalived
    - require:
      - file: /etc/init.d/keepalived

/etc/keepalived:
  file.directory:
    - user: root
    - group: root
[root@linux-node1 keepalived]# cp /usr/local/src/keepalived-1.2.17.tar.gz files/
[root@linux-node1 init.d]# pwd
/usr/local/src/keepalived-1.2.17/keepalived/etc/init.d
[root@linux-node1 init.d]# cp keepalived.init /srv/salt/prod/modules/keepalived/files/
[root@linux-node1 init.d]# cp keepalived.sysconfig /srv/salt/prod/modules/keepalived/files/
[root@linux-node1 keepalived]# tree 
.
├── files
│   ├── keepalived-1.2.17.tar.gz
│   ├── keepalived.init
│   └── keepalived.sysconfig
└── install.sls

View Code

4、Nginx+PHP

（1）Nginx配置管理

[root@linux-node1 modules]# mkdir pcre
[root@linux-node1 pcre]# cat init.sls 
pcre-install:
  pkg.installed:
    - names: 
      - pcre
      - pcre-devel
[root@linux-node1 modules]# mkdir user
[root@linux-node1 user]# cat www.sls 
www-user-group:
  group.present:
    - name: www
    - gid: 1000

  user.present:
    - name: www
    - fullname: www
    - shell: /sbin/nologin
    - uid: 1000
    - gid: 1000
[root@linux-node1 modules]# mkdir nginx/files -p
[root@linux-node1 nginx]# cp /usr/local/src/nginx-1.12.2.tar.gz files/
[root@linux-node1 nginx]# tree 
.
├── files
│   └── nginx-1.12.2.tar.gz
└── install.sls
[root@linux-node1 nginx]# cat install.sls 
include:
  - modules.pcre.init
  - modules.user.www
  - modules.pkg.pkg-init

nginx-source-install:
  file.managed:
    - name: /usr/local/src/nginx-1.12.2.tar.gz
    - source: salt://modules/nginx/files/nginx-1.12.2.tar.gz
    - user: root
    - group: root
    - mode: 755
  cmd.run:
    - name : cd /usr/local/src && tar -zxf nginx-1.12.2.tar.gz && cd nginx-1.12.2 && ./configure --prefix=/usr/local/nginx --user=www --group=www --with-http_ssl_module --with-http_stub_status_module --with-file-aio --with-http_dav_module && make && make install && chown -R www.www /usrl/local/nginx
    - unless: test -d /usr/local/nginx
    - require:
      - user: www-user-group
      - file: nginx-source-install
      - pkg: pcre-install
      - pkg: pkg-init
[root@linux-node1 nginx]# salt 'linux-node1*' state.sls modules.nginx.install saltenv=prod test=True

View Code

（2）PHP配置管理

[root@linux-node1 modules]# mkdir php/files -p
[root@linux-node1 php]# cp /usr/local/src/php-5.6.9/sapi/fpm/init.d.php-fpm files/
[root@linux-node1 php]# cp /usr/local/php/etc/php-fpm.conf.default files/
[root@linux-node1 php]# cp /usr/local/src/php-5.6.9/php.ini-production files/
[root@linux-node1 php]# cp /usr/local/src/php-5.6.9.tar.gz files/
[root@linux-node1 php]# tree 
.
├── files
│   ├── init.d.php-fpm
│   ├── php-5.6.9.tar.gz
│   ├── php-fpm.conf.default
│   └── php.ini-production
└── install.sls
[root@linux-node1 php]# cat install.sls 
include:
  - modules.user.www

pkg-php:
  pkg.installed:
    - names:
      - mysql-devel
      - openssl-devel
      - swig
      - libjpeg-turbo
      - libjpeg-turbo-devel
      - libpng
      - libpng-devel
      - freetype
      - freetype-devel
      - libxml2
      - libxml2-devel
      - zlib
      - zlib-devel
      - libcurl
      - libcurl-devel

php-source-install:
  file.managed:
    - name: /usr/local/src/php-5.6.9.tar.gz
    - source: salt://modules/php/files/php-5.6.9.tar.gz
    - user: root
    - gourp: root
    - mode: 755
  cmd.run:
    - name: cd /usr/local/src && tar -zxf php-5.6.9.tar.gz && cd php-5.6.9 && ./configure --prefix=/usr/local/php -with-pdo-mysql=mysqlnd --with-mysqli=mysqlnd --with-mysql=mysqlnd --with-jpeg-dir --with-png-dir --with-zlib --enable-xml  --with-libxml-dir --with-curl --enable-bcmath --enable-shmop --enable-sysvsem  --enable-inline-optimization --enable-mbregex --with-openssl --enable-mbstring --with-gd --enable-gd-native-ttf --with-freetype-dir=/usr/lib64 --with-gettext=/usr/lib64 --enable-sockets --with-xmlrpc --enable-zip --enable-soap --disable-debug --enable-opcache --enable-zip --with-config-file-path=/usr/local/php-fastcgi/etc --enable-fpm --with-fpm-user=www --with-fpm-group=www && make && make install
    - require:
      - file: php-source-install
      - user: www-user-group
    - unless: test -d /user/local/php

php-ini:
  file.managed:
    - name: /usr/local/php/etc/php.ini
    - source: salt://modules/php/files/php.ini-production
    - user: root
    - group: root
    - mode: 644

php-fpm:
  file.managed:
    - name: /usr/local/php/etc/php-fpm.conf
    - source: salt://modules/php/files/php-fpm.conf.default
    - user: root
    - group: root
    - mode: 644

php-service:
  file.managed:
   - name: /etc/init.d/php-fpm
   - source: salt://modules/php/files/init.d.php-fpm
   - user: root
   - group: root
   - mode: 755
  cmd.run:
    - name: chkconfig --add php-fpm
    - unless: chkconfig --list | grep php-fpm
    - require:
      - file: php-service
  service.running:
    - name: php-fpm
    - enable: True
    - reload: True
    - require:
      - file: php-ini
      - file: php-fpm
      - file: php-service
      - cmd: php-service

View Code

统一使用的功能都抽象成一个模块，如安装以及基本配置（nginx中包含include，php中包含的include，那么就可以将nginx.conf放在功能模块，而虚拟主机配置文件，可以放在业务模块）。 
其它配置和服务启动可以抽象在一个业务模块，每一个业务都是使用不同的配置文件。

服务全部使用www用户，统一id，只开放8080端口，对于web服务只开放ssh的8022端口以及web的8080端口。其余不用的端口一律不开启

这里将nginx，php都抽象成一个模块，把安装和基础配置都放在了modules中，在nginx衍生的业务模块web目录下，做一个bbs的虚拟主机。

[root@linux-node1 base]# vim top.sls 
prod:
  '*':
    - web.bbs
[root@linux-node1 base]# salt '*' state.highstate