Debon Securities using keepalived + nginx deployment of high availability environment
Note: in fact this layer can be removed nginx using 4 keepalived node deployment. (Nadu here as a start deploying a dual nginx, subsequent need to change)
First, demand
1) high availability environment
Nginx two nodes, two nodes cmp, the main primary database
2) Use vip address to access the service cmp
Configuring nginx + keepalived Service
Two, nginx, cmp installation, mysql dual master mode configuration
Reference: Hua Securities --cmp high-availability deployment
Third, related to the environment
| vip nginx:192.168.7.88 vip database: 192.168.7.89 nginx:192.168.7.69、192.168.7.70 cmp:192.168.7.71、192.168.7.72 mysql:192.168.7.73、192.168.7.74 |
Fourth, install and configure the node Nginx keepalived
1. Install keepalived
Method 1: yum install
| yum install -y keepalived |
Second way: rpm installation
| 1）rpm -ivh keepalived-1.3.5-8.el7_6.x86_64.rpm --force --nodeps 2) The lack of library files in usr / lib64 and / usr at / / lib directory libnetsnmpmibs.so.31 libnetsnmpagent.so.31 libnl-3.so.200 libnl Gen-3.so.200 |
2.keepalived service boot from the start
| chkconfig keepalived on |
3. Modify the configuration file keepalived.conf
NOTE: This is the configuration of a standby mode
nginx master node:
| we /etc/keepalived/keepalived.conf MASTER:nginx-1
cat <<EOF > /etc/keepalived/keepalived.conf ! Configuration File for keepalived
global_defs { router_id nginx-1 }
vrrp_script chk_nginx { script "/opt/chk_nginx.sh" interval 2 weight -5 fall 2 rise 1 }
vrrp_instance VI_1 { state MASTER interface eth0 # designated HA monitoring network interfaces ifconfig View mcast_src_ip 192.168.7.69 # Local IP virtual routing virtual_router_id 88 # priority priority 101 # advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.7.88 #vip }
track_script { chk_nginx } } EOF |
nginx from the node:
| we /etc/keepalived/keepalived.conf BACKUP:nginx-2
cat <<EOF > /etc/keepalived/keepalived.conf ! Configuration File for keepalived
global_defs { router_id nginx-2 }
vrrp_script chk_http_port { script "/opt/chk_nginx.sh" interval 2 weight -5 fall 2 rise 1 }
vrrp_instance VI_1 { state BACKUP interface eth0 # designated HA monitoring network interfaces ifconfig View mcast_src_ip 192.168.7.70 # Local IP virtual routing virtual_router_id 88 # priority priority 99 # advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.7.88 #vip }
track_script { chk_http_port } } EOF |
4.keepalived monitoring Nginx script
1) Let keepalived monitor the status of Nginx
| we /opt/chk_nginx.sh
#!/bin/bash counter=$(ps -C nginx --no-heading|wc -l) echo "$counter" if [ "${counter}" = "0" ]; then /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf sleep 2 counter=$(ps -C nginx --no-heading|wc -l) if [ "${counter}" = "0" ]; then /etc/init.d/keepalived stop fi fi |
2）设置脚本文件的权限并测试
| chmod 755 /opt/chk_nginx.sh sh /opt/chk_nginx.sh |
5.keepalived服务相关命令
| service keepalived start #启动服务 service keepalived stop #停止服务 service keepalived restart #重启服务 |
五、在Mysql节点安装和配置keepalived
1.安装和配置keepalived
与nginx配置keepalived大同小异，略。
2.keepalived监控Mysql脚本
1）让keepalived监控Mysql的状态
| vi /opt/chk_mysql.sh
#!/bin/bash counter=$(netstat -na|grep "LISTEN"|grep "3306"|wc -l) echo "$counter" if [ "${counter}" -eq "0" ]; then /etc/init.d/keepalived stop fi |
2）设置脚本文件的权限并测试
| chmod 755 /opt/chk_mysql.sh sh /opt/chk_mysql.sh |
3.验证keepalived服务
1）在mysql两个节点上使用命令查看是否能连接数据库
| mysql -uroot -p -P3306 -h192.168.7.89 |
2）如果不能连接，有以下原因：
- keepalived服务配置失败
- 端口不通
4.修改cmp节点的配置文件
| 1）将fit2cloud.properties、keycloak.properties中的数据库服务器地址改成mysql的vip地址 2）重启cmp服务 service fit2cloud restart |
参考：https://blog.csdn.net/miss1181248983/article/details/89139951
六、配置keepalived日志
1.编辑系统日志配置文件
注：centos7以上是rsyslog
| 1) 在最后一行加上下面的配置 vim /etc/rsyslog.conf
local4.* /etc/keepalived/logs/keepalived.log
2) 重启系统日志服务 service rsyslog restart |
2.编辑keepalived服务文件
| 1）修改最后一行的配置 vi /etc/sysconfig/keepalived
#KEEPALIVED_OPTIONS="-D" KEEPALIVED_OPTIONS="-D -d -S 4"
2）重启keepalived服务 service keepalived restart |
3.查看keeaplived日志
| tail -f /etc/keepalived/keepalived.log |
参考：https://blog.csdn.net/weiyuefei/article/details/78131509
七、验证keepalived服务
1）在主节点使用ip a查看，发现eth0网络接口下多了一个IP（vip地址）
2）关闭主节点的keepalived服务，使用ip a查看发现vip不在eth0下面了；在从节点使用ip a查看，发现eth0网络接口下多了一个IP（vip地址）
图略
3）重启主节点的keepalived服务，使用ip a查看发现vip地址又回到了主节点eth0下面
4）keepalived配置成功
可以通过vip地址访问cmp服务：http://192.168.7.88
八、注意事项
1.在同一个网段内的，若为不同的应用做高可用，不同应用使用不同的VIP，那么vrrp_instance、virtual_router_id的名字在不同的高可用实例必须设置不同的值区分开。
2.主备模式的keepalived设置优先级：设置priority的值，主节点的值设置的大一些，主备节点的优先级的值差距不要太大。
3.nginx不要缺少了重定向配置：
| proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; |
九、安装和配置keepalived服务常见问题
1.离线安装keepalived缺少库文件
解决方法：
1）将缺少的库文件放入/usr/lib64目录下
libnl-3.so.200
libnl-genl-3.so.200
2）重启keepalived服务
2.keepalived.conf文件中的网卡配错了，需要将eth0改为ens192
解决方法：
1）使用ifconfig查看网络配置
2）修改keepalived.conf文件，配置正确的网络接口
3）重启keepalived服务
