Nginx+keepalived dual master configuration
This kind of use of two VIP addresses, the front end uses two machines to install Nginx and Keepalived, each as a backup, and two machines work at the same time. When one of the machines fails, the requests of the two machines are transferred to one machine.
1. Experimental environment:
使用VMware Workstattion来创建虚拟机做实验，虚拟机网卡使用桥接模式，两台单网卡，两台双网卡。Environment:
CentOS7 system, 4 hosts
主机名和IP设置：
两台Nginx，做为反向代理服务器，提供调度器高可用：
Kee-Ngx-01.hjun.com 172.16.100.10
Kee-Ngx-02.hjun.com 172.16.100.20
两台upstream server，提供web服务高可用：
Up-Servs-01.hjun.com 172.16.100.30
Up-Servs-02.hjun.com 172.16.100.40
keepalived的vrrp虚拟路由器的VIP为：
172.16.100.60
172.16.100.80Topology and address planning:
This experiment uses a network simulator and VMware Workstation Pro to achieve:
Client-PC simulates extranet users accessing the services provided by the server area.
2. Network configuration:
(1) The configuration of switches SW-1 and SW-2 are as follows:
(2) The configuration of router AR1:
[AR1]int GigabitEthernet 0/0/1
[AR1-GigabitEthernet0/0/1]ip add 202.1.1.100 24
[AR1]int GigabitEthernet 0/0/0
[AR1-GigabitEthernet0/0/0]ip add 192.168.1.254 24(3) Configure routing:
[SW-2]ip route-static 192.168.1.0 24 Vlanif30 202.1.1.100
[AR1]ip route-static 100.1.20.0 24 GigabitEthernet 0/0/1 202.1.1.101(4), configure NAT:
configure ACL, configure data packet
[AR1]acl 2000
[AR1-acl-basic-2000]rule 10 permit source 192.168.1.0 0.0.0.255
[AR1-acl-basic-2000]rule 200 deny source any #拒绝其他所有地址Configure NAT, when the data matches acl 2000, it will perform address translation (NAT) out of GigabitEthernet 0/0/1:
[AR1]interface GigabitEthernet 0/0/1
[AR1-GigabitEthernet0/0/1]nat outbound 2000Third, the host configuration:
1. Configure the host name and IP address of
each host : (1) Set the host name of each host:
[root@localhost ~]# hostnamectl set-hostname Kee-Ngx-01.hjun.com
[root@localhost ~]# hostnamectl set-hostname Kee-Ngx-02.hjun.com
[root@localhost ~]# hostnamectl set-hostname Up-Servs-01.hjun.com
[root@localhost ~]# hostnamectl set-hostname Up-Servs-02.hjun.com(2) Configure the IP address of each host:
Note: Two nginx proxy servers have two network cards, configure dual IP, the network card connected to the internal network does not need to configure the gateway:
IP configuration of nginx proxy server 1:
IP configuration of nginx proxy server 2
The IP configuration of the two back-end web servers also called upstrem server:
2. Configure host name resolution:
[root@localhost ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
Kee-Ngx-01.hjun.com 172.16.100.10
Kee-Ngx-02.hjun.com 172.16.100.20
Up-Servs-01.hjun.com 172.16.100.30
Up-Servs-02.hjun.com 172.16.100.403. Close selinux and firewalld
~]# setenforce 0
~]# sed -i 's#SELINUX=disabled#SELINUX=enforcing#g' /etc/selinux/config
~]# systemctl stop firewalld
~]# systemctl disable firewalld4. Configure clock synchronization:
I don't have a time server, here I only configure the same time for each:
[root@localhost ~]# date '102715562020.30'5. Install the software package on each node:
(1) Install keepalived nginx on the proxy server:
~]# yum install keepalived nginx -y(2) The back-end server web server is also called upstrem server to install httpd:
~]# yum install httpd -y(3) Configure the web test page in the upstrem server. In order to see the load balancing effect, configure different pages in the two RealServers, as follows:
[root@RS-1 ~] echo "Welcome To Up-Servs-01" > /var/www/html/index.html
[root@RS-2 ~] echo "Welcome To Up-Servs-02 ！！！" > /var/www/html/index.html6. Configure nginx on the nginx reverse proxy server as the reverse proxy server of the back-end upstream server:
(1) The configuration of nginx on the host Kee-Ngx-01.hjun.com is as follows:
First, add the upstream server group of the back-end server in the http configuration section of the main configuration file nginx.conf of nginx; then configure the nginx direction proxy in the server section and call the upstream server group name, so that the client's request will be forwarded by nginx The back-end upstream server provides services.
http {
...............................................
include /etc/nginx/conf.d/*.conf;
upstream websrvs { #指定upstream server的名称为websrvs，调用时就调用websrvs
server 172.16.100.30 weight=1;
server 172.16.100.40 weight=1;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
root /usr/share/nginx/html;
include /etc/nginx/default.d/*.conf;
location / {
proxy_pass http://websrvs/; #调用upstream组名称，做反向代理
}(2) The configuration of the nginx of the host Kee-Ngx-02.hjun.com is the same as that of Kee-Ngx-01, just
copy the nginx configuration of Kee-Ngx-01 to Kee-Ngx-02.
[root@Kee-Ngx-01 ~]# scp -p /etc/nginx/nginx.conf [email protected]'s password:
nginx.conf 100% 2611 1.6MB/s 00:00
[root@Kee-Ngx-01 ~]# (3) Start the nginx service on Kee-Ngx-01.hjun.com and Kee-Ngx-02.hjun.com:
[root@Kee-Ngx-01 ~]# systemctl start nginx
[root@Kee-Ngx-02 ~]# systemctl start nginxAfter the two hosts have installed and started the nginx service, both hosts can proxy the back-end server, and
both can access the services provided by the upstream server of the back-end through the IP of the two hosts ;
To test, first use the IP172.16.100.10 of Kee-Ngx-01.hjun.com to visit:
Refresh the page as follows:
Nginx proxy access to back-end services also achieves the effect of load balancing.
To test again, use the 172.16.100.20 of Ke-nginx01.hjun.com to access the service, the page displays as follows:
Refresh the page as follows: 
This nginx server proxy access to the back-end service also achieves the effect of load balancing.
2, arranged so that two keepalived nginx reverse proxy server can act as a highly available, is to make nginx reverse proxy service
also highly available; moreover achieved here nginx reverse proxy server dual master model;
To configure a dual-master nginx reverse proxy server, you need to define two vrrp instances in the keepalived configuration file, that is, configure
two vrrp virtual routers, and different virtual routers need to use different vrid numbers.
We configure nginx for high-availability use, and we need keepalived to be able to monitor the running status of nginx. When keepalived finds that
nginx fails, the vip of the master of the keepAlived virtual router will be switched to the BACKUP node of the virtual router;
Added killall command to send a signal to the program.
If the killall command needs to install the psmisc package, install:
~]# yum install psmisc -y
[root@Kee-Ngx-01 ~]# killall -0 nginx #使用killall发送信号0给处于启动状态的nginx进程
[root@Kee-Ngx-01 ~]# echo $? #查看命令执行后状态返回值，返回值是0
0
[root@Kee-Ngx-01 ~]# systemctl stop nginx #停止nginx
[root@Kee-Ngx-01 ~]# killall -0 nginx #再使用killall发送信号0给处于关闭状态的nginx进程
nginx: no process found #提示没有找到进程
[root@Kee-Ngx-01 ~]# echo $? #查看命令执行后状态返回值，返回值是1
1
[root@Kee-Ngx-01 ~]#In this way, we can check the status of nginx to determine whether the nginx process exists by sending a 0 signal to nginx using the killall command.
(1) Configure the keepalived configuration file of node 1 (Kee-Ngx-01.hjun.com):
[root@Kee-Ngx-01 ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id Ke-dr01 #指定虚拟路由器的router_id
}
vrrp_script chk_mt #配置检测nginx状态的脚本
{
script "/etc/keepalived/nginx.sh" #转到检测nginx脚本的路径，这个脚本需要在keepalived配置文件之外定义
interval 2
weight -10
}
vrrp_instance VI_1 { #指定第一台虚拟路由器，名称为VI_1
state MASTER #当前节点的虚拟路由器为MASTER
interface ens37
virtual_router_id 60 #虚拟路由器的id
priority 100 #虚拟路由器的优先级
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
100.1.20.60/24 dev ens37 label ens37:1 #虚拟路由器的VIP地址
}
track_script {
chk_mt #调用检测nginx状态的脚本
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
vrrp_instance VI_2 { #指定配置第二台虚拟路由器
state BACKUP # 第二台虚拟路由器在当前节点为BACKUP
interface ens37
virtual_router_id 80
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 2222
}
virtual_ipaddress {
100.1.20.80/24 dev ens37 label ens37:1
}
track_script {
chk_mt
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
[root@Kee-Ngx-01 ~]# (2) Configure the keepalived configuration file of node 2 (Kee-Ngx-02.hjun.com):
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id Ke-dr02
}
vrrp_script chk_mt
{
script "/etc/keepalived/nginx.sh"
interval 2
weight -10
}
vrrp_instance VI_1 {
state BACKUP
interface ens37
virtual_router_id 60
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
100.1.20.60/24 dev ens37 label ens37:1
}
track_script {
chk_mt
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
vrrp_instance VI_2 {
state MASTER
interface ens37
virtual_router_id 80
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 2222
}
virtual_ipaddress {
100.1.20.80/24 dev ens37 label ens37:1
}
track_script {
chk_mt
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
[root@Kee-Ngx-02 ~]# (3) Define scripts for detecting nginx status on both nodes. The scripts are as follows (the two host scripts are the same):
[root@Kee-Ngx-01 keepalived]# cat nginx.sh
#!/bin/bash
killall -0 nginx &> /dev/null
[root@Kee-Ngx-0 keepalived]# Give script execution permissions:
[root@Kee-Ngx-01 keepalived]# chmod a+x nginx.sh (4) Start keepalived after the configuration files of keepalived of the two nodes are configured:
[root@Kee-Ngx-01 ~]# systemctl start keepalived
[root@Kee-Ngx-02 ~]# systemctl start keepalived(5) At this time, both nodes will have VIP addresses, but the VIP addresses are different:
Using 100.1.20.60 access, you can also access the pages provided by the two back-end servers, and you can access different pages each time you refresh to achieve load balancing effects:
Refresh the page and display
Using 100.1.20.80 access, you can also access the pages provided by the two back-end servers, and each refresh can access different pages to achieve load balancing effects:
Refresh the page and display
(6) Failure test:
Stop nginx of node 1 (Kee-Ngx-01), so that both VIPs will be on node 2 (Kee-Ngx-02):
[root@Kee-Ngx-01 ~]# systemctl stop nginxCheck the IP of node 2 (Kee-Ngx-02), such as:
The back-end web service can still serve:
