Table of contents
1. Keepalived High Availability Detailed Explanation
2. Introduction and brief introduction
2. LVS+keepalived configuration example (preemption mode)
1. Configure NFS shared storage
2. Configure the node web service (the two configurations are the same)
3. Configure the active and standby LVS+keepalived load scheduler
4. Explanation and solution of split brain phenomenon
1. Keepalived High Availability Detailed Explanation
1. Application scenarios
In enterprise applications, a single server bears the risk of a single point of failure for the application. Once a single point of failure occurs, enterprise services will be interrupted, causing great harm. Therefore, clusters are required to achieve high availability and ensure service stability.
2. Introduction and brief introduction
Keepalived is a high-availability solution for LVS services based on the VRRP protocol, which can solve the problem of single point of failure in static routing.
Support automatic failover (Failover) and node health status check (Health Checking) - to judge the availability of the LVS load scheduler and node server, when the master host fails, switch to the backup node in time to ensure normal business, when the master failed host recovers, it will It rejoins the cluster and is operational.
In an LVS service cluster, there are usually two servers with the roles of master server (NASTER) and backup server (BEACKUP), but they appear will send vRt notification information to the backup server. When the backup server cannot receive When the vRt message arrives, that is, when the main server is abnormal, the backup server will take over the virtual IP and continue to provide services, thus ensuring high availability.
3. Main modules and functions
core module : the core of keepalived, responsible for the startup and maintenance of the main process and the loading and parsing of global configuration files
vrrp module : It is used to implement the VRRP protocol and is used for checking and switching between the active and standby schedulers.
check module : responsible for health check node server, common methods include port check and URL check.
2. LVS+keepalived configuration example (preemption mode)
LVS-NAT reference LVS-NAT load cluster advantages and deployment examples
LVS-DR reference LVS-DR load cluster advantages and deployment examples
The following experiment uses the preemptive mode of LVS-DR+keepalived to achieve high availability load cluster
Web Server 1: 192.168.116.10 (VIP 192.168.116.100)
Web Server 2: 192.168.116.20 (VIP 192.168.116.100)NFS shared storage: 192.168.116.30
LVS+keepalived main load scheduler: 192.168.116.40 (VIP 192.168.116.100)
LVS+keepalived standby load scheduler: 192.168.116.30 (VIP 192.168.116.100)
Gateway/Router: 192.168.116.2
Client: 192.168.116.50
1. Configure NFS shared storage
systemctl stop firewalld.service
setenforce 0
yum -y install nfs-utils rpcbind
mkdir /opt/nfs/server1 /opt/nfs/server2
chmod -R 777 /opt/nfs
vim /etc/exports
/opt/nfs 192.168.116.0/24(rw,sync)
/opt/nfs/server1 192.168.116.0/24(rw,sync)
/opt/nfs/server2 192.168.116.0/24(rw,sync)
systemctl restart rpcbind.service
systemctl restart nfs.serviceCheck published sharing policies
Add the web test page in the shared directory respectively (the two can be different)
2. Configure the node web service (the two configurations are the same)
Configure the ip address and specify the gateway as the ip address of the gateway/router
Install nginx to provide web services (apache, nginx, etc., only for experiments)
Mount the website root directory to their respective shared directories
Check if the sharing is successful
Add loopback network card virtual ip
cp /etc/sysconfig/network-scripts/ifcfg-lo /etc/sysconfig/network-scripts/ifcfg-lo:0
vim /etc/sysconfig/network-scripts/ifcfg-lo:0
DEVICE=lo:0
IPADDR=192.168.116.100
NETMASK=255.255.255.255
ONBOOT=yes
ifup lo:0
Add a static route (block the packet in the loopback network card)
#临时配置
route add -host 192.168.116.100 dev lo:0
#永久配置
vim /etc/rc.local
/sbin/route add -host 192.168.116.100 dev lo:0
chmod +x /etc/rc.d/rc.localAdjust the kernel's ARP response parameters (prevent updating the VIP's MAC address to avoid conflicts)
vim /etc/sysctl.conf
#添加
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
#加载配置
sysctl -p3. Configure the active and standby LVS+keepalived load scheduler
Turn off both firewalls, download keepalived and ipvsadm, and modify kernel parameters
systemctl stop firewalld.service
setenforce 0
modprobe ip_vs
yum -y install ipvsadm keepalived
vim /etc/sysctl.conf
net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0
sysctl -pModify the main loader configuration file
Modify the standby loader configuration file (consistent with the main configuration, just modify the following part)
Then start keepalived and then start ipvsadm (ipvsadm will automatically load the configuration in keepslived)
#启动keepalived
systemctl restart keepalived.service
ipvsadm-save > /etc/sysconfig/ipvsadm
#启动ipvsadm
systemctl restart ipvsadm.service
view master server
View standby server
Access test (at this time, the main loader should provide services)
Now stop or shut down the main loader service (simulated downtime), and continue to visit
The access is successful, this time it is obviously forwarded by the standby loader
3. Non-preemptive mode
Preemption and non-preemption mode of keepalived
The preemption mode means that after the MASTER recovers from a failure, it will preempt the VIP from the BACKUP node.Non-preemptive mode means that after the recovery of MASTER, the VIP after BACKUP is upgraded to MASTER is not preempted. The state of the two non-preemptive nodes must be BACKUP, and nopreempt must be added before the priority .
Note: After this configuration, we should pay attention to the order of starting the services, and the ones that start first will get the master permission, which has nothing to do with the priority.
4. Explanation and solution of split brain phenomenon
1. Explain
In the preemptive mode, MASTER needs to send messages regularly to notify BACKUP that it is still operating. However, when MASTER is still operating, but the line or switch fails and BACKUP cannot receive the notification, it will be considered that MASTER has failed. At this time BACKUP preempting the VIP will cause the VIP to exist at the same time, which is the split-brain phenomenon .
2. Solution
Dual-link communication is used between the active and standby servers;
Use scripts to monitor the network status of the master and backup in real time, and then take measures according to the script logic (turn off the keepalived server of the master server);
Use a third-party monitoring system to monitor the status of the active and standby servers and the network in real time, and then take measures according to the situation (turn off the keepalived server of the main server).