Article directory
1. Introduction to Keepalived
1. What is Keepalived?
Keepalived is a service software in cluster management that ensures cluster high availability (HA) . Its function is similar to heartbeat and is used to prevent single points of failure .
The two core functions of keepalived are failover (high availability) and health check . The so-called health check uses TCP three-way handshake, ICMP request, HTTP request, UDP echo request, etc. to keep alive the actual server behind the load balancer (usually the server that carries the real business); while failed switching is mainly the application For load balancers configured in active and standby mode, VRRP is used to maintain the heartbeat of the active and standby load balancers. When a problem occurs with the active load balancer, the standby load balancer carries the corresponding business, thereby minimizing traffic loss and Provide stability of services.
2. Working principle of Keepalived
Keepalived is based on the virtual routing redundancy protocol VRRP to achieve high availability. When the backup cannot receive the vrrp packet, it is considered that the master is down. At this time, a backup needs to be elected as the master based on the priority of VRRP. In this way We can ensure the high availability of the cluster.
The full name of VRRP is Virtual Router Redundancy Protocol, which is the virtual router redundancy protocol. It can be considered as a protocol to achieve high availability of routers. It is to form a routing group by forming N routers that provide the same functions. There is a master and multiple backups in this group. There is a VIP on the master that provides services to the outside world. The master will send a group broadcast. When the backup cannot receive the VRRP packet, it will be considered If the master fails, then it is necessary to elect a backup as the master based on the VRRP priority to ensure the high availability of the router.
How to judge who is the master: Look at which server the VIP is on, who is the master
keepalived mainly has three modules, namely core, check and vrrp. The core module is the core of keepalived and is responsible for the startup and maintenance of the main process as well as the loading and parsing of global configuration files. check is responsible for health checks, including various common check methods. The vrrp module is used to implement the VRRP protocol
What is split-brain?
Split-brain: refers to a high availability (HA) system, when two connected nodes are disconnected, the system that was originally a whole is split into two independent nodes. At this time, the two nodes They begin to compete for shared resources, and backup for resources. The master does not think that it will die. They rush to serve the client, which will lead to system chaos and data damage.
2. Practical combat: Keepalived+Nginx (to achieve high availability of Nginx load balancing)
1. Experimental environment and architecture
1) Experimental environment:
A total of five virtual machines are required:
virtual machine 1, virtual machine 2 : install nginx.
Use the web server function of nginx to provide services as a real web server.
Virtual machine 3, virtual machine 4 : Install nginx and keepalived.
There are two functions: 1. Use the load balancing function of nginx to forward user requests to the real server as a load balancing server. 2. Install keepalived to achieve high availability of load balancing. Virtual machine 3 serves as the master and virtual machine 4 serves as the backup.
Virtual machine 5 : As a client, used for cluster testing without any configuration.
2) Experimental architecture
2. Web server setup
In order to save time, the test content is directly written in the nginx default release directory /usr/share/nginx/html/index.htm without rewriting the configuration file.
Virtual machine 1 performs the following operations:
Install and start nginx, and write test content in the nginx default release directory
[root@web-1 ~]# yum -y install nginx
[root@web-1 ~]# systemctl start nginx
[root@web-1 ~]# echo web-1 > /usr/share/nginx/html/index.html
Virtual machine 2 performs the following operations:
Same operation as virtual machine 1
[root@web-2 ~]# yum -y install nginx
[root@web-2 ~]# systemctl start nginx
[root@web-2 ~]# echo web-2 > /usr/share/nginx/html/index.html
Virtual machine 5 performs the following operations
Verify whether the two web servers are set up properly
3. Construction of load balancing server
Virtual machine 3 implements load balancing and performs the following operations:
[root@nginx-master ~]# vim /etc/nginx/nginx.conf
[root@nginx-master ~]# vim /etc/nginx/conf.d/default.conf
[root@nginx-master ~]# nginx -s reload#Rebuild the loading configuration file
Virtual machine 4 implements load balancing and performs the following operations:
The same configuration as virtual machine 3, just change the name of the address pool.
[root@nginx-backup ~]# vim /etc/nginx/nginx.conf
[root@nginx-backup ~]# vim /etc/nginx/conf.d/default.conf
[root@nginx-backup ~]# nginx -s reload#Rebuild the loading configuration file
Virtual machine 5 performs the following operations:
Verify load balancing
By accessing the load balancing server, you can access the real server in turn, and the load balancing configuration is successful.
4. Keepalived implements load balancing and high availability construction
Virtual machine 3 and virtual machine 4 perform the following operations:
1) Install keepalived
[root@nginx-master ~]# yum -y install keepalived
2) Back up the keepalived configuration file
[root@nginx-master ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
3) Modify configuration file
[root@nginx-master ~]# vim /etc/keepalived/keepalived.conf
Clear all file contents and add the following configuration
! Configuration File for keepalived
global_defs {
#全局配置
router_id director1 #用于识别master与backup，在backup服务器中，此项改为director2
}
vrrp_instance VI_1 {
#vrrp协议配置，用于实现高可用
state MASTER #定义此台服务器的角色
interface ens33 #指定VIP地址绑定的接口，设置为网卡名称
virtual_router_id 80
priority 100 #设置优先级，master设置为100，backup设置为50
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.58.140/24 #设置虚拟IP，即VIP，需要与服务器ip在同一网段
}
}
4) Virtual machine 4 performs the same operation as virtual machine 3, only the content of the configuration file is different.
5) Turn on Keepalived (both active and backup units are turned on)
[root@nginx-master ~]# systemctl enable keepalived#Set up to start automatically
[root@nginx-master ~]# systemctl start keepalived#Start keepalived
[root@nginx-backup ~]# systemctl enable keepalived#Set up to start automatically
[root@nginx-backup ~]# systemctl start keepalived#Start keepalived
6) ip aView vip
7) Virtual machine 5 access VIP
5. Keepalived high availability verification
1) Simulate the keepalived cluster failure and master downtime to verify whether high availability is effective.
master server shuts down keepalived
[root@nginx-master ~]# systemctl stop keepalived
Virtual machine 5 access vip
backup server view vip
2) When the master returns to normal, check the VIP of the high availability cluster
Since the master's priority is 100 and the backup's priority is 50, when the master server returns to normal, the VIP will quickly return to the master, and the master server will become the master in the Keepalived cluster again.
6. Health detection function
In this experiment, keepalived high availability can solve the single point of failure of load balancing, but it cannot solve the problem of Nginx web server failure.
In order to solve this problem, enable the health detection function of Keepalived and let Keepalived execute an external script at a certain time interval. The function of the script is to close the local Keepalived when Nginx fails.
1) Write a health detection script (both keepalived servers do the following)
[root@nginx-master ~]# vim /etc/keepalived/check_nginx_status.sh
The script content is as follows:
#!/bin/bash
/usr/bin/curl -I http://localhost &> /dev/null
if [ $? -ne 0 ];then
systemctl start nginx
if [$? -ne 0 ];then
systemctl stop keepalived
fi
fi
2) Introduce the health monitoring module into keepalived and call
[root@nginx-master ~]# vim /etc/keepalived/keepalived.conf
Script upgrade
1) Use the process method to determine whether nginx can be used, as follows:
ps -C nginx --no-heading | wc -l
When nginx is running normally, use this command and the echo will be 2; when nginx is not running, this command will echo 0, so you can use this command to determine whether nginx is available. Downtime.
2) Write a script
The script content is as follows:
counter=`ps -C nginx --no-heading | wc -l`
if [ ${
counter} -eq 0 ];then
systemctl start nginx
counter=`ps -C nginx --no-heading | wc -l`
if [ ${
counter} -eq 0 ];then
systemctl stop keepalived
fi
fi
Someone proposed to implement the function of automatically turning on keepalived when nginx can be started normally in this script. In fact, this is not necessary. The logic of this script is:
① When the nginx service goes down, first try to restart the nginx service. If it can be restarted, it will not enter the stop keepalived judgment statement.
② If restarting nginx fails, keepalived will be stopped, VIP drift will be realized, and the master will be re-elected.
③What needs to be understood is: Some people say that after nginx crashes and nginx can be used normally after troubleshooting, the function of restarting keepalived in this case should be added to the script. But we should understand that when nginx crashes and cannot be restarted, the script will stop keepalived. At this time, keepalived will not detect the script. Even if nginx is repaired later, keepalived will have stopped at this time. The detection script function cannot be implemented. At present, we are not able to automatically troubleshoot nginx and need to manually troubleshoot. When we finish troubleshooting nginx, we can manually enable nginx and keepalived.
3) Verification
1. Close nginx and simulate nginx downtime, but the vip does not drift, indicating that the script takes effect and re-open nginx.
2. Deliberately add errors to the nginx.conf configuration file so that nginx cannot be restarted.
[root@nginx-master ~]# vim /etc/nginx/nginx.conf
