HA high availability service construction
1. HA Cluster high availability cluster
HA is the abbreviation of High Available. HA Cluster refers to a high-availability cluster, which is an effective solution to ensure business continuity. Generally, there are two or more nodes, and they are divided into active nodes and standby nodes.
FailOver: failover automatically, MASTER/BACKUP, MASTER is down, BACKUP acts as the host.
2. Why introduce HA high availability
Answer: It is mainly to solve the single point of failure of Web services. HA high availability software architecture principle: keepalived/mha
3. Overview of Keepalived software
Keepalived software was originally designed for LVS load balancing software to manage and monitor the status of each service node in the LVS cluster system, and later added the VRRP function that can achieve high availability. Therefore, in addition to being able to manage LVS software, Keepalived can also be used as a high-availability solution software for other services (such as: Nginx, HAproxy, MySQL, etc.).
4. Keepalived composition and principle
The Keepalived software mainly realizes the high availability function (failover mode) through the VRRP protocol. VRRP is the abbreviation of Virtual Router Redundancy Protocol (Virtual Router Redundancy Protocol). The purpose of VRRP is to solve the single point of failure problem of static routing. It can ensure that when individual nodes go down, the entire network can run uninterrupted. (FailOver+VIP Drift)
Use Keepalived for VIP (virtual IP address), and all servers share a virtual VIP implementation.
The virtual routing redundancy protocol can be considered as a protocol to achieve high availability of routers, that is, N routers providing the same function form a router group. There is a master and multiple backups in this group. There is a vip on the master that provides services to the outside world ( The default route of other machines in the local area network where the router is located is the vip), the master will send multicast, and when the backup cannot receive the vrrp packet, it will be considered that the master is down. At this time, a backup needs to be elected according to the priority of VRRP. master. In this way, the high availability of the router can be guaranteed.
The functional architecture of Keepalived is roughly divided into two layers: user space (user space) and kernel space (kernel space) .
Kernel space: It mainly includes two parts: IPVS (IP virtual server, used to realize load balancing of network services) and NETLINK (provides advanced routing and other related network functions).
user space:
WatchDog：负则监控checkers和VRRP进程的状况
VRRP Stack：负载均衡器之间的失败切换FailOver，如果只用一个负载均衡
器，则VRRP不是必须的。
Checkers：负责真实服务器的健康检查healthchecking，是keepalived
最主要的功能。换言之，可以没有VRRP Stack，但健康检查
healthchecking是一定要有的。
IPVS wrapper：用户发送设定的规则到内核ipvs代码.
Netlink Reflector：用来设定vrrp的vip地址等。
keepalived mainly uses three modules, namely core, check and vrrp. The core module is the core of keepalived, which is responsible for the startup and maintenance of the main process and the loading and parsing of the global configuration file. check is responsible for health checks, including various common check methods. The vrrp module is to implement the VRRP protocol.
5. Install Keepalived software
Step 1: Turn off the firewall and SELinux, turn off NetworkManager, and time synchronization.
Step 2: Use the yum command to install Keepalived software on Web01 and Web02 (two machines to form a high-availability cluster)
yum install keepalived -y
Step 3: Understand the location of the keepalived configuration file and log information output
Configuration file:
vim /etc/keepalived/keepalived.conf
log file:
cat /var/log/messages
6. Set the Keepalived configuration file
Simultaneous operation of Web01 and Web02:
vim /etc/keepalived/keepalived.conf
Step 1: Use colon: last line mode, switch the cursor to line 35, then press dG to delete all content after line 35
Step 2: Understand the meaning of each line in the configuration file 19~34 lines
vrrp_instance #组建vrrp实例组的组名（Web01与Web02要保持一致） {
state #角色名称(MASTER/BACKUP)
interface #网卡名称(eth0=>ens33)
virtual_router_id #vrrp组的编号（Web01与Web02要保持一致）
priority #权重（选举=>权重高=>当选几率大）
advert_int 1 #心跳间隔时间
authentication {
auth_type PASS #=> 授权类型（PASS密码）
auth_pass 1111 #=> 组密码（想组建集群，其密码必须一致）
}
virtual_ipaddress {
10.1.1.100 #=> VIP，虚拟的IP地址，需要与集群中的机器保持在同一网段
}
}
After setting, save and exit
Step 3: Start the Keepalived software
systemctl start keepalived
systemctl status keepalived
Use the ip a command in Web01/Web02 to check whether the VIP address is mounted on the network card:
ip a
7. Resolve www.shop.com to the VIP virtual IP address
Find the hosts file in Windows:
ip 域名
例如：
192.168.1.100 www.baidu.com
8. Allow VIP to ping
The domain name is resolved to the VIP, and the server providing the service is accessed through the VIP.
Special attention: vrrp_strict should be annotated in versions after 1.3, otherwise the virtual IP cannot be pinged
After the setting is complete, restart the keepalived software
systemctl restart keepalived
9. Simulate server failure, VIP drift
MASTER：网络无法连接，keepalived软件关闭
Web01 # systemctl stop network
10. Let Keepalived monitor Nginx service
Question: When we manually stop Nginx in the MASTER server, will the VIP drift?
Answer: No, because the keepalived software mainly detects the status of the keepalived service and network conditions. As long as these two are normal, this VIP will not drift.
At this time, a problem arises: Nginx has been stopped. In fact, this MASTER server cannot provide web services to the outside world, so the VIP should also drift normally at this time.
Step 1: Write the nginx.sh script to automatically detect the running status of Nginx (Web01/Web02)
# mkdir /scripts
# vim /scripts/nginx.sh
#!/bin/bash
nginx_status=`ps -C nginx --no-header |wc -l`
if [ $nginx_status -eq 0 ]; then
systemctl stop keepalived
fi
Step 2: Add an executable permission to the nginx.sh file
chmod +x /scripts/nginx.sh
After the setup is complete, don't rush to continue down. First test whether the nginx.sh script is available
/scripts/nginx.sh
Step 3: Configure the nginx.sh script in the keepalived.conf file of the two machines Web01/Web02
vim /etc/keepalived/keepalived.conf
vrrp_script 健康检测脚本对象名称 {
script STRING | QUOTED-STRING ## 指定：<外部脚本>的<调用路径>
interval INTEGER ## 设置：<健康跟踪检测>的<时间间隔>，默认为 1 秒
}
The above is just a detection script defined in the keepalived configuration file, but it has not been called yet.
Step 4: Invoke the check_nginx detection script in the vrrp_instance tag
vrrp_instance VI_1 {
...
track_script {
check_nginx
}
...
}
Step 5: After the setup is complete, restart keepalived
systemctl restart keepalived
Two, Keepalived configuration supplement
1. Preemptive and non-preemptive modes
☆ What is preemptive mode
MASTER: Weight 100
BACKUP: Weight 90
When MASTER fails (downtime), VIP will automatically drift from MASTER to BACKUP server. But when the maintenance of MASTER is completed, the moment keepalived is started, it will preempt the VIP from the BACKUP server. We call this situation "preemptive mode".
Extension: In keepalived, how is it decided who is the MASTER?
Answer: Mainly through the weight configuration and the size of the IP address
① If the weights of Web01 and Web02 are different, when we start keepalived at the same time, it will select the server with a larger weight as the MASTER by default.
② If the weights of Web01 and Web02 are the same, the one with the larger IP address will be given priority as MASTER Web02 (10.1.1.13) > Web01 (10.1.1.11)
☆ Non-preemptive mode
Question: What if we make our HA Cluster a non-preemptive mode?
Answer: There are three steps
Step 1: Change the keepalived.conf configuration file
vrrp_instance VI_1 {
virtual_router_id 51
------------------------ 华丽的分割线 ------------------
nopreempt => 非抢占模式
------------------------ 华丽的分割线 ------------------
priority 100
}
Step 2: Configure the state role as BACKUP
vrrp_instance VI_1 {
------------------------ 华丽的分割线 ------------------
state BACKUP => 所有服务器都设置为BACKUP，没有MASTER/BACKUP之分
------------------------ 华丽的分割线 ------------------
virtual_router_id 51
nopreempt
priority 100
}
Step 3: Restart the keepalived software
systemctl restart keepalived