Chapter 1: Introduction to keepalived
The
purpose of VRRP protocol is to solve the problem of single point of failure of static routing
Chapter 2: The working principle of keepalived
2.1 High availability function (failover) as system network service
keepalived高可用功能实现的基本原理为:
两台主机同时安装好keepalived软件并启动服务,开始正常工作时
角色为Master的主机获得所有资源并对用户提供服务
角色为Backup的主机作为Master主机的热备;
当角色为Master的主机失效或出现故障时
角色为Backup的主机将自动接管Master主机的所有工作,包括接管VIP资源及相应资源服务
而当角色为Master的主机故障修复后,又会自动接管回他原来处理的工作
角色为Backup的主机则同时释放Master主机失效时他接管的工作
此时,两台主机将恢复到启动时各自的原始角色及工作状态
2.2 What is VRRP
VRRP,全称Virtual Router Redundancy Protocol,中文名为虚拟路由冗余协议
VRRP的出现就是为了解决静态路由的单点故障问题
VRRP是通过一种竞选机制来将路由的任务交给某台VRRP路由器的.
VRRP通过竞选机制来实现虚拟路由器的功能,所有的协议报文都是通过IP多播(Multicast)包(默认的多播地址224.0.0.18)形式发送的
虚拟路由器由VRID(范围0-255)和一组IP地址组成,对外表现为一个周知的MAC地址,:00-00-5E-00-01-{VRID}.
所以,在一个虚拟路由器中,不管谁是Master,对外都是相同的MAC和IP(称之为VIP).
客户端主机并不需要因Master的改变修改自己的路由配置.对它们来说,这种切换是透明的.
在一组虚拟路由器中,只有作为Master的VRRP路由器会一直发送VRRP广播包,此时Backup不会抢占Master
当Master不可用时,Backup就收不到来自Master的广播包了,此时多台Backup中优先级最高的路由器会抢占为Master.
这种抢占是非常快速的(可能只有1秒甚至更少),以保证服务的连续性,处于安全性考虑,VRRP数据包使用了加密协议进行了加密.
2.3 What to say during the interview
解答:
keepalived高可用对之间是通过VRRP通信的,因此,我从VRRP开始给您讲起.
1)VRRP,全称Virtual Router Reduancy Protocol,中文名为虚拟路由器冗余协议,VRRP的出现是为了解决静态路由的单点故障,
2)VRRP是通过一种竞选协议来将路由任务交给某台VRRP路由器的,
3)VRRP用IP多播的方式,(默认多播地址(224.0.0.18))实现高可用对之间通信.
4)工作时主节点发包,备节点接包,当备节点接收不到主节点发的包的时候,就启动接管程序接管主节点的资源.备节点可以有多个,通过优先级竞选,但一般keepalived系统运维工作中都是一对.
5)VRRP使用了加密协议加密数据,但keepalived官方目前还是推荐用明文的方式配置认证类型和密码.
介绍完了VRRP,接下来我在介绍一下keepalived服务的工作原理;
keepalived高可用对之间是通过VRRP进行通信的,VRRP是通过竞选机制来确定主备的,主的优先级高于备,因此,工作时会优先获得所有的资源,备节点处于等待状态,当主挂了的时候,备节点就会接管主节点的资源,然后顶替主节点对外提供服务.
在keepalived服务对之间,只有作为主的服务器会一直发送VRRP广播包,告诉备他还活着,此时备不会抢占主,当主不可用时,即备监听不到主发送的广播包时,就会启动相关服务接管资源,保证业务的连续性,接管速度最快可以小于一秒
Chapter 3: VRRP protocol
VRRP implements the function of a virtual router through the election mechanism.
All protocol packets are through IP multicast packets. The
default multicast address is 224.0.0.18
VIP prerequisites:
1. The virtual public network IP must be truly available
2. The virtual public network IP cannot be repeated
3. The multicast address must be communicable
Chapter 4: Keepalived installation and configuration
1. Install keepalived
yum install keepalived -y
2. Configuration file explanation
global_defs {
router_id lb01 #设置路由ID,每个主机不一样
}
vrrp_instance VI_1 { #设置VRRP组名,同一组组名相同
state MASTER #设置角色状态,分为MASTER BACKUP
interface eth0 #VIP绑定的网卡
virtual_router_id 50 #虚拟路由id,同一组一样
priority 150 #权重,权重越高,优先级越高
advert_int 1 #发送组播间隔
authentication { #设置验证,密码为明文
auth_type PASS
auth_pass 1111
}
virtual_ipaddress { #设定的虚拟IP,这个虚拟IP必须是存在且合法且没有被使用的。
10.0.0.3
}
}
3.lb01 configuration
[root@lb01 ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id lb01
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 50
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.3
}
}
4.lb02 configuration
[root@lb02 ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id lb02
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.3
}
}
5. Start
systemctl start keepalived
6. Test
Turn off any one and observe whether the VIP will drift.
Restore MASTER and observe whether the VIP of BACKUP will disappear.
Chapter Five: Brain Splitting
1. Install the packet capture tool
yum install tcpdump -y
2.lb02 packet capture view
tcpdump -nn -i any host 224.0.0.18
3.lb02 open a new terminal, and then open the firewall
systemctl start firewalld.service
4.lb02 observe the phenomenon of packet capture
Observe if there are VIPs on both sides
5. Add release rules
firewall-cmd --direct --permanent --add-rule ipv4 filter INPUT 0 --in-interface eth0 --destination 224.0.0.18 --protocol vrrp -j ACCEPT
firewall-cmd --direct --permanent --add-rule ipv4 filter INPUT 0 --in-interface eth1 --destination 224.0.0.18 --protocol vrrp -j ACCEPT
systemctl reload firewalld
6.lb02 observe the phenomenon of packet capture
Observe if there are VIPs on both sides
Chapter 6: Keepalived Dual Master Experiment
1.lb01 configuration file
[root@lb01 ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id lb01
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 50
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.3
}
}
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 2222
}
virtual_ipaddress {
10.0.0.4
}
}
2.lb02 configuration file
[root@lb02 ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id lb02
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.3
}
}
vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 51
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 2222
}
virtual_ipaddress {
10.0.0.4
}
}
3. Restart keepalived and observe the phenomenon
systemctl restart keepalived
Chapter 7: Keepalived combined with nginx reverse proxy load balancing
Nginx configuration of lb server:
note! The Nginx configuration of the two lb servers is exactly the same
1. Back up the original configuration
mkdir /backup
cd /etc/nginx/conf.d
mv * /backup
2. Write Nginx configuration file
[root@lb01 /etc/nginx/conf.d]# cat proxy.conf
upstream web_pools {
server 172.16.1.7;
server 172.16.1.8;
}
server {
listen 80;
server_name (www|bbs).mysun.com ;
location / {
proxy_pass http://web_pools;
include proxy_params;
}
}
3. Test and restart nginx
nginx -t
systemctl restart nginx
Keepalived configuration of lb server:
1.lb01 keepalived configuration
[root@lb01 ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id lb01
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 50
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.3
}
}
2.lb02 keepalived configuration
[root@lb02 ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id lb02
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.3
}
}
Web server configuration:
Attention! Two web servers have exactly the same configuration
1.nginx configuration
[root@web01 ~]# cat /etc/nginx/conf.d/www.conf
server {
listen 80;
server_name www.mysun.com;
location / {
root /code;
index www.html;
}
}
2. Write test file
echo "$(hostname)" >/code/index.html
Chapter 8: Anti-crack brain script
1. Problem phenomenon:
1. Nginx hangs, but keep is still alive
2. There are VIPs on both sides
2. Ideas:
Solve the problem that nginx hangs:
1. Write a script
- Start nginx
- If the startup fails twice, stop your keepalived
2. Keepalived calls this script regularly
3. Realization:
1. How the command is implemented
systemctl start nginx
2. Check the nginx process
[root@lb01 ~]# ps -ef|grep nginx|grep -v "grep"
root 1210 1 0 11:21 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx 1211 1210 0 11:21 ? 00:00:00 nginx: worker process
[root@lb01 ~]# ps -ef|grep nginx|grep -v "grep"|wc -l
2
[root@lb01 ~]# ps -ef|grep nginx|grep -v "grep"|wc -l
0
Script content:
[root@lb01 ~]# cat check_web.sh
!/bin/bash
nginx_status=$(ps -C nginx --no-header|wc -l)
if [[ ${nginx_status} == 0 ]]
then
systemctl start nginx &> /dev/null
sleep 1
nginx_status=$(ps -C nginx --no-header|wc -l)
if [[ ${nginx_status} == 0 ]]
then
systemctl stop keepalived
fi
fi
Keepalived call script:
[root@lb01 ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id lb01
}
vrrp_script check_web {
script "/server/scripts/check_web.sh"
interval 5
weight 50
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 50
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.3
}
track_script {
check_web
}
}
4. The second question: split brain
VIP on both sides
Phenomenon:
Both sides have VIP
2 sides Nginx are alive
Nginx of MASTER opposite is still alive
curl -I -s -w "%{http_code}\n" -o /dev/null 10.0.0.5
But I have a VIP again
ip a |grep "10.0.0.3"|wc -l
I will kill myself
systemctl stop nginx
systemctl stop keepalived
Script content:
[root@lb02 /server/scripts]# cat check_vip.sh
#!/bin/bash
master_status=$(curl -I -s -w "%{http_code}\n" -o /dev/null 10.0.0.5)
my_vip=$(ip a |grep "10.0.0.3"|wc -l)
if [ ${master_status} == 200 -a ${my_vip} == 1 ]
then
systemctl stop nginx
systemctl stop keepalived
fi
Keepalived configuration:
[root@lb02 ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id lb02
}
vrrp_script check_web {
script "/server/scripts/check_web.sh"
interval 5
weight 50
}
vrrp_script check_vip {
script "/server/scripts/check_vip.sh"
interval 5
weight 50
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.3
}
track_script {
check_web
check_vip
}
}
Excerpt from: https://www.jianshu.com/p/ab6735606178