Environmental description
系统： centos 7.4
软件： keepalived: 1.3.5 nginx: 1.12.2 tcpdump工具
主机：192.168.9.222 192.168.9.223
vip地址： 192.168.9.151keepalived description
Keepalived software mainly realizes high-availability functions through the VRRP protocol. VRRP is the abbreviation of Virtual Router Redundancy Protocol (Virtual Router Redundancy Protocol). The purpose of VRRP is to solve the problem of static routing single point of failure. It can ensure that when individual nodes go down, the entire network can run uninterrupted.
1. Three important functions of the keepalived service
Manage LVS load balancing software
Realize health check of LVS cluster nodes
High availability (failover) as system network service
1.2, Keepalived high availability failover principle
The failover transfer between the hosts by the keepalived high-availability service is realized through vrrp (Virtual Routing Redundancy Protocol). Survival, when the main web fails to send heartbeat information, keeplaived will switch the resource vip to the standby node, but when the main node comes alive again, the standby node will release its resources to the main node and restore the original role.
1.3, the working principle of keepalived
keepalived是通过vrrp协议进行通信的，我们首先需要先了解一下vrrp协议的信息 1）vrrp 虚拟路由冗余协议，vrrp最早是为了解决路由单机故障而出现；
2）vrrp是通过一种竟选协议机制来将路由任务交给某台vrrp rs的；
3）vrrp是通过多播的方式实现高可用对之间通信；
4）备节点可以有多个通过优先级竞选，但一般keepalived系统运维工作都是一对；避免竞争产生的问题；
5）vrrp使用了加密协议加密数据，但keepalived官方目前还是推荐用明文的方式配置认证类型和密码2. Installation and configuration
2.1, keepalived installation
# 安装keepalived 这里两台机器都需要安装
Host# yum -y install keepalived libnl3-devel ipset-devel nginx
# 查看安装的相关包
Host# rpm -ql keepalived
/etc/keepalived
/etc/keepalived/keepalived.conf
/etc/sysconfig/keepalived
/usr/bin/genhash
/usr/lib/systemd/system/keepalived.service
/usr/libexec/keepalived
/usr/sbin/keepalivedDefault configuration description
Host # cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
[email protected]
[email protected]
[email protected]
}
notification_email_from [email protected]
smtp_server 192.168.200.1
smtp_connect_timeout 30
# 从这往上是配置邮件信息的
router_id LVS_DEVEL # 用于标识本节点的名称
vrrp_skip_check_adv_addr 默认是不跳过检查。检查收到的VRRP通告中的所有地址可能会比较耗时，设置此命令的意思是，如果通告与接收的上一个通告来自相同的master路由器，则不执行检查(跳过检查)。
vrrp_strict #严格执行VRRP协议规范，此模式不支持节点单播
vrrp_garp_interval 0 # 接口发送ARP之间的延迟
vrrp_gna_interval 0 #
}
vrrp_instance VI_1 {
state MASTER # 状态有两个 MASTER 主 | BACKUP 从
interface eth0 # 对外的网卡接口，ifconfig 或者ip addr show可查看
virtual_router_id 51 # 虚拟路由id，每个节点设置必须一样，相同的ID为一组
priority 100 # 优先级
advert_int 1 # 主往从发送多播消息的间隔时长
authentication { # 认证信息
auth_type PASS
auth_pass 1111
}
virtual_ipaddress { # 虚拟ip 可以多个，但建议一个就OK了
192.168.200.16
192.168.200.17
192.168.200.18
}
}
virtual_server 192.168.200.100 443 { # 虚拟服务器地址 IP 对外提供服务的端口
delay_loop 6 # 健康检查时长 单位秒
lb_algo rr # 负载均衡算法 一般是 rr但 wlc
lb_kind NAT # 负载均衡转发规则，一般用dr,nat调度器会有瓶颈问题
persistence_timeout 50 # http服务会话时长 单位秒
protocol TCP # 协议 tcp
real_server 192.168.201.100 443 { # 真实的对外提供服务的地址跟IP
weight 1 # 权重 权重越高转发优先级越高
SSL_GET { # HTTP_GET | SSL_GET | TCP_CHECK
url {
path /index.html
digest e93e7f6cfbc7c343707f21e2f681dd31
}
connect_timeout 3 # 服务连接端口
nb_get_retry 3 # 服务连接失败重试次数
delay_before_retry 3 # 重试连接间隔 单位 秒
}
}
}Related extensions: About HTTP_GET | SSL_GET | TCP_CHECK usage
2.2, configuration
真实使用配置
! Configuration File for keepalived
global_defs {
notification_email {
[email protected]
[email protected]
[email protected]
}
notification_email_from [email protected]
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id keepalived
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state MASTER
interface ens160
virtual_router_id 222
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
192.168.9.151
}
}
virtual_server 192.168.9.151 80 {
delay_loop 60
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.9.222 8080 { 两台真实主机
weight 1
HTTP_GET {
url {
path /
digest 0b03c354bbc6af44b42712a6f6497dc8
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.9.223 8080 {
weight 1
HTTP_GET {
url {
path /
digest 78d47efe7fe7916ee20e034bfe24c5b7
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
keepalived]# genhash -s 192.168.9.222 -p 8080 -u /index.html
MD5SUM = 0b03c354bbc6af44b42712a6f6497dc8
将这个获取到的值填到 digest xxxxxxxxxxxxxx中
Host# scp keepalived.conf [email protected]:/etc/keepalived/
备节点保需要更改一下接口地址、优先级、状态为BACKUP备节点 其它跟主节点保持一致
vrrp_instance VI_1 {
state BACKUP
interface ens160
virtual_router_id 222
priority 90
两边都启动nginx跟keepalived systemctl start keepalive systemctl start nginx
启动Keepalived 查看接口地址
keepalived]# ip addr show | grep en
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
inet 192.168.9.222/24 brd 192.168.9.255 scope global ens160
inet 192.168.9.151/32 scope global ens1602.3. Inspection
此时这里可以看到主通过Http-get获取这个地址是否有效，
Host# tcpdump -i ens160 dst 192.168.9.223 and port 8080
At this point, we are looking at the nginx log and found that there is also a record per second in the query.
Here delay_loop 6 is too short, let's modify it and change it to every 60 seconds
At this time, through tcpdump, it can be captured that the master has been sending a packet to this multicast address every one second
Host# tcpdump -i ens160 dst 224.0.0.18
FAQ
This component is missing: https://bugzilla.redhat.com/show_bug.cgi?id=1477572
libipset.so.3: cannot open shared object file: No such file or directory
Solved: yum -y install libnl3-devel ipset-devel