LVS+Keepalived high-availability cluster deployment

Keepalived overview

Keepalived is a software similar to the layer3, 4 & 5 exchange mechanism, which is what we usually call the layer 3, layer 4 and layer 5 switching. Keepalived is done automatically, without manual intervention.

The function of Keepalived is to detect the status of the server. If a web server is down or the work fails, Keepalived will detect it and remove the faulty server from the system. At the same time, it will use other servers to replace the work of the server. After it works normally, Keepalived automatically adds the server to the server group. All these tasks are automatically completed without manual intervention. All that is required is to repair the faulty server.

• A health check tool specially designed for LVS and HA
  • Support automatic failover (Failover)
  • Support node health check (Health Checking)
  • Official website: http://www.keepalived.orgl
    

Keepalived implementation principle

• Keepalived adopts VRRP hot backup protocol to realize the multi-machine hot backup function of Linux server
• VRRP (Explicit Virtual Routing Redundancy Protocol) is a backup solution for routers
  • Multiple routers form a hot backup group, and provide services to the outside through a shared virtual IP address
  • There is only one main router in each hot standby group to provide services at the same time, and the other routers are in a redundant state
  • If the currently online router fails, other routers will automatically take over the virtual IP address according to the set priority and continue to provide services

Keepalived case

• Keepalived can realize multi-machine hot backup, and each hot backup group can have multiple servers
• The failover of dual-system hot backup is realized by the drift of virtual IP address, which is suitable for various application servers
• Realize dual machine hot backup based on Web service
  • Drift address: 192.168.10.72
  • Primary and standby servers: 192.168.10.73, 192.168.10.74
  • Application Service Provided: Web
    

Keepalived installation and configuration

When applied in an LVS cluster environment, you also need to use the ipvsadm management tool
YUM to install Keepalived to
enable Keepalived service

keepalived.conf是主配置文件
global_defs {...}  #区段指定全局参数
vrrp_instance 实例名称{...}  #区段指定VRRP热备参数
注释文字以"!”符号开头
目录samples，提供了许多配置样例作为参考

常用配置选项
router id HA_TEST_R1	#本路由器(服务器)的名称
vrrp_instance Vl_1	#定义VRRP热备实例
state MASTER	#热备状态,MASTER表示主服务器
interface ens33	#承载VIP地址的物理接口
virtual_router_id 1	#虚拟路由器的ID号,每个热备组保持—致（组号）
priority 100:优先级 #数值越大优先级越高
advert_int 1 #通告间隔秒数(心跳频率)
auth_type PASS  #认证类型
auth_pass 123456  #密码字串
virtual_ipaddress { vip}  #指定漂移地址(VIP),可以有多个

• There are three options for the configuration of the Keepalived backup server and the master configuration.
  • router_id: set as your own name
  • state: set to BACKUP
  • priority: value is lower than the main server
• Other options are the same as master

LCS-DR installation and deployment

Experimental environment
LVS1: 192.168.110.10 LVS2: 192.168.110.15
Web1: 192.168.110.20
Web2
: 192.168.110.25
Virtual IP (Vip): 192.168.110.5

[root@lvs1 ~]# yum -y install ipvsadm keepalived
LVS2也是（以下所有配置LVS2同步配置）
LVS更改配置文件，关闭路由转发和重定向
[root@lvs1 ~]# vim /etc/sysctl.conf
#末行插入如下配置
#开启ipv4地址转发
net.ipv4.ip_forward = 1
#关闭ipv4全部重定向
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0
#重载配置，使配置生效
[root@lvs1 ~]# sysctl -p
net.ipv4.ip_forward = 1
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0

LVS都创建个服务脚本，放在 init.d 中，方便service进行使用
[root@lvs1 ~]# vim /etc/init.d/DR.sh
#!/bin/bash
GW=192.168.110.2	
VIP=192.168.110.5   #虚拟ip
RIP1=192.168.110.20 #真实web服务器ip
RIP2=192.168.110.25
case "$1" in
start)
                /sbin/ipvsadm --save > /etc/sysconfig/ipvsadm  #保存配置
                systemctl start ipvsadm  #启动服务
                /sbin/ifconfig ens33:0 $VIP broadcast $VIP netmask 255.255.255.255 broadcast $VIP up  
#设置ens33:0地址，广播地址，子网掩码，并开启
                /sbin/route add -host $VIP dev ens33:0  #添加路由网段信息
                /sbin/ipvsadm -A -t $VIP:80 -s rr #指定虚拟服务访问入口，指定轮询算法
                /sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -g #指定真实服务器，dr模式
                /sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -g
                echo "ipvsadm starting --------------------[ok]"
                ;;
stop)
                /sbin/ipvsadm -C   #清空缓存
                systemctl stop ipvsadm  #关闭服务
                ifconfig ens33:0 down  #关闭接口
                route del $VIP   #删除路由信息
                echo "ipvsamd stoped----------------------[ok]"
                 ;;
status)
                if [ ! -e /var/lock/subsys/ipvsadm ];then  #判断文件存在与否决定状态
                echo "ipvsadm stoped---------------"
                exit 1
                                else
                                echo "ipvsamd Runing ---------[ok]"
                fi
                ;;
*)
                echo "Usage: $0 {start|stop|status}"
                exit 1
esac
exit 0
[root@lvs1 ~]# chmod +x /etc/init.d/DR.sh 

Virtual machine settings, put two LVSs in LAN segment 1

shell 中的命令是配置临时 IP 地址，所以在配置文件中再更改一次
[root@lvs1 /]# cd /etc/sysconfig/network-scripts/
[root@lvs1 network-scripts]# cp -p ifcfg-ens33 ifcfg-ens33:0
[root@lvs1 network-scripts]# vim ifcfg-ens33:0
DEVICE=ens33:0
ONBOOT=yes
IPADDR=192.168.110.5
NETMASK=255.255.255.0
[root@lvs01 network-scripts]# vim ifcfg-ens33
BOOTPROTO=static
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.110.20  ##LVS2改成192.168.10.25
PREFIX=255.255.255.0
GATEWAY=192.168.110.2

重启服务并启动虚拟网卡还有shell脚本
[root@lvs1 network-scripts]# systemctl restart network
[root@lvs1 network-scripts]# ifup ens33:0
[root@lvs1 network-scripts]# service DR.sh start
ipvsadm starting --------------------[ok]
[root@lvs1 network-scripts]# systemctl stop firewalld
[root@lvs1 network-scripts]# setenforce 0

Configure WEB server

YUM installs HTTPD
web1, 2 in the network mode of LAN segment 1

更改IP，配置lo：0
[root@web1 network-scripts]# cp -p ifcfg-lo ifcfg-lo:0
[root@web1 network-scripts]# vim ifcfg-lo:0
DEVICE=lo:0
IPADDR=192.168.110.5
NETMASK=255.255.255.0
ONBOOT=yes

[root@web1 network-scripts]# vim ifcfg-ens33 
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.10.20  #web2配置成 192.168.10.25
PREFIX=255.255.255.0
GATEWAY=192.168.110.2
[root@web1 network-scripts]# systemctl restart network

配置arp抑制脚本
[root@web1 network-scripts]# vim /etc/init.d/apa.sh
VIP=192.168.110.5
             case "$1" in
             start)
                ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
                /sbin/route add -host $VIP dev lo:0
                echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore   '//arp忽略'
                echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
                echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
                echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
                sysctl -p >/dev/null 2>&1
                echo "RealServer Start OK "
             ;;
             stop)
                ifconfig lo:0 down
                route del $VIP /dev/null 2>&1
                echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore '//arp开启'
                echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
                echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
                echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
                echo "RealServer Stopd"
             ;;
             *)
                echo "Usage: $0 {start|stop}"
                exit 1
       esac
       exit 0
[root@web1 network-scripts]# chmod +x /etc/init.d/apa.sh 

web 服务配置主页面
[root@web1 html]# echo "<h1>This is 1 Server.</h1>" > index.html
[root@web1 html]# ls
index.html
[root@web2 html]# echo "<h1>This is 2 Server.</h1>" > index.html
[root@web2 html]# ls
index.html

打开、重启各服务
[root@web1 html]# ifup lo:0
[root@web1 html]# service apa.sh start
RealServer Start OK 
[root@web1 html]# systemctl stop firewalld
[root@web1 html]# setenforce 0
[root@web1 html]# systemctl start httpd

Verification experiment

Keepalievd deployment

The experimental environment is based on the above DR deployment to
configure two LVS

[root@lvs1 network-scripts]# vim /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
     [email protected]
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 127.0.0.1  #指向自身环回口IP
   smtp_connect_timeout 30
   router_id LVS1  #两台LVS的id 不能相同，另一台配 LVS2
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}  

vrrp_instance VI_1 {
    state MASTER
    interface ens33  #根据自己的网卡设置
    virtual_router_id 10  #两个虚拟号需要相同
    priority 100  #优先级，越大越优先，所以 02 的优先级可以配90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111   #上面一行和这一行最好不要改，如果改了也要相同
    }
    virtual_ipaddress {
        192.168.110.5
    }   
    }       
        
virtual_server 192.168.110.5 80 {  ##改成浮动VIP  端口80 指向http服务
    delay_loop 6
    lb_algo rr  #轮询
    lb_kind DR  #DR模式
    persistence_timeout 50
    protocol TCP
        real_server 192.168.110.20 80 {  #指向 web1 端口80
        weight 1  #向下删除大概9行
            TCP_CHECK{  #添加如下
            connect_port 80  #添加连接端口
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }   
    }
#删了下面的全部，然后复制上面的 real_server 段
real_server 192.168.110.25 80 {
        weight 1
            TCP_CHECK{
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }   
    }  
}

web2 的配置文件
! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
     [email protected]
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS2
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}  

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 10
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.10.10
    }   
    }       
        
virtual_server 192.168.110.5 80 {
    delay_loop 6
    lb_algo rr 
    lb_kind DR 
    persistence_timeout 50
    protocol TCP
        real_server 192.168.110.20 80 {
        weight 1 
            TCP_CHECK{  添加如下
            connect_port 80 
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }   
    }
real_server 192.168.110.25 80 {
        weight 1
            TCP_CHECK{
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }   
    }  
}

Verification experiment

开启服务
[root@lvs1 network-scripts]# systemctl start keepalived
[root@lvs1 network-scripts]# systemctl restart keepalived
[root@lvs1 network-scripts]# systemctl restart network

visit website

关闭 LVS01
[root@lvs1 network-scripts]# systemctl stop network

Visit again.

Server scheduling is successful and the experiment is complete