surroundings:
- 202.106.0.17：as Client
- 202.106.0.27: as router
- 192.168.205.37: as lvs1
- 192.168.205.47: as lvs2
- 192.168.205.57: as websrv1
- 192.168.205.67: as websrv2
- 192.168.205.77: as websrv3
- 192.168.205.87: as websrv4
Note: All operating systems disabled by default firewalld, iptable empty, closed selinux
version:
- OS: centos 7 1810 with mini install
- keepalived
- httpd
Joint:
Use four keepalived monitoring web server, a set of two, each using a virtual IP address, the vip2 LVS1 is a front VIPl from LVS2 to the main vip2, VIPl from standby each other, and using as a load lvs balanced, and can automatically discover services down the machine and automatically remove the failed host, enabling trouble-free scheduling.
Configuring router
- Turn routing forwarding
[root@router data]#ech 1 > /proc/sys/net/ipv4/ip_forward - Add a 10.1.1.1 on eth1 router, because I go to when lvs1 and lvs2 server must have a route, because it is virtual Ip two hosts are likely to drift, so when one of them down can not determine in which stage on the host, in order to secure the route without the direct use of IP direct connect.
[root@router data]#ip a a 10.1.1.1/24 dev eth0:1Configure four real server
-
Run the following script on both servers app1 192.168.205.57/67
[root@websrv1 data]#cat lvs_dr_rs.sh #!/bin/bash vip=10.1.1.100 gateway=192.168.205.27 mask='24' dev=lo:1 rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null service httpd start &> /dev/null && echo "The httpd Server is Ready!" echo "this is `hostname`" > /var/www/html/index.html case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ip a a $vip/$mask dev $dev #broadcast $vip up ip route add default via $gateway dev eth0 #route add -host $vip dev $dev echo "The RS Server is Ready!" ;; stop) systemctl restart network echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce echo "The RS Server is Canceled!" ;; *) echo "Usage: $(basename $0) start|stop" exit 1 ;; esac -
App2 run the following script on both servers 192.168.205.77/87
[root@websrv3 data]#cat lvs_dr_rs.sh #!/bin/bash vip=10.1.1.200 gateway=192.168.205.27 mask='24' dev=lo:1 rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null service httpd start &> /dev/null && echo "The httpd Server is Ready!" echo "this is `hostname`" > /var/www/html/index.html case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ip address add $vip/$mask dev $dev #broadcast $vip up ip route add default via $gateway dev eth0 #route add -host $vip dev $dev echo "The RS Server is Ready!" ;; stop) systemctl restart network echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce echo "The RS Server is Canceled!" ;; *) echo "Usage: $(basename $0) start|stop" exit 1 ;; esacDo the same on the two movable LVS (unless otherwise stated)
- Install httpd service, the sorry server on the local host of the two
yum install httpd echo server is under maitenance > /var/www/html/index.html - Be sure to add default route to the router, if applied correctly can work, but not normal sorry server
ip route add default via 192.168.205.27 dev eth0 - For easy access to it is preferable to do lvs ssh key authentication between two servers, use the following method would not do it again at 47, and then really achieve mutual authentication
ssh-keygen scp -r /root/.ssh 192.168.205.47:/root - Best to join the two hosts file host name resolution
vi /etc/hosts 192.168.205.37 websrv1 192.168.205.47 websrv2 scp /etc/hosts 192.168.205.47:/etc - Installation keepalive, in order to see how to add the lvs strategy, we have also installed iplvadm
yum install keepalived ipvsadm - Modify the configuration files on lvs1
[root@lsv1 ~]#vi /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalive@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS1 vrrp_mcast_group4 224.0.0.100 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 37 priority 100 advert_int 1 authentication { auth_type PASS auth_pass centos } virtual_ipaddress { 10.1.1.100/24 dev eth0 label eth0:0 } } vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 47 priority 80 advert_int 1 authentication { auth_type PASS auth_pass centos } virtual_ipaddress { 10.1.1.200/24 dev eth0 label eth0:1 } } virtual_server 10.1.1.100 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.205.57 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.205.67 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } virtual_server 10.1.1.200 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.205.77 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.205.87 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } 11. 为了方便将lsv1的keepalive.conf复制到lvs2上，并进行修改 [root@lvs2 ~]#vi /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalive@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS2 vrrp_mcast_group4 224.0.0.100 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 37 priority 80 advert_int 1 authentication { auth_type PASS auth_pass centos } virtual_ipaddress { 10.1.1.100/24 dev eth0 label eth0:0 } } vrrp_instance VI_2 { state MASTER interface eth0 virtual_router_id 47 priority 100 advert_int 1 authentication { auth_type PASS auth_pass centos } virtual_ipaddress { 10.1.1.200/24 dev eth0 label eth0:1 } } virtual_server 10.1.1.100 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.205.57 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.205.67 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } virtual_server 10.1.1.200 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.205.77 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.205.87 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } - Starting keepalived Service
systemctl start keepalived - In lvs1 see the status of the RR scheduling
[root@lsv1 ~]#ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.1.100:80 rr -> 192.168.205.57:80 Route 1 0 0 -> 192.168.205.67:80 Route 1 0 0 TCP 10.1.1.200:80 rr -> 192.168.205.77:80 Route 1 0 0 -> 192.168.205.87:80 Route 1 0 0 - In lvs1 see only a 10.1.1.100IP, in Lvs2 will see only 10.1.1.200IP
[root@lsv1 ~]#ip a 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:56:e1:ea brd ff:ff:ff:ff:ff:ff inet 192.168.205.37/24 brd 192.168.205.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever inet 10.1.1.100/24 scope global eth0:0 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe56:e1ea/64 scope link noprefixroute valid_lft forever preferred_lft forever [root@lvs2 ~]#ip a 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:37:f9:93 brd ff:ff:ff:ff:ff:ff inet 192.168.205.47/24 brd 192.168.205.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever inet 10.1.1.200/24 scope global eth0:1 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe37:f993/64 scope link noprefixroute valid_lft forever preferred_lft forevertest
- Run a cycle test on the client
[root@client ~]#while : ;do curl 10.1.1.100; sleep 0.5; done this is websrv1 this is websrv2 this is websrv1 this is websrv2 this is websrv1 this is websrv2 [root@client ~]#while : ;do curl 10.1.1.200; sleep 0.5; done this is websrv4 this is websrv3 this is websrv4 this is websrv3 this is websrv4 - Stopped web1, then tested and found to only dispatch web2
[root@websrv1 data]#systemctl stop httpd [root@client ~]#while : ;do curl 10.1.1.100; sleep 0.5; done this is websrv2 this is websrv2 this is websrv2 this is websrv2 this is websrv2 - Stopped web2, and then tested for work with a sorry server, and can be seen on lvs1 127.0.0.1 load
[root@websrv2 ~]#systemctl stop httpd [root@client ~]#while : ;do curl 10.1.1.100; sleep 0.5; done server is under maitenance server is under maitenance server is under maitenance server is under maitenance [root@lsv1 ~]#ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.1.100:80 rr -> 127.0.0.1:80 Route 1 0 4 TCP 10.1.1.200:80 rr -> 192.168.205.77:80 Route 1 0 0 -> 192.168.205.87:80 Route 1 0 0 - Restore two websrv1 and websrv2, and stopped lvs2, we found no effect, but you can see two vip all back on lvs1
[root@websrv1 ~]#systemctl start httpd [root@websrv2 ~]#systemctl start httpd [root@lvs2 ~]#systemctl stop keepalived [root@client ~]#while : ;do curl 10.1.1.100; sleep 0.5; done this is websrv2 this is websrv1 this is websrv2 this is websrv1 [root@client ~]#while : ;do curl 10.1.1.200; sleep 0.5; done this is websrv3 this is websrv4 this is websrv3 this is websrv4 [root@lsv1 ~]#ip a 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:56:e1:ea brd ff:ff:ff:ff:ff:ff inet 192.168.205.37/24 brd 192.168.205.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever inet 10.1.1.100/24 scope global eth0:0 valid_lft forever preferred_lft forever inet 10.1.1.200/24 scope global secondary eth0:1 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe56:e1ea/64 scope link noprefixroute valid_lft forever preferred_lft forever - The lvs2 restore the state, because there is preemption back to the original primary and backup
[root@client ~]#while : ;do curl 10.1.1.100; sleep 0.5; done this is websrv2 this is websrv1 this is websrv2 [root@client ~]#while : ;do curl 10.1.1.200; sleep 0.5; done this is websrv4 this is websrv3 this is websrv4 [root@lsv1 ~]#ip a 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:56:e1:ea brd ff:ff:ff:ff:ff:ff inet 192.168.205.37/24 brd 192.168.205.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever inet 10.1.1.100/24 scope global eth0:0 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe56:e1ea/64 scope link noprefixroute valid_lft forever preferred_lft forever [root@lvs2 ~]#ip a 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:37:f9:93 brd ff:ff:ff:ff:ff:ff inet 192.168.205.47/24 brd 192.168.205.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever inet 10.1.1.200/24 scope global eth0:1 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe37:f993/64 scope link noprefixroute valid_lft forever preferred_lft forever