Build LVS-DR load balancing cluster and Keepalived-LVS high-availability load balancing cluster

Others 2022-04-22 12:46:41 views: 0

18.11 LVS DR Mode Construction

Ready to work

Three machines, all three machines have public IP.

  • Scheduler (director)
    IP: 192.168.230.135
  • real server1(real1)
    IP：192.168.230.130
  • real server2(real2)
    IP：192.168.230.145
  • VIP：192.168.230.200

start building

configure director 

[root@cham002 ~]# vim /usr/local/sbin/lvs_dr.sh

#! /bin/bash
echo 1 > /proc/sys/net/ipv4/ip_forward
ipv=/usr/sbin/ipvsadm
vip=192.168.230.200
rs1=192.168.230.130
rs2=192.168.230.145
#注意这里的网卡名字
ifdown ens33
ifup ens33
ifconfig ens33:2 $vip broadcast $vip netmask 255.255.255.255 up
route add -host $vip dev ens33:2
$ipv -C
$ipv -A -t $vip:80 -s rr
$ipv -a -t $vip:80 -r $rs1:80 -g -w 1
$ipv -a -t $vip:80 -r $rs2:80 -g -w 1

执行脚本：
[root@cham002 ~]# sh /usr/local/sbin/lvs_dr.sh 
成功断开设备 'ens33'。
成功激活的连接（D-Bus 激活路径：/org/freedesktop/NetworkManager/ActiveConnection/6）
[root@cham002 ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:b6:9f:e3 brd ff:ff:ff:ff:ff:ff
    inet 192.168.230.135/24 brd 192.168.230.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.230.200/32 brd 192.168.230.200 scope global ens33:2
       valid_lft forever preferred_lft forever
    inet 192.168.230.150/24 brd 192.168.230.255 scope global secondary ens33:0
       valid_lft forever preferred_lft forever
    inet6 fe80::6f15:52d3:ebeb:e193/64 scope link 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:b6:9f:ed brd ff:ff:ff:ff:ff:ff
    inet 192.168.118.147/24 brd 192.168.118.255 scope global ens37
       valid_lft forever preferred_lft forever
    inet 192.168.100.1/24 brd 192.168.100.255 scope global ens37
       valid_lft forever preferred_lft forever
    inet6 fe80::1801:cbbb:ebcc:89a3/64 scope link 
       valid_lft forever preferred_lft forever

Note:  VIP is bound to the ens33 network card.

 Script Explanation

#! /bin/bash
echo 1 > /proc/sys/net/ipv4/ip_forward
#开启端口转发
ipv=/usr/sbin/ipvsadm
vip=192.168.230.200
rs1=192.168.230.130
rs2=192.168.230.145
#注意这里的网卡名字
ifdown ens33
ifup ens33
#在此重启网卡的目的是避免重复设置命令行提供的IP
ifconfig ens33:2 $vip broadcast $vip netmask 255.255.255.255 up
#绑定VIP到dir的虚拟网卡ens33:2
route add -host $vip dev ens33:2
#添加网关
$ipv -C
$ipv -A -t $vip:80 -s wrr
$ipv -a -t $vip:80 -r $rs1:80 -g -w 1
$ipv -a -t $vip:80 -r $rs2:80 -g -w 1
#设置ipvsadm规则，-g=gateway：使用默认网关（DR模式）

configure real server

Configure the following scripts in real1 and real2 respectively:

##real1（130）
[root@cham1 ~]# vi /usr/local/sbin/lvs_rs.sh

#/bin/bash
vip=192.168.230.200
#把vip绑定在lo上，是为了实现rs直接把结果返回给客户端
ifdown lo
ifup lo
ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up
route add -host $vip lo:0
#以下操作为更改arp内核参数，目的是为了让rs顺利发送mac地址给客户端
#参考文档www.cnblogs.com/lgfeng/archive/2012/10/16/2726308.html
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

执行脚本：
[root@cham1 ~]# vi /usr/local/sbin/lvs_rs.sh
[root@cham1 ~]# sh !$
sh /usr/local/sbin/lvs_rs.sh

[root@cham1 ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.230.2   0.0.0.0         UG    100    0        0 ens33
192.168.230.0   0.0.0.0         255.255.255.0   U     100    0        0 ens33
192.168.230.200 0.0.0.0         255.255.255.255 UH    0      0        0 lo

[root@cham1 ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 192.168.230.200/32 brd 192.168.230.200 scope global lo:0
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:90:35:d9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.230.130/24 brd 192.168.230.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::7fe3:4489:d9af:a1ed/64 scope link 
       valid_lft forever preferred_lft forever
    inet6 fe80::6f15:52d3:ebeb:e193/64 scope link tentative dadfailed 
       valid_lft forever preferred_lft forever


##rs2(145)
[root@test ~]# vim /usr/local/sbin/lvs_rs.sh
[root@test ~]# sh !$
sh /usr/local/sbin/lvs_rs.sh

[root@test ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.230.2   0.0.0.0         UG    100    0        0 ens33
192.168.230.0   0.0.0.0         255.255.255.0   U     100    0        0 ens33
192.168.230.200 0.0.0.0         255.255.255.255 UH    0      0        0 lo

[root@test ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 192.168.230.200/32 brd 192.168.230.200 scope global lo:0
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:50:f9:44 brd ff:ff:ff:ff:ff:ff
    inet 192.168.230.145/24 brd 192.168.230.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::9b07:b28d:f5e9:d107/64 scope link 
       valid_lft forever preferred_lft forever

135 

[root@cham002 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.230.200:80 rr
  -> 192.168.230.130:80           Route   1      0          1         
  -> 192.168.230.145:80           Route   1      0          1         
[root@cham002 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.230.200:80 rr
  -> 192.168.230.130:80           Route   1      2          1         
  -> 192.168.230.145:80           Route   1      1          1         
[root@cham002 ~]#

test

Visit VIP: 192.168.230.200 in the browser, refresh the webpage, and the visit result will be alternately replied by real1 and real2.

Open another virtual machine on the same network segment to test!

Type `help' to learn how to use Xshell prompt.
[d:\~]$ ssh 192.168.230.140


Connecting to 192.168.230.140:22...
Connection established.
To escape to local shell, press 'Ctrl+Alt+]'.

Last login: Fri Nov 17 15:14:29 2017 from 192.168.230.1
[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.
[root@cham3 ~]# curl 192.168.230.200
cham  130

[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.
[root@cham3 ~]# curl 192.168.230.200
cham  130

[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.
[root@cham3 ~]# curl 192.168.230.200
cham  130

[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.
[root@cham3 ~]# curl 192.168.230.200
cham  130

 

18.12 Keepalived LVS

The complete architecture requires two servers (the role is dir), and the Keepalived tool is installed separately to achieve high availability, but Keepalived itself also has a load balancing function, so only one Keepalived can be installed for this use. Keepalived has built-in functions of ipvsadm, so there is no need to install the ipvsadm package, nor to write and execute the lvs_dr script.

Ready to work

Three machines:

  • Scheduler director:
    IP: 192.168.230.135; Install Keepalived
  • real server（real1）：
    IP：192.168.230.130
  • real server（real2）：
    IP：192.168.230.145
  • VIP：192.168.230.200

• On two rs, the /usr/local/sbin/lvs_rs.sh script still needs to be executed

• keepalived has a better function, which can no longer forward requests when an rs goes down

• test

Configure director (because we have installed it when we made keepalived high availability)

[root@cham002 ~]# vim /etc/keepalived/keepalived.conf 

vrrp_instance VI_1 {
    #备用服务器上为 BACKUP
    state MASTER
    #绑定vip的网卡为ens33，你的网卡和阿铭的可能不一样，这里需要你改一下
    interface ens33
    virtual_router_id 51
    #备用服务器上为90
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass aminglinux
    }
    virtual_ipaddress {
        192.168.230.200
    }
}
virtual_server 192.168.230.200 80 {
    #(每隔10秒查询realserver状态)
    delay_loop 10
    #(lvs 算法)
    lb_algo wlc
    #(DR模式)
    lb_kind DR
    #(同一IP的连接60秒内被分配到同一台realserver)
    persistence_timeout 0
    #(用TCP协议检查realserver状态)
    protocol TCP
    real_server 192.168.230.130 80 {
        #(权重)
        weight 100
        TCP_CHECK {
        #(10秒无响应超时)
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }
    real_server 192.168.230.145 80 {
        weight 100
        TCP_CHECK {
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
     }
}

执行ipvsadm -C  把之前的ipvsadm规则清空掉
[root@cham002 ~]# ipvsadm -C

重启网络可以把之前的vip关掉
[root@cham002 ~]# systemctl restart network


启动Keepalived服务：
[root@cham002 ~]# systemctl start keepalived
[root@cham002 ~]# ps aux |grep keep
root     11296  0.0  0.1 120720  1404 ?        Ss   1月24   0:00 /usr/sbin/keepalived -D
root     11297  0.0  0.3 127460  3272 ?        S    1月24   0:00 /usr/sbin/keepalived -D
root     11298  0.0  0.3 131656  3032 ?        S    1月24   0:05 /usr/sbin/keepalived -D
root     47689  0.0  0.0 112680   976 pts/1    S+   00:31   0:00 grep --color=auto keep

查看网卡信息：
[root@cham002 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:b6:9f:e3 brd ff:ff:ff:ff:ff:ff
    inet 192.168.230.135/24 brd 192.168.230.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.230.200/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.230.150/24 brd 192.168.230.255 scope global secondary ens33:0
       valid_lft forever preferred_lft forever
    inet6 fe80::6f15:52d3:ebeb:e193/64 scope link 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:b6:9f:ed brd ff:ff:ff:ff:ff:ff
    inet 192.168.118.147/24 brd 192.168.118.255 scope global ens37
       valid_lft forever preferred_lft forever
    inet 192.168.100.1/24 brd 192.168.100.255 scope global ens37
       valid_lft forever preferred_lft forever
    inet6 fe80::1801:cbbb:ebcc:89a3/64 scope link 

[root@cham002 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.230.200:80 wlc
  -> 192.168.230.130:80           Route   100    0          0         
  -> 192.168.230.145:80           Route   100    0          0

After adding Keepalived, the down real server will be automatically cleared from the rs list.

如有一方的ngixn 宕掉了，能自动检测，开启后自动加载回来
[root@cham002 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.230.200:80 wlc
  -> 192.168.230.145:80           Route   100    0          0         

[root@cham002 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.230.200:80 wlc
  -> 192.168.230.130:80           Route   100    0          0         
  -> 192.168.230.145:80           Route   100    0          0

 

test

To escape to local shell, press 'Ctrl+Alt+]'.

Last login: Tue Jan 30 23:15:25 2018 from 192.168.230.1
130宕机
[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.
[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.
[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.
[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.
[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.
[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.
[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.
[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.
[root@cham3 ~]# curl 192.168.230.200
cham  145 backup backup.

145宕机
[root@cham3 ~]# curl 192.168.230.200
cham  130

[root@cham3 ~]# curl 192.168.230.200
cham  130

[root@cham3 ~]# curl 192.168.230.200
cham  130

[root@cham3 ~]# curl 192.168.230.200
cham  130

 

Keepalived+LVS function

  • Keepalived builds high availability to ensure that the server is not paralyzed after the director in LVS goes down
  • If only LVS is used, when a real server in the LVS architecture goes down, the director will continue to send requests to it.

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=325035667&siteId=291194637
Recommended
LFOSSA Yuanlaisusu Open Course | Mastering the Cloud Native Future: Comprehensive Guide to CNCF Certification and Exam Preparation Tips
Ranking
C++ Basic Syntax
bootstrapTable hides a column based on a condition
Why is reentrant lock recommended instead of Synchronized when dynamic high concurrency?
hexo create a blog
[Fully open source and non-encrypted version] Imitation of the eighth district distribution/online signature/multiple sets of download templates/APP distribution hosting/APP packaging and packaging
Polymerization combination
https://www.flysnow.org/2017/05/06/go-in-action-go-log.html
From the perspective of Flutter and the front-end, talk about how to ensure UI fluency under the single-threaded model
nginx-301, 302 redirect
Geolocation by IP Address in ASP.NET
Daily
More
2024-04-26(22)
2024-04-25(32)
2024-04-24(30)
2024-04-23(30)
2024-04-22(5)
2024-04-21(0)
2024-04-20(6)
2024-04-19(5)
2024-04-18(0)
2024-04-17(31)

Code World

© 2018 codetd.com