Efficient and versatile cluster LVS load balancing cluster (DR mode)

Others 2021-03-19 16:01:14 views: null

table of Contents

1. The working principle of LVS-DR

1. Data packet flow analysis

  • The first step: the client sends a request to the Director Server (load balancer), and the requested data packet arrives in the kernel space.
    • Data message
      • Source IP ------ Client's IP
      • Target IP ------ VIP
      • Source MAC ------ client's MAC
      • Destination MAC ------ Director Server's MAC
  • Step 2: The kernel space determines that the destination IP of the data packet is the local VIP. At this time, IPVS (IP virtual server) compares whether the service requested by the data packet is a cluster service, and re-encapsulates the data packet if it is a cluster service. Then the data packet is sent to the Real Server selected according to the load balancing algorithm. (Director Server and Real Server are on the same network, and data is transmitted through the second data link layer.)
    • Data message
      • Source IP ------ client's IP
      • Target IP ------ VIP
      • Source MAC ------ Director Server's MAC
      • Destination MAC ------ MAC of Real Server
  • Step 3: The MAC address of the request message arriving at the Real Server is its own MAC address, and the message is received. The data packet re-encapsulates the message, and the response message is transmitted to the physical network card through the lo interface and then sent out.
    • Data message
      • Source IP ------ VIP
      • Destination IP ------ the IP of the client
      • Source MAC ------ MAC of Real Server
      • Destination MAC ------ client's MAC
  • Step 4: Transmit the response message to the client through the switch and router. The client receives the reply message and gets the desired service, but it does not know which server processed it.

2. Features of DR mode

  • Director Server and Real Server must be in the same physical network.
  • Real Server can use a private address or a public network address. If you use a public network address, you can directly access RIP via the Internet.
  • Director Server serves as the access portal for the cluster, but not as a gateway.
  • All request messages go through Director Server, but reply response messages cannot go through Director Server.
  • The Real Server gateway is not allowed to point to the Director Server IP, that is, the data packets sent by the Real Server are not allowed to pass through the Director Server.
  • The lo interface on the Real Server is configured with the IP address of the VIP.

Second, the ARP problem in LVS-DR

Question one:

  • In the LVS-DR load balancing cluster, both the load balancing and the node server must be configured with the same VIP address.
  • Having the same IP address in the local area network will inevitably cause the disorder of the ARP communication between the servers.
    • Solutions:
      • When the ARP broadcast is sent to the LVS-DR cluster, because the load balancer and the node server are connected to the same network, they will both receive the ARP broadcast.
      • Only the front-end load balancer responds, and other node servers should not respond to ARP broadcasts.
  • Process the node server so that it does not respond to ARP requests for VIPs.
    • Solution:
      • Use virtual interface lo:0 to carry VIP addresses
      • Set the kernel parameter arp_ignore=1: the system only responds to ARP requests whose destination IP is the local IP

Question two:

  • RealServer returns the message (the source IP is VIP) and is forwarded by the router. When re-encapsulating the message, the MAC address of the router must be obtained first.
  • When sending an ARP request, Linux uses the source IP address of the IP packet (ie VIP) as the source IP address in the ARP request packet by default, instead of using the IP address of the sending interface
    • Such as: ens33
  • After the router receives the ARP request, it will update the ARP table entry
  • The original VIP corresponding to the Director's MAC address will be updated to the VIP corresponding to the MAC address of the RealServer
  • According to the ARP table entry, the router forwards the new request message to RealServer, causing the Director's VIP to become invalid
    • Solution:
      • To process the node server, set the kernel parameter arp_announce=2: the system does not use the source address of the IP packet to set the source address of the ARP request, but selects the IP address of the sending interface.

Set up methods to solve the two problems of ARP

  • Modify the /etc/sysctl.conf file
net.ipv4.conf.lo.arp_ignore=1
net.ipv4.conf.lo.arp_announce=2
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

Three, LVS load balancing DR mode cluster deployment

Set up the environment:

Host operating system IP address Service required
DR server (load scheduler) CentOS7 ens33：192.168.163.10
ens33:0 (VIP)：192.168.163.100		 ipvsadm
Web node server 1 CentOS7 ens33 ： 192.168.163.12
lo: 0 (VIP) ： 192.168.163.100		 nfs-utils、rpcbind、httpd
Web node server 2 CentOS7 ens33：192.168.163.13
lo:0 (VIP)：192.168.163.100		 nfs-utils、rpcbind、httpd
NFS server CentOS7 192.168.163.14 rpcbind、nfs-utils
Client Windows10 192.168.163.15

Note: Set up in the same local area network this time, you do not need gateway and DNS when setting up the network, just comment. If they are not in the same network segment, you need to configure the gateway.

1. Deploy shared storage

NFS server: 192.168.163.14

systemctl stop firewalld.service
systemctl disable firewalld.service
setenforce 0

yum -y install nfs-utils rpcbind

systemctl start rpcbind.service
systemctl start nfs.service
systemctl enable nfs.service
systemctl enable rpcbind.service

mkdir /opt/test1
mkdir /opt/test2

chmod 777 /opt/test1
chmod 777 /opt/test2

vim /etc/exports
/opt/test1 192.168.163.0/24(rw,sync)
/opt/test2 192.168.163.0/24(rw,sync)

exportfs -rv

Insert picture description here

2. Configure the node server

Web node server 1: ens33: 192.168.163.12 lo:0 (VIP): 192.168.163.100
Web node server 2: ens33: 192.168.163.13 lo:0 (VIP): 192.168.163.100
Next is the same configuration of the two servers

systemctl stop firewalld.service
systemctl disable firewalld.service
setenforce 0

Insert picture description here

(1) Configure virtual IP address (VIP: 192.168.163.100)

#此地址仅用做发送 Web 响应数据包的源地址，并不需要监听客户机的访问请求（改由调度器监听并分发）。
#因此使用虚接口 lo:0 来承载 VIP 地址，并为本机添加一条路有记录，将访问 VIP 的数据限制在本地，以避免通信紊乱。

vim /etc/sysconfig/network-scripts/ifcfg-lo:0
DEVICE=lo:0
IPADDR=192.168.163.100
NETMASK=255.255.255.255
ONBOOT=yes

ifup lo:0
ifconfig lo:0

#设置临时的路由，重启失效；禁锢路由
route add -host 192.168.163.100 dev lo:0
#查看路由
route -n

#开机自动添加路由，生产环境应该用这个
vim /etc/rc.local
/sbin/route add -host 192.168.163.100 dev lo:0
chmod +x /etc/rc.d/rc.local

Insert picture description here

(2) Adjust the kernel's ARP response parameters to prevent the VIP MAC address from being updated and avoid conflicts

vim /etc/sysctl.conf
......
net.ipv4.conf.lo.arp_ignore = 1 #系统只响应目的IP为本地IP的ARP请求
net.ipv4.conf.lo.arp_announce = 2 #系统不使用IP包的源地址来设置ARP请求的源地址，而选择发送接口的IP地址
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2

sysctl -p

yum install -y nfs-utils rpcbind httpd
systemctl start rpcbind
systemctl enable rpcbind
systemctl start httpd.service
systemctl enable httpd.service

Insert picture description here

The following two server settings are a bit different.
Web node server 1: ens33: 192.168.163.12 lo:0 (VIP): 192.168.163.100

showmount -e 192.168.163.14

mount.nfs 192.168.163.14:/opt/test1 /var/www/html
echo 'this is test1 web!' > /var/www/html/index.html

Insert picture description here

Web node server 2: ens33: 192.168.163.13 lo:0 (VIP): 192.168.163.100

showmount -e 192.168.163.14

mount.nfs 192.168.163.14:/opt/test2 /var/www/html
echo 'this is test2 web!' > /var/www/html/index.html

Insert picture description here

3. Configure the load scheduler

Load scheduler: 192.168.163.10 lo:0 (VIP): 192.168.163.100
(1) Turn off the firewall and load the ip_vs module

systemctl stop firewalld.service 
systemctl disable firewalld.service 
setenforce 0

#加载ip_vs模块，并安装ipvsadm工具
modprobe ip_vs
cat /proc/net/ip_vs
yum install -y ipvsadm

Insert picture description here

(2) Configure virtual IP address (VIP: 192.168.163.100)

vim /etc/sysconfig/network-scripts/ifcfg-ens33:0
DEVICE=ens33:0
ONBOOT=yes
IPADDR=192.168.163.100
NETMASK=255.255.255.255

ifup ens33:0
ifconfig ens33:0

Insert picture description here
(3) Adjust proc response parameters

#由于 LVS 负载调度器和各节点需要共用 VIP 地址，应该关闭Linux 内核的重定向参数响应，不充当路由器，
vim /etc/sysctl.conf
net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0

sysctl -p

Insert picture description here

(4) Configure load distribution strategy

ipvsadm-save > /etc/sysconfig/ipvsadm
或者
ipvsadm --save > /etc/sysconfig/ipvsadm

systemctl start ipvsadm.service

#清除原有策略
ipvsadm -C
ipvsadm -A -t 192.168.163.100:80 -s rr
ipvsadm -a -t 192.168.163.100:80 -r 192.168.163.12:80 -g #如果这里是隧道模式，直接将-g替换成-i即可
ipvsadm -a -t 192.168.163.100:80 -r 192.168.163.13:80 -g

#查看节点状态，Route代表 DR模式
ipvsadm -ln

Insert picture description here

4. Test verification

Visit http://192.168.163.100/ on the client, refresh to test whether the load balance is successful
Insert picture description here

Insert picture description here

Guess you like

Origin blog.csdn.net/weixin_51326240/article/details/113148641
Recommended
LFOSSA Yuanlaisusu Open Course | Mastering the Cloud Native Future: Comprehensive Guide to CNCF Certification and Exam Preparation Tips
Ranking
C++ Basic Syntax
bootstrapTable hides a column based on a condition
Why is reentrant lock recommended instead of Synchronized when dynamic high concurrency?
hexo create a blog
[Fully open source and non-encrypted version] Imitation of the eighth district distribution/online signature/multiple sets of download templates/APP distribution hosting/APP packaging and packaging
Polymerization combination
https://www.flysnow.org/2017/05/06/go-in-action-go-log.html
From the perspective of Flutter and the front-end, talk about how to ensure UI fluency under the single-threaded model
nginx-301, 302 redirect
Geolocation by IP Address in ASP.NET
Daily
More
2024-04-26(22)
2024-04-25(32)
2024-04-24(30)
2024-04-23(30)
2024-04-22(5)
2024-04-21(0)
2024-04-20(6)
2024-04-19(5)
2024-04-18(0)
2024-04-17(31)

Code World

© 2018 codetd.com