LVS load balancing cluster deployment-detailed steps

1. The meaning of cluster:

• Cluster, cluster, cluster
• Consists of multiple hosts, but only externally appears as a whole

2. Problems in modern enterprise networks:

• In Internet applications, as the site’s hardware performance, response speed, and service stability
• The requirements for qualitative and data reliability are getting higher and higher, and a single server is not enough

3. Solution:

• Use expensive minicomputers and mainframes
• Use ordinary servers to build service clusters

4. According to the target difference of the cluster, it can be divided into three types:

• Load balancing cluster
• Highly available cluster
• High-performance computing cluster

1. Load Balance Cluster

• Improve the responsiveness of the application system, handle as many access requests as possible,
• Reduce latency as the goal, obtain high concurrency, high load (LB) overall performance
• The load distribution of LB depends on the distribution algorithm of the master node

  2. High Availability Cluster

• Improve the reliability of the application system, reduce the interruption time as much as possible, ensure the continuity of the service, and achieve the fault tolerance effect of high availability (HA)
• The working mode of HA includes duplex and master-slave modes

  3. High Performance Computer Cluster (High Performance Computer Cluster)

• The goal is to increase the CPU computing speed of the application system, expand hardware resources and analysis capabilities, and obtain high-performance computing (HPC) capabilities equivalent to large and supercomputers
• High performance relies on "distributed computing" and "parallel computing". Through dedicated hardware and software, the CPU, memory and other resources of multiple servers are integrated to achieve the computing power that only large and supercomputers have.

Five, load balancing structure

• The first layer, load scheduler (Load Balancer or Director)
• The second layer, the server pool (Server Pool)
• The third layer, shared storage (Share Storage)

Six, load balancing network architecture, there are three common

1. NAT mode:

• Network Address Translation, referred to as NAT mode
• Similar to the private network structure of the firewall, the load scheduler acts as the gateway of all server nodes, that is, as the access entrance of the client, and also the access exit of each node in response to the client
• The server node uses a private IP address and is located on the same physical network as the load scheduler, and the security is better than the other two methods

2. TUN mode:

• IP tunnel lPTunnel, TUN mode for short
• Adopting an open network structure, the load scheduler only serves as the client's access portal, and each node directly responds to the client through its own Internet connection, instead of passing through the load scheduler
• The server nodes are scattered at different locations in the Internet, have independent public IP addresses, and communicate with the load scheduler through a dedicated IP tunnel

3.DR mode

• Direct Routing, referred to as DR mode
• It adopts a semi-open network structure, which is similar to the structure of the TUN model, but the nodes are not scattered everywhere, but are located on the same physical network as the scheduler
• The load scheduler is connected to each node server through the local network, without the need to establish a dedicated IP tunnel

Seven, about LVS virtual server

• Linux Virtual Server
• Load balancing solution for Linux kernel
• Founded by Dr. Zhang Wensong in my country in May 1998
• Official website: http://www.linuxvirtualserver.org/

8. LVS load scheduling algorithm (4 types)

1. Round Robin

• The received access requests are allocated to each node (real server) in the cluster in turn in order, and each server is treated equally, regardless of the actual number of connections and system load of the server

2.Weighted Round Robin

• Distribute requests according to the weight value set by the scheduler. The node with the higher weight value will get the task first, and the more requests are allocated
• Ensure that the server with strong performance bears more access traffic

3. Least Connections

• Assign according to the number of connections established by the real server, and prioritize the received access requests to the node with the least number of connections

4.Weighted Least Connections

• When the performance difference of server nodes is large, the weights can be automatically adjusted for real servers. Nodes with higher performance will bear a larger proportion of active connection load

Nine, LVS cluster creation and management methods

1. Create a virtual server
2. Add and delete server nodes
3. View cluster and node status
4. Save assigned strategy

10. Experimental design

1. Load the ip_vs module, install the ipvsadm tool
2. Enable routing and forwarding
3. Create a new LVS virtual server and add a node server
4. Build NFS shared storage service
   4.1 Introduction to nfs
   ① Network File System, which relies on RPC (Remote Procedure Call)
   ② Need to install nfs-utils, rpcbind software package
   ③ System service: nfs, rpcbind
   ④ Shared configuration file: /etc/ exports
   4.2 Use NFS to publish shared resources
   ① Install the nfs-utils, rpcbind software package
   ② Set the shared directory
   ③ Start the NFS service program
   ④ View the NFS shared directory published by the machine
   4.3. Access NFS shared resources in the client
   ① Install the rpcbind software package, And start the rpcbind service
   ② Manually mount the NFS shared directory
   ③ Fstab automatically mount settings
5. Configure node server
   5.1 Establish a test website
   5.2 Mount NFS shared storage
   5.3 Establish a test web page
6. Save the rules and test

Experimental topology

1. Build a scheduler

[root@localhost network-scripts]# nmcli connection  //查看UUID号，绑定网卡
NAME               UUID                                TYPE      DEVICE 
ens33           	    e700afd3-d9d3-4e75-ba0f-1c472d256ce8   ethernet   ens33  
virbr0            	ec8ec847-c505-4e0a-ac6c-a9d1131d768c   bridge     virbr0 
Wired connection 1 	032bc5ee-d4a0-3374-a2a1-8b3079c39070  ethernet   ens36  
[root@localhost network-scripts]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 20.0.0.11  netmask 255.255.255.0  broadcast 20.0.0.255
		……省略部分
ens36: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.30.11  netmask 255.255.255.0  broadcast 192.168.30.255
        ……省略部分
[root@localhost ~]# yum -y install ipvsadm
[root@localhost ~]# ipvsadm -v   //查看ipvsadm信息
ipvsadm v1.27 2008/5/15 (compiled with popt and IPVS v1.2.1)
[root@localhost ~]# modprobe ip_vs   //加载ip_vs功能，确认内核对ip_vs的支持
[root@localhost ~]# cat /proc/net/ip_vs   //查看信息
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port Forward Weight ActiveConn InActConn
###############创建虚拟服务器规则################################
[root@localhost ~]# ipvsadm -A -t 20.0.0.11:80 -s rr
##############添加Web服务器节点规则##############################
[root@localhost ~]# ipvsadm -a -t 20.0.0.11:80 -r 192.168.30.22 -m -w 1
[root@localhost ~]# ipvsadm -a -t 20.0.0.11:80 -r 192.168.30.33 -m -w 1
##############保存规则，并记录到/opt/ipvsadm文件中###############
[root@localhost ~]# ipvsadm-save > /opt/ipvsadm
[root@localhost ~]# cat /opt/ipvsadm 
-A -t localhost.localdomain:http -s rr
-a -t localhost.localdomain:http -r 192.168.30.22:http -g -w 1
-a -t localhost.localdomain:http -r 192.168.30.33:http -g -w 1
###############做路由转发####################################
[root@localhost ~]# vi /etc/sysctl.conf 
net.ipv4.ip_forward=1
[root@localhost ~]# sysctl -p //查看转发信息
net.ipv4.ip_forward = 1

#################如果有iptables防火墙，需要做SNAT##############
[root@localhost ~]# iptables -t nat -A POSTROUTING -s 192.168.30.0/24 -o ens33 -j SNAT --to-source 20.0.0.11
#########################################.#######################

2. Build a server pool

2.1 Configure WEB1

[root@mysql2 ~]# ifconfig   //查看ip
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.30.22  netmask 255.255.255.0  broadcast 192.168.30.255
[root@mysql2 ~]# route -n   //查看路由表，看是否配置网关
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.30.11   0.0.0.0         UG    100    0        0 ens33
192.168.30.0    0.0.0.0         255.255.255.0   U     100    0        0 ens33
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0
[root@localhost ~]# yum -y install nfs-utils  //使用showmount需要安装这个工具
[root@localhost ~]# showmount -e 192.168.30.44  //查看共享状况
Export list for 192.168.30.44:
/opt/web2 192.168.30.0/24
/opt/web1 192.168.30.0/24
[root@mysql2 ~]# yum -y install httpd  //安装httpd
[root@mysql2 ~]# systemctl start httpd
[root@mysql2 ~]# systemctl enable httpd

2.2 Configure WEB2

[root@localhost ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.30.33  netmask 255.255.255.0  broadcast 192.168.30.255
[root@localhost ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.30.11   0.0.0.0         UG    100    0        0 ens33
192.168.30.0    0.0.0.0         255.255.255.0   U     100    0        0 ens33
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0
[root@localhost ~]# yum -y install nfs-utils
[root@localhost ~]# showmount -e 192.168.30.44   //若查看不到，可能是nfs服务器发布失败，去nfs服务器再次发布一下：exportsfs
Export list for 192.168.30.44:
/opt/web2 192.168.30.0/24
/opt/web1 192.168.30.0/24
[root@mysql2 ~]# yum -y install httpd
[root@mysql2 ~]# systemctl start httpd
[root@mysql2 ~]# systemctl enable httpd

3. Build shared storage

[root@localhost ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.30.44  netmask 255.255.255.0  broadcast 192.168.30.255
        inet6 fe80::a52a:406e:6512:1c66  prefixlen 64  scopeid 0x20<link>
[root@localhost ~]# route -n   //查看路由表，看网关
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.30.11   0.0.0.0         UG    100    0        0 ens33
192.168.30.0    0.0.0.0         255.255.255.0   U     100    0        0 ens33
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0
[root@localhost ~]# rpm -q nfs-utils  //查看nfs是否安装
nfs-utils-1.3.0-0.61.el7.x86_64
[root@localhost ~]# rpm -q rpcbind  //查看rpcbind是否安装
rpcbind-0.2.0-47.el7.x86_64
[root@localhost ~]# yum -y install nfs-utils  //确实安装了
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
Package 1:nfs-utils-1.3.0-0.61.el7.x86_64 already installed and latest version
Nothing to do
[root@localhost ~]# yum -y install rpcbind
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
Package rpcbind-0.2.0-47.el7.x86_64 already installed and latest version
Nothing to do
[root@localhost ~]# systemctl start nfs
[root@localhost ~]# systemctl enable nfs
Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.
[root@localhost ~]# systemctl start rpcbind
[root@localhost ~]# systemctl enable rpcbind
[root@localhost ~]# vi /etc/exports
/opt/web1 192.168.30.0/24(rw,sync)
/opt/web2 192.168.30.0/24(rw,sync)
[root@localhost ~]# systemctl restart nfs
[root@localhost ~]# systemctl restart rpcbind
[root@localhost ~]# showmount -e
Export list for localhost.localdomain:
/opt/web2 192.168.30.0/24
/opt/web1 192.168.30.0/24
[root@localhost web2]# exportfs -vr
exporting 192.168.30.0/24:/opt/web2
exporting 192.168.30.0/24:/opt/web1
[root@localhost ~]# mkdir /opt/web1/ /opt/web1/
[root@localhost ~]# vi /opt/web1/index.html
<html>
<title>I'm Web1</title>
<body><h1>I'm Web1</h1></body>
<img src="web1.jpg" />
</html>
[root@localhost ~]# vi /opt/web2/index.html
<html>
<title>I'm Web2</title>
<body><h1>I'm Web2</h1></body>
<img src="web2.png" />
</html>

4. Test shared storage

Log in to http://192.168.30.22, the test is normal.

Login to http://192.168.30.33, the test is normal

5. Test polling

Log in to http://20.0.0.11 and find that the data is read from the web1 and web2 servers in turn. The first visit is web1, refreshed every 1 minute (because there is a cache), and the web2 is accessed. Polling is normal

-----The explanation of common commands is as follows: ------

There are four most commonly used LVS load scheduling algorithms: round-robin algorithm (rr), weighted round-robin (wrr), least round-robin (Ic), and weighted least round-robin (wlc)

1) Create a virtual server (Note: NAT mode requires two network cards, and the address of the scheduler is the address of the external network port)

The VIP address of the cluster is 20.0.0.11, which provides load-distribution services for TCP port 80 and uses the round-robin scheduling algorithm. For the load balancing scheduler, the VIP must be the actual IP address of the machine
ipvsadm -A -t 20.0.0.11:80 -s rr //option "-A" means adding a virtual server, "-t" is used Specify the VIP address and TCP port, "-S" is used to specify the load scheduling algorithm-rr, wrr, lc, wlc.

2) Add server node

ipvsadm -a -t 20.0.0.11:80 -r 192.168.80.33:80 -m
ipvsadm -a -t 20.0.0.11:80 -r 192.168.80.44:80 -m
//The option "-a" means to add a real server, "-T" is used to specify VIP address and TCP port, "-r" is used to specify RIP address and TCP port, "-m" means to use NAT cluster mode ("-g" is DR mode, "-i" is TUN Horizontal)
{The -m parameter can also be followed by the -w parameter. The "-w" not done here is used to set the weight (when the weight is 0, the node is suspended)}

3) Delete the server node

ipvsadm -d -r 192.168.30.22:8o -t 20.0.0.11:80 //When you need to delete a node from the server pool, use the option "-d". To perform the delete operation, you must specify the target object, including the node address, Virtual IP address. The operation shown above will delete the node 192.168.30.22 in the LVS cluster 20.0.0.11

If you need to delete the entire virtual server, use option -D and specify the virtual IP address, no need to specify the node. For example: "ipvsadm -D -t 20.0.0.11:80", then delete this virtual service.
ipvsadm -L //View node status, add "-n" to display address and port information in digital form

4) Backup and restore rules

ipvsadm-save> /etc/sysconfig/ipvsadm1/ save strategy
Use the export/import tool ipvsadm-save/ipvsadm-restore to save and restore LVS strategy, the method is similar to the export and import of iptables rules
(you can use ipvsadm-restore </ dev/null to clear all rules)