Web Server Cluster--The Principle and Deployment of LVS Load Balancing Cluster

Introduction

The name cluster (or cluster) comes from the English word "Cluwster", which means a group or a bunch, and when used in the server field, it means a collection of a large number of servers to distinguish it from a single server. Depending on the actual environment of the enterprise, the functions provided by the cluster are also different, and the technical details used may also have their own merits. Here, we will give a brief introduction to the cluster structure, working mode, LVS virtual application, and NFS shared storage.

1. Principle of load balancing cluster

1.1 Overview of enterprise cluster applications

The meaning of clusters

• Cluster, cluster, cluster
• Consists of multiple hosts, but only externally appears as a whole

problem

• In Internet applications, as sites have higher and higher requirements for hardware performance, response speed, service stability, data reliability, etc., a single server is unable to do so

Solution

• Use expensive minicomputers and mainframes
• Use ordinary servers to build service clusters

Supplementary introduction:

• The SLB in Alibaba Cloud is a typical load balancing scheduler, and ECS is a cloud host (virtual machine);
• SLB schedules ECS, and multiple ECSs form a resource pool, forming the basis of cloud computing.

1.2 Classification of Enterprise Clusters

• No matter what kind of cluster, it includes at least two node servers

• Classification according to the target difference of the
  cluster Load balancing cluster
  High availability cluster
  High performance computing cluster

1.2.1 Load Balance Cluster

• Improve the responsiveness of the application system, process as many access requests as possible, reduce latency as the goal, and obtain high concurrency and high load (LB) overall performance. For example, DNS polling, application layer switching, reverse proxy, etc. can all be used as load balancing clusters.
• The load distribution of LB relies on the offloading algorithm of the master node to share the access requests from the client to multiple server nodes, thereby alleviating the load pressure of the entire system.

1.2.2 High Availability Cluster

• The goal is to improve the reliability of the application system, reduce the interruption time as much as possible, ensure the continuity of the service, and achieve the fault tolerance effect of high availability (HA). For example, failover, dual-system hot backup, and multi-system hot backup are all highly available cluster technologies;
• The working mode of HA includes duplex and master-slave modes. Duplex means working at the same time; master-slave means that only the master node is online, but when a failure occurs, the slave node can automatically switch to the master node.

1.2.3 High Performance Computer Cluster

• The goal is to improve the CPU computing speed of the application system, expand the hardware resources and analysis capabilities, and obtain the high-performance computing (HPC) capabilities equivalent to large-scale, supercomputers. For example, cloud computing and grid computing can also be regarded as a type of high-performance computing;

• High performance relies on "distributed computing" and "parallel computing". The CPU, memory and other resources of multiple servers are integrated through dedicated hardware and software to achieve computing capabilities that only large and supercomputers have.

1.3 Layered structure of load balancing

1.3.1 Overview

• In a typical load balancing cluster, there are three levels of components. There is at least one load scheduler at the front end to respond to load and distribute access requests from clients; the back end is composed of a large number of real servers to form a server pool to provide actual application services. The scalability of the entire cluster is accomplished by adding or deleting server nodes. The process is transparent to the client; in order to maintain service consistency, all nodes use shared storage devices.

1.3.2 Load Balancing Architecture

• The first layer, load scheduler (Load Balancer or Director)
• The second layer, the server pool (Server Pool)
• The third layer, shared storage (Share Storage)

1.3.3 Detailed introduction of load balancing architecture

• Load scheduler: This is the only entrance to access the entire cluster system. It uses the VIP (virtual IP) address common to all servers, also known as the cluster IP address. Usually two schedulers, main and backup schedulers are configured to achieve hot backup. When the main scheduler fails, it is smoothly replaced to the backup scheduler to ensure high availability;
• Server pool: The cluster provides application services and is borne by the server pool. Each node in the cluster has an independent RIP (real IP) address, and only processes client requests distributed by the scheduler; when a node fails, the load scheduler The fault-tolerant mechanism will isolate it and wait for the error to be eliminated before re-entering it into the server pool;
• Shared storage: Provide stable and consistent file access services for all nodes in the server pool to ensure the unity of the entire cluster.

1.4 Work Mode of Load Balancing

• Regarding the cluster load scheduling technology, it can be distributed based on IP, port, content, etc., of which the efficiency of IP-based load scheduling is the highest. In the IP-based load balancing mode, there are three common working modes: address translation, IP tunnel, and direct routing.

1.4.1 Network Address Translation

• Referred to as NAT mode, it is similar to the private network structure of a firewall. The load scheduler acts as the gateway of all server nodes, that is, as the access entrance of the client, and also the access exit of each node in response to the client;
• The server node uses a private IP address and is located on the same physical network as the load scheduler, and the security is better than the other two methods.

1.4.2 IP Tunnel

• Referred to as TUN mode, it adopts an open network structure. The load scheduler is only used as the client's access portal, and each node directly responds to the client through its own Internet connection without passing through the load scheduler;
• Server nodes are scattered in different locations on the Internet, have independent public IP addresses, and communicate with the load scheduler through a dedicated IP tunnel.
  

1.4.3 Direct Routing

• Referred to as DR mode, it adopts a semi-open network structure, which is similar to the structure of TUN mode, but the nodes are not scattered everywhere, but are located on the same physical network as the scheduler;
• The load scheduler is connected to each node server through the local network, and there is no need to establish a dedicated IP tunnel.
  
  The difference between DR and TUN mode:
• The same point: all web nodes directly respond to the client;
• Different: TUN mode has an independent public test, but DR does not; TUN web nodes communicate with the scheduler IP tunnel, while DR web nodes communicate with the local area network; TUN mode web nodes respond directly, and DR mode web nodes need to respond through a router.

1.5 The difference between the three working modes

Difference in working mode	NAT mode	TUN mode	DR mode
Real server (node ​​server)
Server number (number of nodes)	Low 10-20	High 100	High 100
Real gateway	Load scheduler	Own router	Free router
IP address	Public network + private network	public net	Private network
advantage	High security	Wan environment encrypted data	Highest performance
Disadvantage	Low efficiency and high pressure	Need tunnel support	Can't span LAN

2.LVS virtual server

2.1 Overview

• Linux Virtual Server is a load balancing solution for the Linux kernel, created in May 1998 by Dr. Wensong Zhang from my country. LVS is actually equivalent to virtualized applications based on IP addresses, and proposes an efficient solution for load balancing based on IP addresses and content request distribution.
• LVS has now become a part of the Linux kernel, compiled as an ip_vs module by default, and can be called automatically when necessary.

[root@localhost~]# modprobe ip_vs   '//加载ip_vs模块，确认内核对LVS的支持'
[root@localhost~]# cat /proc/net/ip_vs	'//查看ip_vs版本信息'

2.2 LVS load scheduling algorithm

2.2.1 Round Robin

• The received access requests are allocated to each node (real server) in the cluster in turn in order, and each server is treated equally, regardless of the actual number of server connections and system load.

2.2.2 Weighted Round Robin

• According to the processing capacity of the real server, the received access requests are allocated in turn. The scheduler can automatically query the load status of each node and dynamically adjust its weights to ensure that servers with strong processing capabilities bear more access traffic.

2.2.3 Least Connections

• The allocation is made according to the number of connections established by the real server, and the received access request is preferentially allocated to the node with the least number of connections. If all server nodes have similar performance, this method can better load balance.

2.2.4 Weighted Least Connections

• In the case of large differences in the performance of server nodes, the weights can be automatically adjusted for real servers, and nodes with higher weights will bear a greater proportion of the active connection load.

2.3 Use ipvsadm management tool

LVS cluster creation and management

• Create a virtual server
• Add, delete server nodes
• View cluster and node status
• Save load distribution strategy

3. Build LVS load balancing cluster

3.1 Case environment

• As the gateway of the Web server pool, the LVS scheduler has two LVS network cards (VM1 and VM2), respectively connected to the internal and external networks, using the round-robin (rr) scheduling algorithm

LVS load balancing cluster-address translation mode (LVS-NAT) environment

(1) One dispatch server

• IP address: 192.168.70.10 (intranet VM1)
• IP address: 192.168.80.10 (external network VM2)

(2) Two Web servers

• IP address: 192.168.70.11 (SERVER AA)
• IP address: 192.168.70.12 (SERVER AB)
  Note: The gateway of the Web server needs to point to the internal network card of the dispatcher

(3) NFS shared server

• IP address: 192.168.70.13

(4) One client, taking win7 as an example, for testing and verification

• IP address: 192.168.70.14 (intranet)
• IP address: 192.168.80.14 (external network)

Note: It is necessary to ensure that the same network segment can communicate with each other

3.2 Experimental purpose

• The win10 client accesses the 192.168.70.10 website, through the nat address translation, polling access to the Apache1 and Apache2 hosts;
• Build nfs network file storage service.

3.3 Deployment steps

• Load the ip_vs module, install the ipvsadm tool
• Turn on routing and forwarding;
• Create a new LVS virtual server and add a node server;
• Configure the node server:
  establish a test website,
  mount NFS shared storage,
  establish a test web page
• Save the rules and test

3.4 Project steps

3.4.1 Turn off the firewall

iptables -F
setenforce 0
systemctl stop firewalld

3.4.2 Configure NFS storage server

[root@nfs ~]# rpm -qa | grep rpcbind		//默认虚拟机已安装rpcbind模块
rpcbind-0.2.0-42.el7.x86_64
[root@nfs ~]# yum -y install nfs-utils	//确认是否安装nfs-utils软件包
已加载插件：fastestmirror, langpacks
base                                                     | 3.6 kB     00:00     
Loading mirror speeds from cached hostfile
 * base: 
软件包 1:nfs-utils-1.3.0-0.48.el7.x86_64 已安装并且是最新版本
无须任何处理
[root@nfs ~]# mkdir /opt/web1
[root@nfs ~]# mkdir /opt/web2
[root@nfs ~]# echo "<h1>this is web1.</h1>" > /opt/web1/index.html
[root@nfs ~]# echo "<h1>this is web2.</h1>" > /opt/web2/index.html
[root@nfs ~]# vi /etc/exports
/opt/web1 192.168.70.11/32 (ro)
/opt/web2 192.168.70.12/32 (ro)
[root@nfs ~]# systemctl restart nfs
[root@nfs ~]# systemctl restart rpcbind
[root@nfs ~]# showmount -e
Export list for nfs:
opt/web2 (everyone)
/opt/web1 (everyone)

3.4.3 Configure Web Site Server

yum -y install httpd
[root@web1 ~]# showmount -e 192.168.70.13
Export list for 192.168.70.13:
opt/web2 (everyone)
/opt/web1 (everyone)
[root@web1 ~]# mount 192.168.70.13:/opt/web1 /var/www/html
[root@web1 ~]# systemctl restart httpd
[root@web1 ~]# netstat -anpt | grep httpd
tcp6       0      0 :::80                   :::*                    LISTEN      55954/httpd   

[root@web2 ~]# mount 192.168.70.13:/opt/web2 /var/www/html
[root@web2 ~]# systemctl restart httpd
[root@web2 ~]# netstat -anpt | grep httpd
tcp6       0      0 :::80                   :::*                    LISTEN      54695/httpd 

3.4.4 Configure on the LVS dispatch server

[root@lvs ~]# vi /etc/sysctl.conf
...
net.ipv4.ip_forward=1
[root@lvs ~]# sysctl -p		'//开启路由地址转换'
net.ipv4.ip_forward=1
[root@lvs ~]# modprobe ip_vs	'//加载ip_vs模块'
[root@lvs ~]# cat /proc/net/ip_vs
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@lvs ~]# yum -y install ipvsadm
[root@lvs ~]# vi nat.sh
#!/bin/bash
ipvsadm -C
ipvsadm -A -t 192.168.80.10:80 -s rr
ipvsadm -a -t 192.168.80.10:80 -r 192.168.70.11:80 -m
ipvsadm -a -t 192.168.80.10:80 -r 192.168.70.12:80 -m
ipvsadm -Ln
//
-C：'表示清除缓存'
-A：'添加地址为192.168.80.10:80的虚拟地址，指定调度算法为轮询'
-a: '指定真实服务器，指定传输模式为NAT'
-t：'访问的入口地址，VIP'
rr：'表示轮询'
-m：'指的是NAT模式'

Run the script to start the service

[root@lvs ~]# sh nat.sh
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.80.10:80 rr
  -> 192.168.70.11:80             Masq    1      0          0         
  -> 192.168.70.12:80             Masq    1      0          0         

• Confirm whether the scheduler can access two web pages
  
  

3.4.5 Verification result

• Visit in browser
  
  
• View cluster real scheduling details

[root@lvs ~]# ipvsadm -lnc		'//查看真实调度明细'
IPVS connection entries
pro expire state       source             virtual            destination
TCP 01:20  FIN_WAIT    192.168.80.1:51398 192.168.80.10:80   192.168.70.11:80
TCP 00:44  TIME_WAIT   192.168.80.1:51378 192.168.80.10:80   192.168.70.12:80
TCP 00:29  TIME_WAIT   192.168.80.1:51371 192.168.80.10:80   192.168.70.11:80
TCP 00:17  TIME_WAIT   192.168.80.1:51356 192.168.80.10:80   192.168.70.12:80
TCP 11:39  ESTABLISHED 192.168.80.1:51301 192.168.80.10:80   192.168.70.11:80