The early website traffic and business functions were relatively simple, and a single server was sufficient to meet basic needs. However, with the development of the Internet, business traffic is increasing and business logic is becoming more and more complex. The point of failure problem is highlighted, so multiple servers are needed to expand performance horizontally and avoid single point of failure, so how to distribute the request traffic of different users to different servers?

2. The principle and processing flow of load balancing

System expansion can be divided into vertical expansion and horizontal expansion.

Vertical expansion is to improve the processing capacity of the server by increasing the hardware processing capacity of the system from the perspective of a single machine; horizontal expansion is to meet the processing capacity of large-scale website services by adding machines.

There are two important roles involved here: "application cluster" and "load balancer".

Application cluster: Deploy the same application to multiple machines to form a processing cluster, receive requests distributed by load balancing devices, process them and return response data.
Load balancer: Distributes user access requests to a server in the cluster for processing according to the corresponding load balancing algorithm.

3. The role of load balancing

Solve the high concurrency pressure of the server and improve the processing performance of the application;
Provide failover to achieve high availability;
Enhance the scalability of the website by adding or reducing the number of servers;
Filtering on the load balancer can improve the security of the system.

4. Common processing methods for load balancing

4.1. Method 1: Manual selection by the user

This method is relatively primitive, and the main way to achieve it is to provide different lines and different server link methods on the homepage of the website, allowing users to choose the specific server they visit to achieve load balancing.

4.2. Method 2: DNS polling method

DNS: Domain Name System (Service) Protocol (DNS) is a distributed network directory service, mainly used for mutual conversion between domain names and IP addresses.

Most domain name registrars support adding multiple A records to the same host name (the A record is the IP address), which is DNS round robin. The DNS server randomly assigns the resolution requests to different IPs in the order of the A records, so that Simple load balancing can be done. The cost of DNS polling is very low, and it is often used on some unimportant servers.

验证：
ping www.baidu.com

清空本地的dns缓存
ipconfig/flushdns

We found that using DNS to implement round robin does not require too much investment. Although DNS round robin is cheap, DNS load balancing has obvious disadvantages:

Low reliability: Suppose a domain name DNS polls multiple servers. If one of the servers fails, all requests to access the server will not be responded. In time, you remove the IP of the server from the DNS. However, because major broadband access providers store many DNS in the cache to save access time, the DNS will not be updated in real time. Therefore, DNS polling solves the load balancing problem to a certain extent, but there is a problem of low reliability. shortcoming.
Unbalanced load balancing: DNS load balancing uses a simple round-robin load algorithm, which cannot distinguish between servers, cannot reflect the current operating status of servers, and cannot allocate more requests to servers with good performance. In addition, the local computer will also cache The mapping of resolved domain names to IP addresses will also cause users using the DNS server to access the same web server within a certain period of time, thus causing load imbalance of the web server.

Unbalanced load will cause certain servers to have low compliance, while other servers have high loads, and the processing speed of requests is slow. Servers with high configurations are allocated fewer requests, while servers with low configurations are allocated more requests.

4.3, Layer 4/7 load balancing

OSI (Open System Interconnection, Open System Interconnection Model) is a network architecture specified by the International Organization for Standardization ISO that is not based on specific models, operating systems, or companies. The model divides the work of network communication into seven layers.

OSI七层模型;
7、应用层：为应用程序提供网络服务；
6、表示层：对数据进行格式化、编码、加密、压缩等操作；
5、会话层：建立、维护、管理会话连接；
4、传输层：建立、维护、管理端到端的连接，常见的有TCP/UDP；
3、网络层：IP寻址和路由选择；
2、数据链路层：控制网络层与物理层之间的通信；
1、物理层：比特流传输。

The so-called four-layer load balancing refers to the transport layer in the OSI seven-layer model, mainly based on IP+PORT load balancing:

实现四层负载均衡的方式：
    硬件：F5、BIG-IP、Radware等；
    软件：LVS、Nginx、Hayproxy等。

The so-called seven-layer load balancing refers to the application layer, mainly based on virtual URL or host IP load balancing.

实现七层负载均衡的方式：
    软件：Nginx、Hayproxy等。

The difference between Layer 4 and Layer 7 load balancing:

Layer 4 load balancing packets are distributed at the bottom layer, while layer 7 load balancing packets are distributed at the top, so the efficiency of layer 4 load balancing is higher than that of layer 7 load balancing;
Layer-4 load balancing does not recognize domain names, while layer-7 load balancing recognizes domain names.

In addition to Layer 4 and Layer 7 load balancing, there are actually Layer 2 and Layer 3 load balancing. Layer 2 load balancing is based on MAC addresses at the data link layer to achieve load balancing, and layer 3 is generally using virtual IP addresses to achieve load balancing at the network layer.

The method adopted in the actual environment: four-layer load (LVS) + seven-layer load (Nginx).

5. Nginx seven-layer load balancing

Nginx needs to use the proxy_pass proxy module configuration to achieve seven-layer load balancing. Nginx supports this module by default, and we don't need to do anything else. Nginx's load balancing is based on Nginx's reverse proxy to distribute user requests to a group of [upstream virtual service pools] according to a specified algorithm.

5.1, Nginx seven layer load balancing instructions

5.1.1, upstream command

upstream: This command is used to define a group of servers, which can be servers listening to different ports, and can also be servers listening to TCP and Unix sockets at the same time. The server can specify a different weight, the default is 1.

grammar	upstream name{...}
Defaults	——
Location	http

5.1.2, server command

server: This command is used to specify the name of the backend server and some parameters, which can use domain name, IP, port or unix socket.

grammar	server name [parameters];
Defaults	——
Location	upstream

5.2, Nginx load balancing configuration

	# 服务器1
	server {
		listen 9001;
		server_name localhost;
		default_type text/html;
		
		location / {
			return 200 '<h1>server:9001</h1>';
		}
	}
	# 服务器2
	server {
		listen 9002;
		server_name localhost;
		default_type text/html;
		
		location / {
			return 200 '<h1>server:9002</h1>';
		}
	}
	# 服务器3
	server {
		listen 9003;
		server_name localhost;
		default_type text/html;
		
		location / {
			return 200 '<h1>server:9003</h1>';
		}
	}
	
	# 代理服务器
	# 设置服务器组
	upstream backend {
		server localhost:9001;
		server localhost:9002;
		server localhost:9003;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location / {
			# backend 就是服务器组的名称
			proxy_pass http://backend/;
		}
	}

5.3, Nginx load balancing status

The states of the proxy server in charge of balanced scheduling are as follows:

state	overview
down	The current server does not participate in load balancing temporarily
backup	reserved backup server
max_fails	The number of failed requests allowed
fail_timeout	Service pause time after max_fails failures
max_conns	Limit the maximum number of receive connections

5.3.1. Status: down

down: Mark the server as permanently unavailable, then the proxy server will not participate in load balancing.

	# 代理服务器
	# 设置服务器组
	upstream backend {
		server localhost:9001 down;
		server localhost:9002;
		server localhost:9003;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location / {
			# backend 就是服务器组的名称
			proxy_pass http://backend/;
		}
	}

This status is generally set for servers that need to be shut down for maintenance.

5.3.2. Status: backup

backup: Mark this server as a backup server, which will be used to deliver requests when the primary server is unavailable.

	# 代理服务器
	# 设置服务器组
	upstream backend {
		server localhost:9001 down;
        # 9002 作为 9003 的备份服务器
		server localhost:9002 backup;
		server localhost:9003;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location / {
			# backend 就是服务器组的名称
			proxy_pass http://backend/;
		}
	}

At this time, access to port 9003 needs to be disabled to simulate that when the only service that can provide access to the outside world goes down, the backup server of the backup will start to provide services to the outside world. At this time, in order to test and verify, we need to use a firewall to intercept.

Introduce a tool fire-wall-cmd, which is provided by Linux to operate firewall.

-查询防火墙中指定的端口是否开放
firewall-cmd --query-port=9001/tcp

-开放一个指定的端口
firewwall-cmd --permanent --add-port=9002/tcp

-批量添加开发端口
firewall-cmd --permanent --add-port=9001-9003/tcp

-移除一个指定的端口
firewall-cmd --permanent --remove-port=9003/tcp

-重新加载
firewall-cmd --reload

5.3.3. Status: max_fails

max_fails=numer: Set the number of times the proxy server is allowed to fail, the default is 1.

5.3.4. Status: fail_timeout

fail_timeout=time: Set the service suspension time after max_fail failures, the default is 10 seconds.

	# 代理服务器
	# 设置服务器组
	upstream backend {
		server localhost:9001 down;
        # 9002 作为 9003 的备份服务器
		server localhost:9002 backup;
		server localhost:9003 max_fails=3 fail_timeout=15;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location / {
			# backend 就是服务器组的名称
			proxy_pass http://backend/;
		}
	}

5.3.5. Status: max_conns

max_conns=number: It is used to set the maximum number of active connections of the proxy server at the same time. The default is 0, which means no limit. Using this configuration, it can be set according to the concurrent amount of requests processed by the back-end server to prevent the back-end server from being overwhelmed.

5.4, Nginx load balancing strategy

After introducing the relevant instructions of Nginx load balancing, we have been able to distribute user requests to different servers, so what kind of load algorithm can we use besides the default distribution method?

The upstream of Nginx supports the following six distribution algorithms, namely:

algorithm name	illustrate
polling	default method
weight	weight method
ip_hash	According to the ip allocation method
least_conn	According to the least connection method
url_hash	According to URL distribution method
fair	By response time method

5.4.1, Polling

Round robin is the default load balancing strategy of the upstream module, and each request will be assigned to different backend servers one by one in chronological order. Polling requires no additional configuration.

	# 代理服务器
	# 设置服务器组
	upstream backend {
		server localhost:9001;
		server localhost:9002;
		server localhost:9003;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location / {
			# backend 就是服务器组的名称
			proxy_pass http://backend/;
		}
	}

5.4.2, weight weighting (weighted round robin)

weight=number: used to set the weight of the server, the default is 1, the greater the weight number, the greater the probability of being assigned to the request. The weight value is mainly adjusted for different back-end server hardware configurations in the actual working environment, so this strategy is more suitable for situations where the server hardware configurations are quite different.

	# 代理服务器
	# 设置服务器组
	upstream backend {
		server localhost:9001 weight=10;
		server localhost:9002 weight=5;
		server localhost:9003 weight=3;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location / {
			# backend 就是服务器组的名称
			proxy_pass http://backend/;
		}
	}

5.4.3、ip_hash

When performing load balancing on multiple dynamic application servers at the backend, the ip_hash command can locate a client IP request to the same backend server through a hash algorithm. In this way, when a user from a certain IP logs in on the back-end web server A, and then accesses other URLs of the site, it can be guaranteed that the user accesses the back-end web server A.

grammar	ip_hash;
Defaults	——
Location	upstream

	# 代理服务器
	# 设置服务器组
	upstream backend {
        ip_hash;
		server localhost:9001;
		server localhost:9002;
		server localhost:9003;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location / {
			# backend 就是服务器组的名称
			proxy_pass http://backend/;
		}
	}

Note: Using the ip_hash command cannot guarantee the load balance of the back-end servers, which may cause some back-end servers to receive more requests, and some back-end servers to accept fewer requests, and methods such as setting back-end server weights will not work.

5.4.4、least_conn

least_conn: The least connection, forward the request to the backend server with fewer connections. The polling algorithm is to forward the requests to each backend evenly, so that their loads are roughly the same; however, some requests take a long time, which will lead to a higher load on the backend where they are located. In this case, least_conn can achieve better load balancing effect.

	# 代理服务器
	# 设置服务器组
	upstream backend {
        least_conn;
		server localhost:9001;
		server localhost:9002;
		server localhost:9003;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location / {
			# backend 就是服务器组的名称
			proxy_pass http://backend/;
		}
	}

This load balancing is suitable for situations where the server is overloaded due to different request processing times.

5.4.5、url_hash

Allocate requests according to the hash result of the accessed url, so that each url is directed to the same backend server, and it should be used in conjunction with cache hits. Multiple requests for the same resource may arrive at different servers, resulting in unnecessary multiple downloads, low cache hit rate, and waste of some resource time. Using url_hash can make the same url (that is, the same resource request) reach the same server. Once the resource is cached and the request is received again, it can be read from the cache.

	# 代理服务器
	# 设置服务器组
	upstream backend {
        hash $request_uri;
		server localhost:9001;
		server localhost:9002;
		server localhost:9003;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location / {
			# backend 就是服务器组的名称
			proxy_pass http://backend/;
		}
	}

5.4.6、fair

Fair does not use the balancing algorithm used by the built-in load balancing, but can intelligently perform load balancing according to the page size and loading time. So how to use the fair load balancing strategy of the third-party module?

	# 代理服务器
	# 设置服务器组
	upstream backend {
        fair;
		server localhost:9001;
		server localhost:9002;
		server localhost:9003;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location / {
			# backend 就是服务器组的名称
			proxy_pass http://backend/;
		}
	}

However, if you use it directly, an error will be reported, because fair belongs to the load balancing implemented by a third-party module. Need to add nginx-upstream-fair, how to add the corresponding module:

Download the nginx-upstream-fair module.

下载地址：
https://github.com/gnosek/nginx-upstream-fair

Upload the downloaded file to the server and unzip it.
```
unzip nginx-upstream-fair-master.zip
```
Rename the resource.
```
mv nginx-upstream-fair-master fair
```
Add resources to the Nginx module using the ./configure command.
```
./configure --add-module=/root/fair
```

compile.

make

-编译可能会出现如下错误：
nginx_http_upstream_srv_conf_t结构中缺少default_port

-解决方案
在Nginx的源码中src/http/nginx_http_upstream.h，找到ngx_http_upstream_srv_conf_s，在
模块中添加default_port属性：
    in_port_t default_port
然后再进行make。

Update Nginx.

1、将sbin目录下的nginx进行备份
    mv /usr/local/nginx/sbin/nginx /usr/local/nginx/sbin/nginxold

2、将安装目录下的objs中的nginx拷贝到sbin目录
    cd objs
    cp nginx /usr/local/nginx/sbin

3、更新
    cd ../
    make upgrade

Compilation tests use Nginx.

5.5, Nginx seven-layer load balancing case

5.5.1, Case 1 - load balancing of general polling rules for all requests

	# 代理服务器
	# 设置服务器组
	upstream backend {
		server localhost:9001;
		server localhost:9002;
		server localhost:9003;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location / {
			# backend 就是服务器组的名称
			proxy_pass http://backend/;
		}
	}

5.5.2, Case 2 - Implement load balancing of weighted round-robin rules for all requests

	# 代理服务器
	# 设置服务器组
	upstream backend {
		server localhost:9001 weight=7;
		server localhost:9002 weight=5;
		server localhost:9003 weight=3;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location / {
			# backend 就是服务器组的名称
			proxy_pass http://backend/;
		}
	}

5.5.3, Case 3 - to achieve load balancing for specific resources

	# 代理服务器
	# 设置服务器组
	upstream videobackend {
		server localhost:9001;
		server localhost:9002;
	}
	upstream filebackend {
		server localhost:9003;
		server localhost:9004;
	}
	server {
		listen 8080;
		server_name localhost;
		
		location /video/ {
			# backend 就是服务器组的名称
			proxy_pass http://videobackend;
		}
		location /file/ {
			proxy_pass http://filebackend;
		}
	}

5.5.4, Case 4 - Implement load balancing for different domain names

	# 代理服务器
	# 设置服务器组
	upstream aaabackend {
		server 192.168.200.146:9001;
		server 192.168.200.146:9002;
	}
	upstream bbbbackend {
		server 192.168.200.146:9003;
		server 192.168.200.146:9004;
	}
	server {
		listen 8080;
		server_name www.aaa.com;
		location / {
			proxy_pass http://aaabackend;
		}
	}
	server {
		listen 8081;
		server_name www.bbb.com;
		location / {
			proxy_pass http://bbbbackend;
		}
	}

5.5.5, Case 5 - Implementing load balancing with URL rewriting

	# 代理服务器
	# 设置服务器组
	upstream backend {
		server 192.168.200.146:9001;
		server 192.168.200.146:9002;
	}
	server {
		listen 8080;
		server_name localhost;
		
		# URL重写
		location /file/ {
			rewrite ^(/file/.*) /server/$1 last;
		}
		
		location /server {
			proxy_pass http:backend;
		}
	}

6. Nginx four-layer load balancing

After Nginx 1.9, a stream module was added to implement forwarding, proxying, load balancing, etc. of the four-layer protocol. The usage of the stream module is similar to that of http, which allows us to configure a set of monitoring protocols such as TCP or UDP, and then forward our requests through proxy_pass, and add multiple backend services through upstream to achieve load balancing.

The implementation of layer-4 protocol load balancing generally uses LVS, HAProxy, F5, etc., which are either expensive or cumbersome to configure, while the configuration of Nginx is relatively simpler and can complete the work more quickly.

6.1. Add support for stream module

Nginx does not compile this module by default. If you need to use the stream module, you need to add --with-stream when compiling.

Complete the implementation steps of adding --with-stream:

将原有/usr/local/nginx/sbin/nginx进行备份；
拷贝nginx之前的配置信息；
在nginx的安装源码进行配置指定对应模块 ./configure --with-stream
通过make模板进行编译；
将objs下面的nginx移动到/usr/local/nginx/sbin下；
在源码目录下执行 make upgrade 进行升级，这个可以实现不停机添加新模块的功能

6.2. Nginx four-layer load balancing instructions

6.2.1, stream instruction

stream: This directive provides the configuration file context in which to specify the stream server. Same level as http command.

grammar	stream {...}
Defaults	——
Location	main

6.2.2, upstream command

upstream: This command is similar to the upstream command of http.

6.3, Nginx four-layer load balancing case

demand analysis:

Implementation steps:

Prepare the Redis server, prepare three Redis on one server (192.168.200.146), the ports are 9379, 9378, 9377;

1、上传redis的安装包
    redis-4.0.14.tar.gz

2、将安装包进行解压缩
    tar -zxf -redis-4.0.14.tar.gz

3、进入redis的安装包
    cd redis-4.0.14

4、使用make和install进行编译和安装
    make PREFIX=/usr/local/redis/redis01 install

5、拷贝redis配置文件redis.conf到/usr/local/redis/redis01/bin目录中
    cp redis.conf /usr/local/redis/redis01/bin

6、修改redis.conf配置文件
    port 9379; # redis 的端口
    deamonize yes; # 后台启动redis

7、将redis01复制两份为redis02,redis03
    cp -r -redis01/ redis02
8、修改redis02,redis03的配置redis.conf配置文件   
    port 9378; # redis 的端口
    deamonize yes; # 后台启动redis

    port 9377; # redis 的端口
    deamonize yes; # 后台启动redis

9、修改3个redis的redis.conf中如下内容：
    bind 127.0.0.1 ==> bind 0.0.0.0

10、分别启动，即可获取3个redis，并查看
    ps -ef | grep redis
使用Nginx将请求分发到不同的Redis服务器上。

Prepare Tomcat server.

1、上传tomcat的安装包，apache-tomcat-8.5.56.tar.gz

2、将安装包进行解压缩
    tar -zxf apache-tomcat-8.5.56.tar.gz

3、进入tomcat的bin目录
    cd apache-tomcat-8.5.56/bin
    ./startup

# 配置nginx四层负载均衡相关内容
stream {
	# redis 服务器群组
	upstream redisbackend {
		server 192.168.200.146:9379;
		server 192.168.200.146:9378;
		server 192.168.200.146:9377;
	}
	# 监听并转发至redis服务器组
	server {
		listen 81;
		# server_name 指令在该块不能使用
		# redisbackend 为服务群组的名称
		proxy_pass redisbackend;
	}
	
	# tomcat 服务器群组
	upstream tomcatbackend {
		server 192.168.200.146:8080;
	}
	# 监听并转发至tomcat服务器
	server {
		listen 82;
		proxy_pass tomcatbackend;
	}
}

Nginx - Nginx load balancing

1. Overview of load balancing

2. The principle and processing flow of load balancing

3. The role of load balancing

4. Common processing methods for load balancing

4.1. Method 1: Manual selection by the user

4.2. Method 2: DNS polling method

4.3, Layer 4/7 load balancing

5. Nginx seven-layer load balancing

5.1, Nginx seven layer load balancing instructions

5.1.1, upstream command

5.1.2, server command

5.2, Nginx load balancing configuration

5.3, Nginx load balancing status

5.3.1. Status: down

5.3.2. Status: backup

5.3.3. Status: max_fails

5.3.4. Status: fail_timeout

5.3.5. Status: max_conns

5.4, Nginx load balancing strategy

5.4.1, Polling

5.4.2, weight weighting (weighted round robin)

5.4.3、ip_hash

5.4.4、least_conn

5.4.5、url_hash

5.4.6、fair

5.5, Nginx seven-layer load balancing case

5.5.1, Case 1 - load balancing of general polling rules for all requests

5.5.2, Case 2 - Implement load balancing of weighted round-robin rules for all requests

5.5.3, Case 3 - to achieve load balancing for specific resources

5.5.4, Case 4 - Implement load balancing for different domain names

5.5.5, Case 5 - Implementing load balancing with URL rewriting

6. Nginx four-layer load balancing

6.1. Add support for stream module

6.2. Nginx four-layer load balancing instructions

6.2.1, stream instruction

6.2.2, upstream command

6.3, Nginx four-layer load balancing case

Guess you like

Nginx - Nginx load balancing

1. Overview of load balancing

2. The principle and processing flow of load balancing

3. The role of load balancing

4. Common processing methods for load balancing

4.1. Method 1: Manual selection by the user

4.2. Method 2: DNS polling method

4.3, Layer 4/7 load balancing

5. Nginx seven-layer load balancing

5.1, Nginx seven layer load balancing instructions

5.1.1, upstream command

5.1.2, server command

5.2, Nginx load balancing configuration

5.3, Nginx load balancing status

5.3.1. Status: down

5.3.2. Status: backup

5.3.3. Status: max_fails

5.3.4. Status: fail_timeout

5.3.5. Status: max_conns

5.4, ​​Nginx load balancing strategy

5.4.1, Polling

5.4.2, weight weighting (weighted round robin)

5.4.3、ip_hash

5.4.4、least_conn

5.4.5、url_hash

5.4.6、fair

5.5, Nginx seven-layer load balancing case

5.5.1, Case 1 - load balancing of general polling rules for all requests

5.5.2, Case 2 - Implement load balancing of weighted round-robin rules for all requests

5.5.3, Case 3 - to achieve load balancing for specific resources

5.5.4, Case 4 - Implement load balancing for different domain names

5.5.5, Case 5 - Implementing load balancing with URL rewriting

6. Nginx four-layer load balancing

6.1. Add support for stream module

6.2. Nginx four-layer load balancing instructions

6.2.1, stream instruction

6.2.2, upstream command

6.3, Nginx four-layer load balancing case

Guess you like

5.4, Nginx load balancing strategy