Talk about those things about load balancing

Interview-oriented blogs are presented in Q / A style.

---

Question1: What about load balancing?

Answer1：

meaning:

Load balancing refers to a mechanism that provides a cheap, effective, and transparent method to expand the bandwidth of network devices and servers, increase throughput, strengthen network data processing capabilities, and increase network flexibility and availability on top of existing network structures.

There are two types of load balancing, four-layer load balancing and seven-layer load balancing, as shown in the following figure:

Layer 4 load balancing (destination address and port exchange)

Mainly through the destination address and port in the message, plus the server selection method set by the load balancing device, the final selection of the internal server is determined.

Taking common TCP as an example, when the load balancing device receives the first SYN request from the client, it selects an optimal server in the above manner, and modifies the target IP address in the message (changed to the back-end server IP), forwarded directly to the server. TCP connection establishment, that is, the three-way handshake is established directly by the client and the server, and the load balancing device only acts as a router-like forwarding action. In some deployment situations, in order to ensure that the packet returned by the server can be correctly returned to the load balancing device, the original source address of the packet may be modified while forwarding the packet.

The software to achieve four-layer load balancing are:

F5: Hardware load balancer, with good functions but high cost.
lvs: heavy-duty four-layer load software.
nginx: Lightweight four-layer load software with cache function, regular expressions are more flexible.
haproxy: Simulates layer 4 forwarding, which is more flexible.

Seven-layer load balancing (content exchange)

The so-called seven-layer load balancing, also known as "content exchange", is to determine the internal server that is ultimately selected through the truly meaningful application layer content in the message, plus the server selection method set by the load balancing device.

The benefit of the seven-layer application load is to make the entire network more intelligent. For example, user traffic visiting a website can forward requests for image classes to specific image servers and can use caching technology through seven layers; requests for text classes can be forwarded to specific text servers and compression can be used technology.

The software that achieves seven layers of load balancing is:

haproxy: inherent load balancing skills, full support for seven-layer proxy, session maintenance, marking, path transfer;
nginx: only functions better on http and mail protocols, performance is similar to haproxy;
apache: poor function
Mysql proxy: function is still can.

---

Question2: Briefly introduce the load balancing algorithm / strategy?

Answer2：

1. Round Robin

Each request from the network is distributed to the internal servers in turn, from 1 to N and then restarted. This kind of balancing algorithm is suitable for all servers in the server group have the same software and hardware configuration and the average service request is relatively balanced.

2. Weighted Round Robin

According to the different processing capabilities of the server, each server is assigned different weights so that it can accept service requests with corresponding weights. For example, the weight of server A is designed to be 1, the weight of B is 3, and the weight of C is 6, then servers A, B, and C will receive 10%, 30%, and 60% service requests, respectively. This balancing algorithm can ensure that high-performance servers get more usage and avoid overloading low-performance servers.

3. Random equilibrium (Random)

Randomly assign requests from the network to multiple servers in the interior.

4. Weighted Random

This kind of equalization algorithm is similar to the weight round robin algorithm, but it is a random selection process when processing request sharing.

5. Balanced response speed (Response Time detection time)

The load balancing device sends a probe request (such as Ping) to each internal server, and then determines which server responds to the client's service request based on the fastest response time of the internal server to the probe request. This balancing algorithm can better reflect the current running state of the server, but the fastest response time only refers to the fastest response time between the load balancing device and the server, not the fastest response time between the client and the server.

6. Least Connection Balance

The minimum connection number balancing algorithm has a data record for each server in the internal load, which records the number of connections currently being processed by the server. When there is a new service connection request, the current request will be assigned to the least connected number The server makes the balance more in line with the actual situation and the load is more balanced. This balancing algorithm is suitable for long-time processing request services, such as FTP.

7. Balanced processing capacity (CPU, memory)

This balancing algorithm will distribute the service request to the internal processing load (converted according to the server CPU model, number of CPUs, memory size and current number of connections). The lightest server, due to the internal server's processing capacity and current network Running status, so this balancing algorithm is relatively more accurate, especially suitable for the seventh layer (application layer) load balancing.

8. DNS response balance (Flash DNS)

Under this balancing algorithm, load balancing devices located in different geographic locations receive the domain name resolution request of the same client, and at the same time resolve this domain name to the IP address of the corresponding server and return it to the client, then The client will continue to request service with the domain name resolution IP address received first, while ignoring other IP address responses. When this kind of balancing strategy is suitable for application in global load balancing, it is meaningless for local load balancing.

9. Hash algorithm

Consistent Hash Consistency Hash, requests with the same parameters are always sent to the same provider. When a provider hangs up, the request originally sent to the provider is based on the virtual node, and it is evenly distributed to other providers without causing drastic changes.

10. Hash of IP address (ensure that the correspondence between the client and server is stable)

An algorithm that uniformly forwards packets from the same sender (or packets sent to the same destination) to the same server by managing the hash of the sender IP and destination IP addresses. When the client has a series of services to be processed and must repeatedly communicate with a server, the algorithm can use streams (sessions) as a unit to ensure that communications from the same client can always be processed in the same server.

11. URL hashing
An algorithm that forwards requests sent to the same URL to the same server by managing the hash of the URL information requested by the client.

---

Question3: Introduce LVS?

Answer3：

LVS's IP load balancing technology is implemented through the IPVS module. IPVS is the core software of the LVS cluster system. Its main function is to install on the Director Server and at the same time virtualize an IP address on the Director Server. The user must pass this Virtual IP address to access the server. This virtual IP is generally called the VIP of LVS, or Virtual IP. The access request first reaches the load scheduler through the VIP, and then the load scheduler selects a service node from the Real Server list to respond to the user's request. After the user's request reaches the load scheduler, how the scheduler sends the request to the Real Server node that provides the service, and how the Real Server node returns data to the user is the key technology implemented by IPVS.

ipvs: working in the kernel space, mainly used to make user-defined strategies take effect.
ipvsadm: Work in user space, mainly used for users to define and manage cluster service tools.

ipvs works on the INPUT chain in the kernel space. When a user requests a cluster service, it passes through the PREROUTING chain, and after checking the local routing table, it is sent to the INPUT chain. When entering the netfilter INPUT chain, ipvs forces the request message. The path of the cluster service strategy defined by ipvsadm is changed to the FORWORD chain, and the message is forwarded to the real host that provides the service at the back end.

---

Question4: What are the four modes of LVS (NAT, DR, TUN, FULLNAT)?

Answer4：

1. LVS NAT mode

①. The client sends the request to the front-end load balancer. The source address of the request message is CIP (client IP), which is collectively referred to as CIP later, and the destination address is VIP (the front-end address of the load balancer, which is collectively referred to as VIP later).
②. After receiving the message, the load balancer finds that the request is the address existing in the rule, then it changes the target address of the client request message to the RIP address of the back-end server and sends the message according to the algorithm.
③. After the message is sent to Real Server, since the destination address of the message is itself, it will respond to the request and return the response message to LVS.
④. Then lvs changes the source address of this message to this machine and sends it to the client.

Note: In NAT mode, the gateway of the Real Server must point to LVS, otherwise the message cannot be delivered to the client.

The characteristics of LVS NAT mode:
1. NAT technology needs to rewrite the address of the request message and the response message. Therefore, when the website visits are relatively large, the LB load balancing scheduler has a relatively large bottleneck, which generally requires the most Can be 10-20 nodes.
2. You only need to configure a public IP address on the LB.
3. The gateway address of each internal realserver must be the intranet address of the scheduler LB.
4. The NAT mode supports the conversion of IP addresses and ports. That is, the port requested by the user and the port of the real server may be different.

Advantages of LVS NAT mode:
The physical servers in the cluster can use any operating system that supports TCP / IP, and only the load balancer needs a legal IP address.

Disadvantages of LVS NAT mode:
limited scalability. When the server node (common PC server) grows too much, the load balancer will become the bottleneck of the entire system, because all the request and response packets flow through the load balancer. When there are too many server nodes, a large number of data packets are converged on the load balancer, and the speed will be slower!

2. LVS DR mode (LAN rewrite mac address)

①. The client sends the request to the front-end load balancer, the source address of the request message is CIP, and the destination address is VIP.
②. After receiving the message, the load balancer finds that the request is the address existing in the rule, then it changes the source MAC address of the client request message to its own DIP MAC address, and the target MAC to the RIP MAC address And send this packet to RS.
③.RS finds that the destination MAC in the request message is itself, and it will receive the secondary message. After processing the request message, the response message will be sent to the eth0 network card through the lO interface and sent directly to the client.

Note: The VIP that needs to be set to the lO interface cannot respond to arp requests in the local network.

Features of LVS DR mode:
1. Forwarding is achieved by modifying the destination MAC address of the data packet on the scheduler LB. Note that the source address is still CIP, and the destination address is still a VIP address.
2. The requested message passes through the scheduler, and the RS response processed message does not need to pass through the scheduler LB, so the use efficiency is very high when the number of concurrent accesses is large (compared to the NAT mode)
3. Because the DR mode is rewritten through the MAC address The mechanism realizes forwarding, so all RS nodes and scheduler LB can only be in a local area network.
4. The RS host needs to bind the VIP address to the LO interface (mask 32 bits), and ARP suppression needs to be configured.
5. The default gateway of the RS node does not need to be configured as LB, but is directly configured as the gateway of the superior route, which allows the RS to go directly to the network.
6. Since the DR mode scheduler only rewrites the MAC address, the scheduler LB cannot rewrite the target port, so the RS server has to use the same port as the VIP to provide services.
7. For direct external services such as WEB, RS's IP is best to use public IP. For external services, such as databases, it is best to use intranet IP.

Advantages of LVS DR mode

Like TUN (Tunnel Mode), the load balancer only distributes the request, and the response packet is returned to the client through a separate routing method. Compared with VS-TUN, this implementation of VS-DR does not require a tunnel structure, so most operating systems can be used as physical servers.

The efficiency of DR mode is very high, but the configuration is a little more complicated, so for companies that are not particularly heavy in traffic, you can use
haproxy / nginx instead . 1000-2000W PV per day or 10,000 concurrent requests can be considered haproxy / nginx.

Disadvantages of LVS DR mode
All RS nodes and scheduler LB can only be in one LAN.

3. LVS TUN mode (IP encapsulation, cross-network segment)

①. The client sends the request to the front-end load balancer, the source address of the request message is CIP, and the destination address is VIP.
②. After receiving the message, the load balancer finds that the request is the address existing in the rule, then it will encapsulate another layer of IP message in the header of the client request message, and change the source address to DIP, the destination address Change to RIP and send this packet to RS.
③. After receiving the request message, RS will first unpack the first layer of encapsulation, and then find that there is also a layer of IP header whose target address is the VIP on its lO interface, so it will process the second request message and send the response message The text is sent to the eth0 network card through the lo interface and sent directly to the client.

Note: VIPs that need to set the lO interface cannot appear on the public network.

Features of LVS TUN mode
1. TUNNEL mode must be bound to the IP address of VIP on all realserver machines 2. Vip of
TUNNEL mode ------> realserver packet communication through TUNNEL mode, both internal and external networks Can communicate, so there is no need for lvs vip and realserver to be in the same network segment.
3. TUNNEL mode realserver will send the packet directly to the client, but not to lvs.
4.Tunnel mode is a tunnel mode, so it is difficult to operate and maintain, so it is generally not needed.

LVS TUN mode advantages

The load balancer is only responsible for distributing the request packet to the back-end node server, while RS sends the response packet directly to the user. Therefore, a large amount of data flow of the load balancer is reduced. The load balancer is no longer a bottleneck of the system and can handle a huge amount of requests. In this way, one load balancer can distribute many RSs. And running on the public network can be distributed in different regions.

Disadvantages of LVS TUN mode
RS nodes in tunnel mode require legal IP. This method requires all servers to support the "IP Tunneling" (IP
Encapsulation) protocol. The server may be limited to some Linux systems.

4. LVS FULLNAT mode

Whether it is DR or NAT mode, it is inevitable that there is a problem: LVS and RS must be in the same VLAN, otherwise LVS cannot serve as the gateway of RS. The two problems raised by this are:

1. The limitation of the same VLAN leads to inconvenient operation and maintenance, and RSs across VLANs cannot be accessed.
2. The horizontal expansion of LVS is restricted. When RS is expanded horizontally, the single-point LVS on it will one day become a bottleneck.

Full-NAT was born from this, the problem of cross-VLAN between LVS and RS was solved, and after the cross-VLAN problem was solved, there was no longer a subordinate relationship between VLANs on LVS and RS, and multiple LVS could correspond to multiple RSs. The problem of horizontal expansion.

The main improvement of Full-NAT compared to NAT is that on the basis of SNAT / DNAT, plus another conversion, the conversion process is as follows:

1. In the process of transferring packets from LVS to RS, the source address is replaced from the client IP to the intranet IP of LVS. Intranet IP can communicate across VLANs through multiple switches. The target address is changed from VIP to RS IP.
2. When RS processes the received packet and returns after processing, the target address is changed to LVS ip, the original address is changed to RS IP, and finally the packet is returned to the LVS intranet IP, this step is not limited to VLAN .
3. After receiving the packet, the LVS changes the source address in the NAT mode, and then changes the destination address in the packet sent by the RS from the LVS intranet IP to the client's IP, and changes the original address to VIP.

The main idea of ​​Full-NAT is to change the communication between the gateway and the machines below it to ordinary network communication, thus solving the problem of cross-VLAN. In this way, the deployment of LVS and RS will no longer have any restrictions on the VLAN, which greatly improves the convenience of O & M deployment.

Features of LVS FULLNAT mode

1. FULL NAT mode does not require LBIP and realserver ip to be on the same network segment;
2. full nat because of the need to update the sorce ip, the performance is normally 10% lower than that of nat mode

---

Question5: Introduce Keepalive?

Answer5：

Keepalive was originally designed for LVS, specifically used to monitor the status of each service node of lvs, and later added the function of vrrp, so in addition to lvs, it can also be used as a high-availability software for other services (nginx, haproxy). VRRP is an abbreviation of virtual router redundancy protocol. The emergence of VRRP is to solve the single point of failure of static routing, it can ensure that the network can run uninterruptedly and stably. So keepalive has LVS cluster node healthcheck function on the one hand, and LVS director failover on the other hand.

---

Question6: Introduce how Nginx reverse proxy is load balanced?

Answer6：

Load balancing under Nginx reverse proxy

Ordinary load balancing software, such as LVS, only implements the forwarding and delivery of request packets. From the point of view of load balancing: point server, the received request is still a real user from the client accessing the load balancer ; The reverse proxy is different. After receiving the request from the access user, the reverse proxy server will re-initiate the node server under the request proxy and finally return the data to the client user. From the perspective of the node server, the client users of the accessed node server are reverse proxy servers, rather than the actual website access users.

upstream_module and health check

ngx_http_upstream_module is a load balancing module, which can realize the load balancing function of the website, that is, the health check of the node. The upstream module allows Nginx to define one or more groups of node server groups. When using it, it can send the website request to the pre-defined through proxy_pass Corresponds to the name of the Upstream group.

Parameters in the upstream module	Parameter Description
weight	Server weight
max_fails	When Nginx fails to connect to the backend host, this value is used in conjunction with the three parameters proxy_next_upstream, fastcgi_next_upstream, and memcached_next_upstream. When Nginx receives the status codes defined by these three parameters returned by the back-end server, it will forward this request to the normally working back-end server. Such as 404, 503, 503, max_files = 1.
fail_timeout	max_fails and fail_timeout are generally used in association. If a server fails to connect max_fails times during fail_timeout time, Nginx will think that it has hung up, so it will not request it again during fail_timeout time, fail_timeout default is 10s, max_fails default It is 1, which means that the server is hung as long as an error occurs by default. If max_fails is set to 0, it means to cancel this check.
backup	Indicates that the current server is a standby server, and only when other non-backup back-end servers are down or busy, will it allocate requests to it
down	Flag server is never available, can be used with ip_hash

upstream lvsServer{
server 191.168.1.11 weight=5 ;
server 191.168.1.22:82;
server example.com:8080 max_fails=2 fail_timeout=10s backup;
#域名的话需要解析的哦，内网记得 hosts
}

proxy_pass request forwarding

The proxy_pass directive belongs to the ngx_http_proxy_module module. This module can forward the request to another server. In the actual reverse proxy work, it will match the specified URI through the location function, and then throw the request that receives the service matching URI through proyx_pass to the definition A good upstream node pool.

location /download/ {
proxy_pass http://download/vedio/;
}
#这是前端代理节点的设置
#交给后端 upstream 为 download 的节点

proxy module parameters	Explanation
proxy_next_upstream	When to pass the request to the next upstream
proxy_limite_rate	Limit the rate at which responses are read from the back-end server
proyx_set_header	Set the HTTP request header and pass it to the back-end server node, for example, it can be achieved to allow the server node of the proxy back-end to access the client. This is the ip
client_body_buffer_size	Client request body buffer size
proxy_connect_timeout	The timeout time for the agent to connect to the back-end node server
proxy_send_timeout	Timeout time for back-end node data return
proxy_read_timeout	Set the time for Nginx to obtain information from the back-end server of the proxy, indicating that after the connection is successfully established, Nginx waits for the response time of the back-end server
proxy_buffer_size	Set buffer size
proxy_buffers	Set the number and size of buffers
proyx_busy_buffers_size	Used to set the size of proxy_buffers that can be used when the system is busy, recommended proxy_buffers * 2
proxy_temp_file_write_size	Specify the size of the proxy cache temporary file