LVS (Load Balancing)
LVS is the abbreviation of linux virtual server. It is a virtual server cluster system, which can realize the load balancing cluster function under the unix/linux platform. The project was organized and established by Dr. Zhang Wensong in May 1998.
LVS consists of 2 parts of programs, including ipvs and ipvsadm.
- ipvs (ip virtual server): A piece of code that works in the kernel space, called ipvs, is the code that actually takes effect to implement scheduling.
- ipvsadm: The other section is working in user space, called ipvsadm, responsible for writing rules for the ipvs kernel framework, defining who is the cluster service and who is the real server (Real
Server)
LVS related terms:
- DS: Director Server. Refers to the front-end load balancer node.
- RS: Real Server. The real working server at the back end.
- VIP: Request from the outside directly to the user as the target IP address of the user's request.
- DIP: Director Server IP, the IP address mainly used to communicate with internal hosts.
- RIP: Real Server IP, the IP address of the back-end server.
- CIP: Client IP, visit the IP address of the client
Brief description of LVS architecture and working principle
The LVS cluster load balancer accepts requests from all ingress clients of the service, and then decides which cluster node to handle the client's request according to the scheduling algorithm.
The LVS virtual server system is shown in the figure below. A group of servers are connected to each other through a high-speed local area network or a geographically distributed wide area network. There is a load balancer before this group of servers. The load scheduler is responsible for scheduling the client's request to the real server. In this way, the structure of this group of server clusters is transparent to users. Customers accessing the cluster system are just like accessing a high-performance, highly available server. The client program is not affected by the server cluster and does not make any changes.
Explain: The
client sends a request to the load balancing server. The load balancer accepts the client's request, and then first decides which node server to send the request to according to the LVS scheduling algorithm (10 types). Then, according to your own working mode (3 types), see how to send these client requests to the node server, and how the node server should send the response packet back to the client.
Three working modes are introduced:
NAT mode-Network address translation Virtualserver via Network address translation (VS/NAT)
This is achieved through the method of network address translation. First, when the scheduler (LB) receives the client's request packet (the destination IP of the request is VIP), it decides which back-end real server (RS) to send the request to according to the scheduling algorithm. Then the scheduling changes the target IP address and port of the request packet sent by the client to the IP address (RIP) of the back-end real server, so that the real server (RS) can receive the client's request packet. After the real server responds to the request, check the default route (in NAT mode, we need to set the default route of RS to the LB server.) Send the response data packet to LB, and LB receives the response packet and then sends the source address of the packet Change it to a virtual address (VIP) and send it back to the client.
Detailed diagram of the IP packet of the scheduling process:
Brief description of the schematic diagram:
- The client requests data, the target IP is VIP
- The request data arrives at the LB server, and the LB modifies the destination address to the RIP address and corresponding port according to the scheduling algorithm (this RIP address is obtained according to the scheduling algorithm.) and records the connection in the connection HASH table.
- The data packet arrives from the LB server to the RS server webserver, and then the webserver responds. The gateway of the webserver must be LB, and then the data is returned to the LB server.
- After receiving the returned data from RS, modify the source address VIP & destination address CIP according to the connection HASH table, and the corresponding port 80. Then the data will depart from LB and arrive at the client.
- The client can only see VIP\DIP information.
Advantages and disadvantages of NAT mode:
- NAT technology needs to rewrite the address of the request message and the response message through LB. Therefore, when the website traffic is relatively large, the LB load balancing scheduler has a relatively large bottleneck, and generally requires only 10-20 nodes at most.
- You only need to configure a public IP address on the LB.
- The gateway address of each internal node server must be the intranet address of the dispatcher LB.
- NAT mode supports translation of IP address and port. That is, the port requested by the user and the port of the real server can be inconsistent.
TUN mode-tunnel virtual server via ip tunneling mode
When the NAT mode is adopted, since the request and response messages must be rewritten by the scheduler address, when more and more client requests, the processing capacity of the scheduler will become a bottleneck. To solve this problem, the scheduler forwards the requested message to the real server through the IP tunnel. The real server directly returns the processed data to the client. In this way, the scheduler only processes the request inbound message. Since the general network service response data is much larger than the request message, the maximum throughput of the cluster system can be increased by 10 times after the VS/TUN mode is adopted.
The working flow chart of VS/TUN is shown below. The difference between VS/TUN and NAT mode is that the transmission between LB and RS does not need to rewrite the IP address. Instead, the client request packet is encapsulated in an IP tunnel, and then sent to the RS node server. After receiving it, the node server unlocks the IP tunnel and performs response processing. And directly send the packet to the client through its own external network address without going through the LB server.
Tunnel principle flow chart:
Brief description of the schematic process:
- The client requests a data packet, and the destination address VIP is sent to the LB.
- The LB receives the client request packet and performs IP Tunnel encapsulation. That is, add the IP Tunnel header to the original header. Then send it out.
- The RS node server receives the request packet according to the IP Tunnel header information (this is another logical invisible tunnel, only understood between LB and RS), and then unpacks the IP
Tunnel header information to get the client's request packet and respond deal with. - After the response is processed, the RS server uses its own line out of the public network to send the response packet to the client. The source IP address is still the VIP address. (RS node server needs to configure VIP on the local loopback interface)
DR mode-direct routing mode Virtual server via direct routing (vs/dr)
In DR mode, the request is sent to the real server by rewriting the target MAC address of the request message, and the processing result after the real server responds is directly returned to the client user. Like TUN mode, DR mode can greatly improve the scalability of the cluster system. Moreover, the DR mode does not have the overhead of an IP tunnel, and there is no need for the real servers in the cluster to support the requirements of the IP tunnel protocol. But it is required that the scheduler LB and the real server RS have a network card connected to the same physical network segment, and they must be in the same LAN environment.
The DR mode is a mode that is more frequently used on the Internet.
Schematic diagram of DR mode:
Brief description of the principle and process of DR mode:
The working flowchart of VS/DR mode is shown in the figure. Its connection scheduling and management are the same as those in NAT and TUN, and its message forwarding method is different from the previous two. The DR mode routes the message directly to the target real server. In the DR mode, the scheduler dynamically selects a server according to the load situation of each real server, the number of connections, etc., does not modify the target IP address and target port, and does not encapsulate the IP message, but changes the request message The target MAC address of the data frame is changed to the MAC address of the real server. Then send the modified data frame on the LAN of the server group. Because the MAC address of the data frame is the MAC address of the real server, and it is in the same LAN. Then according to the communication principle of the local area network, the real server must be able to receive the data packet sent by the LB. When the real server receives the request packet, the target IP after unpacking the IP header is VIP. (At this time, only your own IP can be received if it matches the target IP, so we need to configure VIP on the local loopback interface. In addition: because the network interface will respond ARP broadcast, but other machines in the cluster have this VIP lo interface , The response will conflict. So we need to turn off the ARP response of the lo interface of the real server.) Then the real server makes a request response, and then sends the response packet back to the client according to its own routing information, and the source IP The address is still VIP.
DR mode summary:
- The forwarding is realized by modifying the destination MAC address of the data packet on the scheduler LB. Note that the source address is still the CIP, and the destination address is still the VIP address.
- The requested message goes through the scheduler, and the message processed by the RS response does not need to go through the scheduler LB, so the use efficiency is very high when the amount of concurrent access is large (compared with NAT mode)
- Because the DR mode realizes forwarding through the MAC address rewriting mechanism, all RS nodes and scheduler LB can only be in one LAN
- The RS host needs to bind the VIP address to the LO interface, and needs to configure ARP suppression.
- The default gateway of the RS node does not need to be configured as LB, but directly configured as the gateway of the upper-level routing, so that the RS can directly go out of the network.
- Since the scheduler in the DR mode only rewrites the MAC address, the scheduler LB cannot rewrite the target port, so the RS server must use the same port as the VIP to provide services.
Comparison summary table of three official load technologies
Production environment selection of LVS scheduling algorithm:
- General network services, such as http, mail, mysql and other commonly used LVS scheduling algorithms are: a. Basic round-robin scheduling rr b. Weighted least connection scheduling wlc
c. Weighted round-robin scheduling wrc - The least connection lblc based on locality and the least connection given locality lblcr with replication are mainly suitable for web cache and DB cache
- The source address hash scheduling SH and the destination address hash scheduling DH can be used in combination in a firewall cluster to ensure that the entrance and exit of the entire system are unique.
In actual application, these algorithms have a wide range of application. It is best to refer to the implementation principle of the connection scheduling algorithm in the kernel, and then select a reasonable type according to specific business requirements.