[System architecture] What is load balancing

Load balancing is a key component of a highly available network infrastructure. With load balancing, we can usually deploy multiple application servers, and then distribute user requests to different servers through load balancing to improve the website, The performance and reliability of applications, databases or other services.

Why introduce load balancing

First look at a web architecture without load balancing mechanism:

What are the flaws in the architecture in the figure above? First of all, the user is directly connected to the web server through the network. Imagine if the server is down (this situation may happen at any time), then the user's request will not be responded to, and the website will not be able to be accessed. This is The well-known single point of failure problem, this is definitely not enough. Generally speaking, the reliability of commercial websites needs to reach at least 4 9s, that is, 99.99& above.

Secondly, even if the server is working normally, if many users access the server at the same time and the processing capacity of the server is exceeded, the response speed will be slow or even unable to connect, which is also unacceptable for users.

The emergence of load balancing can solve the above two problems well. By introducing a load balancer and at least two web servers, the above two problems can be effectively solved. Note: Under normal circumstances, all back-end servers will guarantee to provide the same content so that users can receive consistent content no matter which server responds.

As shown in the above architecture, now, even if App 01 is hung up, load balancing will forward the user's request to App 02, which is working normally. This solves the first problem above; secondly, according to business needs, load balance the back-end App It can be easily extended, so that the second problem above can be solved. However, now the single point of failure problem is transferred to the load balancer, which can be alleviated by introducing a second load balancer, which will be discussed later.

How does load balancer choose the backend server to forward

The load balancer generally decides which server to forward the request to based on two factors.
1: Ensure that the selected back-end server is working properly and can respond to user requests;
2: Select from the healthy server pool according to the preset load balancing algorithm.
Since the load balancer should only select back-end servers that can respond normally, it needs a mechanism to determine whether the back-end servers it is connected to are working properly. In order to monitor the running status of the back-end server, the running status check service will periodically try to use the protocol and port defined by the forwarding rule to connect to the back-end server. If a server fails the health check, it will be removed from the health pool to ensure that traffic will not be forwarded to the server until it passes the health check again.

Load balancing algorithm

The load balancing algorithm determines which healthy servers on the back end will be selected. The following are a few commonly used algorithms. Here are just a brief introduction. I will not study the algorithm implementation in detail. I will use an article to summarize it later:
Polling: Select the first backend server in the health pool for the first request , And then select sequentially, until the last one, and then loop.

Minimal connection: The backend server with the least number of connections is preferred, that is, the backend server with the least pressure. This method can be considered in the case of a long session.

Hash: Select the server to be forwarded according to the hash of the IP of the request source. This way can ensure to a certain extent that specific users can connect to the same server. If your application needs to handle state and requires users to be able to connect to the same server as before, consider this approach.
Finally, if you want to solve the single point of failure of the load balancer, you can connect the second load balancer to the first to form a cluster. As shown in the following figure:

When the main load balancer fails, user requests need to be forwarded to the second load balancer. Since DNS changes usually take a long time to take effect, there is a need for a flexible solution to IP address remapping, such as floating IP (floating IP). In this way, the domain name can remain associated with the same IP, and the IP itself can be moved between servers. The following is a dynamic diagram of a load balancing architecture using floating IP:

Recommended reading:

Carefully organized |
From theory to practice, the article catalog in the second half of 2017 has a comprehensive understanding
of the lifetime of DNS (practice) storing data packets (part 2)
understanding and using Docker (part 2)
comparison of various string hash functions

Focus on server background technology stack knowledge summary sharing

Welcome to pay attention to communication and common progress

Coding

The code farmer has the right way to provide you with easy-to-understand technical articles to make technology easier!