Don’t understand high-performance load balancing design? Architects take you flying
In the architectural design of software systems, the load balancing design of the cluster is an indispensable solution in the optimization of high-performance systems. Load balancing is essentially used to balance and decompress user traffic, so its importance is self-evident in large-traffic projects on the Internet.
1. What is load balancing?
In early Internet applications, due to relatively small user traffic and relatively simple business logic, a single server could often meet the load demand. As the traffic of the Internet is getting larger and larger, the number of visits to a slightly better system is very large, and the system functions are becoming more and more complex. Therefore, no matter how well the performance of a single server is optimized, it cannot support such a large amount of traffic. When the access pressure from the number of users increases, it is necessary to use multiple machines and design a high-performance cluster to cope with it.
So, how do multiple servers balance traffic and form a high-performance cluster?
At this time, you need to bring out the "load balancer".
Load Balancer refers to evenly distributing user access traffic to multiple back-end servers through a "load balancer" according to a certain forwarding strategy. The back-end servers can independently respond to and process requests. This achieves the effect of spreading the load. Load balancing technology improves the service capabilities of the system and enhances the availability of applications.
(You can understand it according to the picture, the picture comes from the Internet)
2. How many load balancing solutions are there?
There are three main types of load balancing technology solutions currently on the market:
- Based on DNS load balancing
- Hardware-based load balancing
- Software-based load balancing
Each of the three solutions has its own advantages and disadvantages. DNS load balancing can achieve regional traffic balancing. Hardware load balancing is mainly used to meet load requirements in large server clusters, while software load balancing is mostly based on machine-level traffic balancing. In actual scenarios, these three types can be combined and used together. Let’s talk about it in detail:
Based on DNS load balancing
(Network picture)
Load balancing based on DNS is actually the simplest implementation solution, which can be achieved by making a simple configuration on the DNS server.
The principle is that when a user accesses a domain name, it will first resolve the IP address corresponding to the domain name to the DNS server. At this time, we can let the DNS server return different IPs according to users in different geographical locations. For example, users from the south will return the IP of our business server in Guangzhou. If users from the north come to visit, I will return the IP of the business server in Beijing.
In this mode, users are equivalent to realizing the offloading of requests according to the "proximity principle", which not only reduces the load pressure of a single cluster, but also improves the user's access speed.
The natural advantage of using DNS as a load balancing solution is that it is simple to configure, has very low implementation costs, and requires no additional development and maintenance work.
But there is also an obvious disadvantage: when the configuration is modified, it does not take effect in time. This is caused by the characteristics of DNS. DNS generally has multi-level cache, so when we modify the DNS configuration, due to caching, the IP change will not be timely, thus affecting the load balancing effect.
In addition, when using DNS for load balancing, most of them are based on region or simply do IP polling directly. There is no more advanced routing strategy, so this is also a limitation of the DNS solution.
Hardware-based load balancing
(Network picture)
Hardware load balancing is quite powerful, such as the famous F5 Network Big-IP, which is what we often call F5. It is a network device. You can simply understand it as something similar to a network switch. , it resists pressure entirely through hardware, and its performance is very good. The number of requests it can handle per second reaches millions, that is, a load of several million/second. Of course, the price is also very, very expensive, ranging from hundreds of thousands to hundreds of thousands. There are 10,000 yuan.
Because this type of equipment is generally used at the front end of the traffic portals of large Internet companies, and is used by governments, state-owned enterprises and other companies that are not short of money. Ordinary small and medium-sized companies are reluctant to use it.
If you use hardware such as F5 for load balancing, the main purpose is to save worry and trouble. Just buy one and it will be done. It has powerful performance and can handle general business. It also supports many flexible strategies in terms of load balancing algorithms, and also has some security functions such as firewalls. But the shortcomings are also obvious, one word: expensive.
Software-based load balancing
(Network picture)
Software load balancing refers to the use of software to distribute and balance traffic. Software load balancing is divided into layer 7 protocols and layer 4 protocols.
There are seven layers of network protocols. The solution based on the fourth transport layer for traffic distribution is called layer 4 load balancing, such as LVS, while the solution based on the seventh layer application layer for traffic distribution is called layer 7 load balancing, such as Nginx. There are some differences between the two in terms of performance and flexibility.
The load balancing performance based on layer 4 is higher and can generally reach hundreds of thousands/second of processing capacity, while the processing capacity of load balancing based on layer 7 is generally only tens of thousands/second.
The characteristics of software-based load balancing are also obvious and cheap. It can be deployed on a normal server. No additional purchases are required. Just invest in a little technology to optimize and optimize. Therefore, this method is the most commonly used method among Internet companies.
3. What are the commonly used equalization algorithms?
The common load balancing technical solutions have been discussed above, so let’s take a look at what balancing algorithms can generally be used in actual solution applications?
Polling strategy
Load strategy
Response strategy
Hash strategy
Let’s introduce the characteristics of these balancing algorithms/strategies:
Polling strategy
The polling strategy is actually easy to understand. When a user request comes, the "load balancer" forwards the request to different back-end business servers in turn. This strategy is often used in DNS solutions. There is no need to pay attention to the status of the back-end service. As long as there are requests, they will be forwarded to the back-end in turn. It is very simple and practical.
In practical applications, there are many ways to poll, including sequential polling, random polling, and weight-based polling. The first two are easier to understand, and the third is to poll according to weight, which means to set a weight value for each back-end service. For example, a server with high performance will have a higher weight, and a server with low performance will have a lower weight. In this way If set, when allocating traffic, giving more traffic to those with higher weights can fully utilize the performance of the back-end machine.
Load policy
The load policy means that when the "load balancer" forwards traffic to the backend, it will first evaluate the load pressure of each backend server, and forward fewer requests to backend servers with greater pressure. , for backend servers with less pressure, more requests can be forwarded to it.
This method fully combines the running status of the back-end server to dynamically allocate traffic, which is more scientific than the polling method.
However, this method also brings some disadvantages, because it needs to dynamically evaluate the load pressure of the back-end server. In addition to forwarding requests, this "load balancer" also has to do a lot of additional work, such as collecting the number of connections and the number of requests. , CPU load indicators, IO load indicators, etc., by calculating and comparing these indicators, we can determine which back-end server has greater load pressure.
Therefore, while this method brings effect advantages, it also increases the implementation difficulty and maintenance cost of the "load balancer".
Response strategy
The response strategy means that when a user makes a request, the "load balancer" will give priority to forwarding the request to the backend server with the fastest response at the current moment.
In other words, no matter whether the load of the back-end server is high or not, and no matter how the configuration is, as long as it is felt that the server can respond to the user's request the fastest at the current moment, then the request will be forwarded to it first. In this way, for the user , and the experience is the best.
So how does the "load balancer" know which backend service has the best response ability at the current moment?
This requires the "load balancer" to continuously count the processing speed of each back-end server's request, for example, once a minute to generate a ranking of the processing speed of the back-end server. Then the "load balancer" forwards services based on this ranking.
Then the problem here is the cost of statistics. Constantly doing these statistical calculations will consume some performance, and it will also increase the implementation difficulty and maintenance cost of the "load balancer".
Hash strategy
Hash strategy is also easy to understand. It is to hash a certain information in the request, and then take the modulo according to the number of back-end servers to get a value. Requests with the same value will be forwarded to the same back-end server. .
A common usage is to implement this policy on the user's IP or ID, and then the "load balancer" can ensure that the same IP source or the same user will always be sent to the same back-end server. It is generally used to handle caching, It is especially useful for functions such as conversations.
The above are the common technical solutions and strategies to achieve high-performance load balancing. Everyone is welcome to discuss them.
Original link: https://blog.csdn.net/jsjwk/article/details/82466989