- No matter how optimized a single server is, no matter how good hardware is used, there will always be a performance ceiling. When the performance of a single server cannot meet business needs, a high-performance cluster needs to be designed to improve the overall processing performance of the system
- The essence of high-performance clusters is very simple. Increase the overall computing power of the system by adding more servers. The calculation itself has a characteristic: the same input data and logic, no matter which server is executed, should get the same output. Therefore, the complexity of high-performance cluster design is mainly reflected in the task allocation. It is necessary to design a reasonable task allocation strategy and allocate computing tasks to multiple servers for execution.
- The complexity of high-performance clusters is mainly reflected in the need to add a task allocator and choose an appropriate task allocation algorithm for the task. For task distributors, the more popular general term is "load balancer". However, this name is somewhat misleading, which makes people subconsciously think that the purpose of task allocation is to maintain the load of each computing unit in a balanced state, but in fact task allocation does not only consider the load balance of the computing unit. Different task allocation algorithms have different goals, some are based on load considerations, some are based on performance (throughput, response time> considerations, and some are based on business considerations. Load balancing is not just for the load of the computing unit to reach a balanced state.
Load balancing classification
Common load balancing systems include three types: DNS load balancing, hardware load balancing, and software load balancing.
DNS load balancing
DNS is the simplest and most common method of load balancing, and is generally used to achieve geographic level balance. For example, users in the north visit the computer room in Beijing, and users in the south visit the computer room in Shenzhen. The essence of DNS load balancing is that DNS resolution of the same domain name can return different IP addresses. For example, it is also www.baidu.com. The address obtained is 61.135.165.224, and the address obtained by Southern users after analysis is 14.2 15.177.38
DNS load balancing
DNS load balancing is simple to implement and low cost, but it also has disadvantages such as too much granularity and few load balancing algorithms.
- Simple and low cost: The load balancing work is handled by the DNS server, no need to develop or maintain load balancing equipment by yourself
- Access nearby to improve access speed: DNS resolution can be based on the source IP of the request and resolve to the server address closest to the user, which can speed up access and improve performance
- Update is not timely: DNS cache time is relatively long. After modifying the DNS configuration, due to caching, many users will continue to access the IP before modification. Such access will fail, which will not achieve the purpose of load balancing, and will also affect The user normally uses the business
- Poor scalability: DNS load balancing is controlled by the domain name provider, and it is impossible to do more customized functions and extended features for it based on business characteristics
- The distribution strategy is relatively simple: DNS load balancing supports few algorithms: it cannot distinguish the difference of servers (the load cannot be judged according to the status of the system and services); it also cannot perceive the status of the back-end server
In view of some shortcomings of DNS load balancing, for delay and fault-sensitive services, some companies have implemented the function of HTTP-DNS themselves, that is, use the HTTP protocol to implement a private DNS system. This scheme is the opposite of the advantages and disadvantages of general DNS.
Hardware load balancing
Hardware load balancing is to achieve load balancing function through a separate hardware device. This type of device is similar to a router switch and can be understood as a basic network device for load balancing. There are currently two typical hardware load balancing devices in the industry: FS and A10. This type of equipment has strong performance and powerful functions, but the price is not cheap. Generally, only "local tyrants" companies will consider using such equipment. Ordinary business-level companies can't afford it, and the business volume is not that big, so using these equipment is a waste.
The advantages and disadvantages of hardware load balancing are as follows
- Powerful functions: Full support for load balancing at all levels, support for comprehensive load balancing algorithms, and support for global load balancing
- Powerful performance: By comparison, the software load balancing support up to 100,000 level is already very powerful, and the hardware load balancing can support more than 1 million concurrency
- High stability: Commercial hardware load balance, passed a good and rigorous test, after large-scale use, high stability
- Support security protection: In addition to load balancing functions, hardware balancing equipment also has security functions such as firewalls and DDOS attacks
- Expensive: hundreds of thousands to millions
- Poor scalability: hardware devices can be configured according to the business, but cannot be extended and customized
Software load balancing
- Software load balancing implements the load balancing function through load balancing software. Common ones are Nginx and LVS. Nginx is the 7-layer load balancing of the software, and LVS is the 4-layer load balancing of the Linux kernel. The difference between layer 4 and layer 7 lies in protocol and flexibility. Nginx supports HTTP and E-mail protocols, while LVS is a 4-layer load balancing, independent of the protocol, almost all applications can be done, such as chat, database, etc.
- The main difference between software and hardware is performance. Hardware load balancing performance is much higher than software load balancing performance. The performance of Ngxin is 10,000, and an Nginx on a general Linux server can reach 50,000 per second: the performance of LVS is 100,000, which is said to reach 800,000 per second; and the self-performance is millions, from 200 Ten thousand per second to 8 million per second, of course, the biggest advantage of software load balancing is cheap
- In addition to using an open source system for load balancing, if the business is special, it may also be customized based on the open source system (for example, Nginx plug-in), or even self-research.
Nginx's load balancing architecture
The advantages and disadvantages of software load balancing are as follows:
- Simple: Both deployment and maintenance are relatively simple
- Cheap: Just buy a Linux server and install the software
- Flexible: Layer 4 and Layer 7 load balancing can be selected according to the business; it can also be easily expanded based on the business, for example, the Nginx plug-in can be used to achieve business customization functions
- Average performance: one Nginx can support approximately 50,000 concurrency
- Not as powerful as hardware load balancing
- Generally do not have security functions such as firewall and DDOS attack prevention
Load balancing architecture
DNS load balancing, hardware load balancing, software load balancing, each method has some advantages and disadvantages, but it does not mean that in actual applications, you can only choose one or the other based on their advantages and disadvantages, but based on them. The advantages and disadvantages are used in combination. Specifically, the basic principles of the combination are: DNS load balancing is used to achieve geographic-level load balancing; hardware load balancing is used to achieve cluster-level load balancing; software load balancing is used to achieve machine-level load balancing
Load balancing algorithm
There are a large number of load balancing algorithms, and they can be customized and developed according to some business characteristics. Regardless of the differences in details, according to the expected goals of the algorithms, they can be roughly divided into the following categories:
- Task level classification: The load balancing system equally distributes the received tasks to the server for processing. The "average" here can be the average of absolute numbers, or the average of proportions or weights.
- Load balancing: The load balancing system is distributed according to the load of the server. The load well here is not necessarily the number of connections, I/O usage rate, and network card throughput in the usual sense to measure the pressure of the system.
- Optimal performance category: The load balancing system allocates tasks according to the server's response time, and prioritizes new tasks to the server with the fastest response.
- Hash type: The load balancing system performs Hash calculations based on some key information in the task, and distributes requests with the same hash value to the same server. Common source address hash, target address hash, session id hash, user id hash, etc.
polling
After the load balancing system receives the request, it is allocated to the server in turn in order. Polling is the simplest strategy, without paying attention to the status of the server itself, for example:
- A certain server currently enters an infinite loop due to a program bug, which causes a high CPU load. The load balancing system is not aware of it, or it will continue to send requests to it continuously
- There are new machines in the cluster with 32 cores, and the old machines with 16 cores. The load balancing system is not concerned. The number of tasks allocated to the new and old machines is the same.
It should be noted that the load balancing system does not need to pay attention to "the state of the server itself", the key word here is "itself". That is to say, as long as the server is running, the running status is not concerned, but if the server is directly rocked, or the server and the load balancing system are disconnected, the load balancing system is aware of it, and needs to be dealt with accordingly . For example, it is obviously unreasonable to delete a server from the list of assignable servers, otherwise it will appear that the servers are all rock machines and tasks are continuously assigned to it. All in all, "simple" is the advantage and disadvantage of the polling algorithm
Weighted polling
The load balancing system allocates tasks based on server weights. The weights here are generally statically configured based on the hardware configuration. Dynamic calculations will be more suitable for the business, but the complexity will also be higher. Weighted polling is a special form of polling, and its main purpose is to solve the problem of different server processing capabilities. For example, if there are new machines with 32 cores in the cluster and the old machines with 16 cores, then theoretically we can assume that the processing power of the new machine is twice that of the old machine, and the load balancing system can be based on a ratio of 2:1 Allocate more tasks to the new machine to make full use of the new machine’s performance
Weighted polling solves the problem that the polling algorithm cannot perform task allocation based on the difference in server configuration, but there is also the problem that task allocation cannot be performed based on the difference in server status.
Lowest load first
The load balancing system assigns tasks to the server with the lowest current load. The load here can be measured with different indicators according to different task types and business scenarios. E.g:
- LVS is a 4-layer network load balancing device that can judge the status of the server by the "number of connections". The greater the number of server connections, the greater the server pressure
- Nginx, a 7-layer network load system, can judge server status based on the "number of HTTP requests". The load balancing algorithm built in CNginx does not support this method and needs to be extended)
- If we develop our own load balancing system, we can choose indicators to measure system pressure based on business characteristics. If it is CPU-intensive, you can use "CPU load" to measure system pressure: If it is I/O-intensive, you can use "IIO load" to measure system pressure
The least-load-first algorithm solves the problem of the inability to perceive server status in the polling algorithm, and the cost of this is a lot of complexity. E.g:
- The algorithm with the least number of connections first requires the load balancing system to count the connections currently established by each server. Its application scenario is limited to any connection request received by the load balancing will be forwarded to the server for processing, otherwise if the load balancing system and the server are fixed The connection pool method is not suitable for adopting this algorithm. For example, LVS can adopt this algorithm for load balancing, while a load balancing system that connects to the MySQL cluster through the connection pool is not suitable for adopting this algorithm for load balancing.
- The algorithm with the lowest CPU load priority requires the load balancing system to collect the CPU load of each server in some way, and it is necessary to determine whether the load of 1 minute is the standard or the load of 15 minutes. If there is no one minute is definitely more than 15 Minutes are better or worse. The optimal time interval for different businesses is different. Too short a time interval may cause frequent fluctuations, and a too long time interval may cause slow response when peaks come.
The lowest load priority algorithm can basically solve the shortcomings of the polling algorithm perfectly, because after adopting this algorithm, the load balancing system needs to perceive the current running status of the server. Of course, the price is a substantial increase in complexity. In layman's terms, polling may be an algorithm that can be implemented in 5 lines of code, while the lowest-load priority algorithm may take 1,000 lines to implement, and it even requires the development of code for the load balancing system and the server. If the lowest-load priority algorithm itself is not well designed or is not suitable for the operating characteristics of the business, the algorithm itself may become a performance bottleneck or cause many inexplicable problems. So although the effect of the lowest-load priority algorithm looks very good, in fact, there are not so many polling (including weighted polling) scenarios that are actually used.
Best performance class
The lowest-load priority algorithm is allocated from the perspective of the server, while the best-performing priority algorithm is allocated from the perspective of the client. The task is assigned to the server with the fastest processing speed first. Way to achieve the fastest response to the client.
Similar to the lowest-load priority algorithm, the best-performing priority algorithm essentially perceives the status of the server, and only measures the status of the server through the external standard of response time. Therefore, the problems of the optimal performance priority algorithm are similar to those of the lowest load priority algorithm, and the complexity is high, mainly reflected in:
- The load balancing system needs to collect and analyze the response time of each task of each server. In a large number of task processing scenarios, this collection and statistics itself will consume more performance.
- In order to reduce this statistical consumption, statistics can be taken by sampling, that is, not counting the response time of all tasks, but sampling the response time of some tasks to estimate the response time of the overall task, although sampling statistics can reduce performance consumption , But it makes the complexity further increase, because to determine the appropriate sampling rate, the sample rate is too low will lead to inaccurate results, the sampling rate is too high will lead to high performance consumption, finding a suitable sample rate is also a complicated matter
- Regardless of all statistics or sampling statistics, you need to choose a suitable period: the best performance in 10 seconds, the best performance in 1 minute, or the best performance in 5 minutes...There is no one-size-fits-all cycle. It is necessary to make judgments and choices based on actual business. This is also a relatively complicated matter. Even after the system is online, it needs to be continuously tuned to achieve the optimal design.
Hash class
The load balancing system performs Hash operation according to some key information in the task, and allocates requests with the same Hash value to the same server. The purpose of this is to meet specific business requirements. E.g:
- The source address hash
assigns tasks originating from the same source IP address to the same server for processing, which is suitable for businesses that have transactions and sessions. For example, when we log in to online banking through a browser, a session information will be generated. This session is temporary and will become invalid after closing the browser. There is no need to persist the session information in the backend of online banking, just save the session temporarily on a certain server, but it is necessary to ensure that the user can access the same server every time during the existence of the session. This kind of business scenario can be used Source address Hash to achieve - ID Hash
assigns a service identified by an ID to the same server for processing. The ID here is generally the ID of temporary data (for example, session id). For example, in the online banking login example above, session id hash can also be used During the same session, the user visits the same server every time