Internet Architecture -- Load Balancing
What is load balancing
Load Balance (Load Balance) is one of the factors that must be considered in the design of distributed system architecture. It usually refers to the [even] distribution of requests/data to multiple operation units for execution. The key to load balance is [even].
Common load balancing solutions
Common Internet distributed architectures are divided into: client layer, reverse proxy nginx layer, site layer, service layer, and data layer. It can be seen that each downstream has multiple upstream calls. As long as each upstream accesses each downstream evenly, it is possible to "evenly distribute requests/data to multiple operation units for execution".
[Client layer -> reverse proxy layer] load balancing
The load balancing of this layer is achieved through "DNS polling": DNS-server is configured with multiple resolution IPs for a domain name. Each time a DNS resolution request is made to access DNS-server, these IPs will be polled and returned to ensure that each IP The analytical probabilities are the same. These IPs are the external network IPs of nginx, so that the request distribution of each nginx is also balanced.
[Reverse proxy layer -> site layer] load balancing
The load balancing of this layer is achieved through "nginx". By modifying nginx.conf, various load balancing strategies can be implemented:
- Request polling: Similar to DNS polling, requests are routed to each web-server in turn
- Least connection routing: Which web-server has fewer connections and which web-server is routed to
- IP hash: route web-server according to the IP hash value of the visiting user. As long as the user's IP distribution is uniform, the request is theoretically uniform. To the same web-server, this strategy is suitable for stateful services, such as session (it can be done, but it is strongly not recommended. The statelessness of the site layer is one of the basic principles of distributed architecture design, and the session is best placed in the data tier storage).
[Client layer -> site layer] load balancing
In fact, from the client layer to the site layer, you can use cloud platforms (AWS, Alibaba Cloud, Tencent Cloud) for load balancing without nginx and DNS polling, which can also solve the problem. Moreover, it is simpler and more convenient to maintain, and there are also traffic reports, and it is more convenient to reduce and increase services. Then use nginx as the reverse proxy layer to access the real web server through the Nginx server.
[Site Layer -> Service Layer] Load Balancing
The load balancing of the service layer is realized through the "service connection pool". The upstream connection pool will establish multiple connections with the downstream service, and each request will "randomly" select a connection to access the downstream service.
[Data Layer] Load Balancing
In the case of a large amount of data, since the data layer (db, cache) involves the horizontal segmentation of data, the load balancing of the data layer is more complicated. It is divided into "data balance" and "request balance". .
- Balance of data: For each service (db, cache) after horizontal segmentation, the amount of data is similar. Workaround: Split horizontally by range
- Balance of requests: For each service (db, cache) after horizontal segmentation, the request volume is similar. Solution: split by hash level
Split horizontally by range
Each data service stores a certain range of data. The above figure is an example:
- user0 library, storage uid range 1-1kw
- user1 library, storage uid range 1kw-2kw
advantage
- The rules are simple, the service can route to the corresponding storage service only by judging the uid range;
- The data is well balanced;
- It is relatively easy to expand, you can add a uid[2kw,3kw] data service at any time;
shortcoming
- The load of the request is not necessarily balanced. Generally speaking, the newly registered users will be more active than the old users, and the service request pressure of the large range will be greater;
Split by hash level
Each database stores part of the data hashed by a certain key value. The above figure is an example:
- user0 library, storing even uid data
- user1 library, storing odd uid data
advantage
- The rules are simple, the service only needs to hash the uid to route to the corresponding storage service;
- The data is well balanced;
- The request uniformity is better;
shortcoming
- It is not easy to expand, expand a data service, when the hash method is changed, data migration may be required;
Summarize
负载均衡(Load Balance)是分布式系统架构设计中必须考虑的因素之一，它通常是指，将请求/数据【均匀】分摊到多个操作单元上执行，负载均衡的关键在于【均匀】。
- 【客户端层】到【反向代理层】的负载均衡，是通过“DNS轮询”实现的
- 【反向代理层】到【站点层】的负载均衡，是通过“nginx”实现的
- 【站点层】到【服务层】的负载均衡，是通过“服务连接池”实现的
- 【数据层】的负载均衡，要考虑“数据的均衡”与“请求的均衡”两个点，常见的方式有“按照范围水平切分”与“hash水平切分”