Load Balance is one of the factors that must be considered in the design of distributed system architecture.
It usually means that the request/data is [evenly] distributed to multiple operation units for execution. The key to load balancing is [even].
The uniformity here is not the 10 ÷ 2 = 5 we imagined, which is even. no.
The common Internet distributed architecture is as above, which is divided into client layer, reverse proxy nginx layer, site layer, service layer, and data layer.
Common load balancing solutions
The common Internet distributed architecture is as above, which is divided into client layer, reverse proxy nginx layer, site layer, service layer, and data layer. It can be seen that each downstream has multiple upstream calls. As long as each upstream accesses each downstream evenly, it is possible to "evenly distribute requests/data to multiple operation units for execution".
[Client layer -> reverse proxy layer] load balancing
The load balancing from [client layer] to [reverse proxy layer] is achieved through "DNS polling": DNS-server is configured with multiple resolution IPs for a domain name, and each DNS resolution request accesses DNS-server, These IPs will be polled and returned to ensure that the resolution probability of each IP is the same. These IPs are the external network IPs of nginx, so that the request distribution of each nginx is also balanced.
[Reverse proxy layer -> site layer] load balancing
The load balancing from [reverse proxy layer] to [site layer] is achieved through "nginx". By modifying nginx.conf, various load balancing strategies can be implemented:
1) Request polling: Similar to DNS polling, requests are routed to each web-server in turn
2) Least connection routing: Which web-server has fewer connections and which web-server is routed to
3) IP hash: The web-server is routed according to the IP hash value of the visiting user. As long as the user's IP distribution is uniform, the request is theoretically uniform, and the IP hash equalization method can be achieved. The request of the same user It is fixed on the same web-server. This strategy is suitable for stateful services, such as session (58 Shen Jian Note: This can be done, but it is strongly not recommended. The statelessness of the site layer is one of the basic principles of distributed architecture design. , the session is best stored in the data layer)
4)…
[Site Layer -> Service Layer] Load Balancing
The load balancing from [site layer] to [service layer] is achieved through the "service connection pool".
The upstream connection pool will establish multiple connections with the downstream service, and each request will "randomly" select a connection to access the downstream service.
The previous article "RPC-client Implementation Details" contains detailed descriptions of load balancing, failover, and timeout processing. Please click the link to view it, and it will not be expanded here.
[Data Layer] Load Balancing
In the case of a large amount of data, since the data layer (db, cache) involves the horizontal segmentation of data, the load balancing of the data layer is more complicated. It is divided into "data balance" and "request balance". .
The balance of data refers to: the amount of data for each service (db, cache) after horizontal segmentation is similar.
The balance of requests refers to the fact that each service (db, cache) after horizontal segmentation has the same amount of requests.
There are several common horizontal segmentation methods in the industry:
1. Split according to the range level
Each data service stores a certain range of data. The above figure is an example:
user0 service, storage uid range 1-1kw
user1 service, storage uid range 1kw-2kw
The benefits of this program are:
(1) The rules are simple, the service only needs to judge the uid range to route to the corresponding storage service
(2) The data balance is better
(3) It is relatively easy to expand, you can add a uid[2kw,3kw] data service at any time
The shortcomings are:
(1) The load of the request is not necessarily balanced. Generally speaking, the newly registered users will be more active than the old users, and the service request pressure of the large range will be greater.
Second, according to the id hash level segmentation
Each data service stores part of the data hashed by a certain key value. The above figure is an example:
user0 service, storing even uid data
user1 service, storing odd uid data
The benefits of this program are:
(1) The rules are simple, the service only needs to hash the uid to route to the corresponding storage service
(2) The data balance is better
(3)请求均匀性较好
不足是：
(1)不容易扩展，扩展一个数据服务，hash方法改变时候，可能需要进行数据迁移
总结
负载均衡(Load Balance)是分布式系统架构设计中必须考虑的因素之一，它通常是指，将请求/数据【均匀】分摊到多个操作单元上执行，负载均衡的关键在于【均匀】。
(1)【客户端层】到【反向代理层】的负载均衡，是通过“DNS轮询”实现的
(2)【反向代理层】到【站点层】的负载均衡，是通过“nginx”实现的
(3)【站点层】到【服务层】的负载均衡，是通过“服务连接池”实现的
(4)【数据层】的负载均衡，要考虑“数据的均衡”与“请求的均衡”两个点，常见的方式有“按照范围水平切分”与“hash水平切分”