1 Overview
In Spring Cloud, load balancing is implemented based on the component Netflix Ribbon. The Ribbon client component provides a series of complete configurations (timeout, retry, etc.). Based on the user-configured policy (or default policy), it will access the services registered after the Load Balancer (LB for short) according to the policy to ensure that the request Distribute to different machines according to policies to achieve load balancing. This article will analyze in detail the implementation principle of Ribbon in Spring Cloud and the principle of implementing load balancing strategy.
2.Principle analysis
2.1 Load balancing concept
Load balancing refers to balancing and allocating loads (work tasks) to multiple operating units, such as FTP servers, Web servers, enterprise Core application servers and other main task servers, etc., to complete work tasks collaboratively. The above explanation comes from Baidu and is applied to the microservice architecture. That is, when a service has multiple instances, the caller can access these services evenly. Requests are made to these instances to optimize resource usage, maximize throughput, and minimize response time, while also avoiding service overload.
2.2 The importance of load balancing to microservices
In microservices, if there is no load balancing, requests may be unevenly distributed (some instances will receive a large number of requests and become paralyzed, while other instances will receive single-digit requests). As a result, some instances may go down first. In turn, requests will continue to flow to normal instances, eventually causing all instances to be paralyzed. It can be said that the load balancing strategy plays a vital role in the healthy operation of microservices. In the absence of sufficient resources, this method can effectively ensure the full application of resources and the healthy operation of services.
2.3 Ribbon usage in SpringCloud
2.3.1 Introducing dependencies
<!--添加ribbon的依赖-->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</dependency>
Most service discovery in springCloud has introduced ribbon, such as feign component, eurker-server, nacos-discovery, etc. If these packages have been introduced, there is no need to introduce the above dependencies.
2.3.1 Add @LoadBalanced annotation
@Configuration
public class RestConfig {
@Bean
@LoadBalanced
public RestTemplate restTemplate() {
return new RestTemplate();
}
The @LoadBalanced annotation is the specific implementation of ribbon load balancing. By adding this annotation, the requested service will be obtained in polling mode by default to ensure load balancing.
2.3.2 Remote calling using RestTemplate
@Service
public class OrderService {
@Autowired
private RestTemplate restTemplate;
public R findUserByUserId(Integer id) {
String url = "http://userService/findOrderByUserId/"+id;
R result = restTemplate.getForObject(url,R.class);
return result;
}
}
2.4 Ribbon implementation principle
2.4.1 Ribbon working principle
In the above case, take the order service querying user information based on userId as an example. The order service (orderService) initiates a query request: http://userService/getUserInfo. The Ribbon load balancer will intercept the http request, obtain the service list registered by userService in the registration center, and then based on the configured strategy (polling, random, haha) Hope, etc.), filter a userService from the service list to send a request and receive the returned results.
2.4.2 Ribbon load balancing strategy
Ribbon’s built-in core load balancing strategy class diagram is as follows:
The detailed explanation of each strategy is as follows:
RoundRobinRule: Polling strategy (Ribbon default strategy). If the obtained server is unavailable, or the server cannot be obtained more than 10 times, an empty server will be returned;
RandomRule: Random strategy, randomly selects a server and returns it. If the randomly obtained server is null or unavailable, it will be continuously selected in a loop;
RetryRule: Retry strategy, retry in a loop within a certain period of time. The RoundRobinRule strategy is used by default, or a specified strategy can be passed in. It will determine whether the server is empty or alive after each election, and continuously obtain the server within 500ms;
BestAvailableRule
AvailabilityFilteringRule< /span>: Available filtering strategy, which is an extension of the polling strategy. The selected server is judged for timeout, number of connections exceeding the limit, etc., and will only be returned if the conditions are met;: average response time policy, This strategy is an extension of the polling strategy. It calculates the average response time of each service. The shorter the time, the greater the weight. If there is insufficient information at the beginning, the polling strategy will be used. When the statistical information is sufficient, this strategy will be used. . WeightedResponseTimeRule: Zone trade-off strategy. This strategy is an extension of the polling strategy. It filters out servers with timeouts and large number of connections, and also filters out servers that do not meet the requirements. All nodes in the zone area are always guaranteed to poll service instances in a zone/computer room;
ZoneAvoidanceRule: Minimum number of connections strategy, traverse the serverList, select an available server with the smallest number of connections; if the server is empty, use the polling strategy to re-acquire;
2.5 Ribbon source code interpretation in Spring Cloud
The loading sequence of Ribbon in SpringCloud core code is as follows:
In the LoadBalancerAutoConfiguration configuration class, an interceptor LoadBalancerInterceptor is added to the RestTemplate class with the @LoadBalanced annotation, which intercepts After receiving the request, the getLoadBalancer() method in RibbonLoadBalancerClient will be called to obtain the available servers. This method calls the choose() method of IRule to obtain the policy and select the server according to the policy.
2.5.1 LoadBalancerInterceptor
public class LoadBalancerInterceptor implements ClientHttpRequestInterceptor {
private LoadBalancerClient loadBalancer;
private LoadBalancerRequestFactory requestFactory;
public LoadBalancerInterceptor(LoadBalancerClient loadBalancer, LoadBalancerRequestFactory requestFactory) {
this.loadBalancer = loadBalancer;
this.requestFactory = requestFactory;
}
public LoadBalancerInterceptor(LoadBalancerClient loadBalancer) {
// for backwards compatibility
this(loadBalancer, new LoadBalancerRequestFactory(loadBalancer));
}
@Override
public ClientHttpResponse intercept(final HttpRequest request, final byte[] body,
final ClientHttpRequestExecution execution) throws IOException {
//获取请求路径uri
final URI originalUri = request.getURI();
//获取服务名称
String serviceName = originalUri.getHost();
Assert.state(serviceName != null, "Request URI does not contain a valid hostname: " + originalUri);
//此处真正执行的方法是：RibbonLoadBalancerClient.execute()方法
return this.loadBalancer.execute(serviceName, requestFactory.createRequest(request, body, execution));
}
}
The core method in the above code is the intercept() method. The debug case is as follows. After obtaining the service name, it will carry the service name to query and obtain the available servers.
2.5.2 RibbonLoadBalancerClient
@Override
public <T> T execute(String serviceId, LoadBalancerRequest<T> request) throws IOException {
//获取服务列表
ILoadBalancer loadBalancer = getLoadBalancer(serviceId);
//进行负载均衡操作，获取server
Server server = getServer(loadBalancer);
if (server == null) {
throw new IllegalStateException("No instances available for " + serviceId);
}
RibbonServer ribbonServer = new RibbonServer(serviceId, server, isSecure(server,
serviceId), serverIntrospector(serviceId).getMetadata(server));
return execute(serviceId, ribbonServer, request);
}
protected Server getServer(ILoadBalancer loadBalancer) {
if (loadBalancer == null) {
return null;
}
//默认采用轮询策略获取server
return loadBalancer.chooseServer("default"); // TODO: better handling of key
}
// BaseLoadBalancer类中使用chooseServer()方法
public Server chooseServer(Object key) {
if (counter == null) {
counter = createCounter();
}
counter.increment();
if (rule == null) {
return null;
} else {
try {
//根据传入的服务id，从负载均衡器中为指定的服务选择一个服务实例
return rule.choose(key);
} catch (Exception e) {
logger.warn("LoadBalancer [{}]: Error choosing server for key {}", name, key, e);
return null;
}
}
}
//根据传入的服务id，指定的负载均衡器中的服务实例执行请求
@Override
public <T> T execute(String serviceId, ServiceInstance serviceInstance, LoadBalancerRequest<T> request) throws IOException {
Server server = null;
if(serviceInstance instanceof RibbonServer) {
server = ((RibbonServer)serviceInstance).getServer();
}
if (server == null) {
throw new IllegalStateException("No instances available for " + serviceId);
}
RibbonLoadBalancerContext context = this.clientFactory
.getLoadBalancerContext(serviceId);
RibbonStatsRecorder statsRecorder = new RibbonStatsRecorder(context, server);
try {
T returnVal = request.apply(serviceInstance);
statsRecorder.recordStats(returnVal);
return returnVal;
}
// catch IOException and rethrow so RestTemplate behaves correctly
catch (IOException ex) {
statsRecorder.recordStats(ex);
throw ex;
}
catch (Exception ex) {
statsRecorder.recordStats(ex);
ReflectionUtils.rethrowRuntimeException(ex);
}
return null;
}
2.6 Ribbon hunger loading mechanism
Ribbon uses lazy loading by default. LoadBalanceClient will be created only when accessed for the first time, and the request time will be very long.
Hungry loading will be created when the project starts, reducing the time of the first visit. Enable hungry loading through the following configuration:
ribbon:
eager-load:
enabled: true # 开启饥饿加载
clients: userservice # 指定服务名为userservice的服务饥饿加载
As can be seen from the above figure, after configuring hungry loading, LoadBalanceClient will be generated directly after the service is started.
2.7 Customized load balancing strategy
2.7.1 Bean injection specifies IRule implementation class
@Bean
public IRule rule(){
return new RandomRule();
}
2.7.1 Configuration through configuration file
userservice:
ribbon:
NFLoadBalancerRuleClassName: com.netflix.loadbalancer.RoundRobinRule # 负载均衡规则
3. Summary
1. This article introduces the principle, implementation process and source code analysis of load balancing in detail;
2. Ribbon hungry loading mechanism, which only creates LoadBalanceClient when used; 4. Load balancing policy configuration can be performed through bean injection and configuration file configuration.
3. When configuring starvation loading, specify the service name;
4. References
1.https://www.bilibili.com/video/BV1LQ4y127n4
2.https://www.cnblogs.com/WarBlog/p/15397657.html
3.https://zhuanlan.zhihu.com/p/88431514