[Spring Cloud] Ribbon implements load balancing principles, strategies and hungry loading

---

Preface

In the previous article "In-depth understanding of the principles of Eureka registration center, service registration and discovery" , we introduced how to use Eureka to realize service registration and pull, and @LoadBalancedachieved load balancing by adding annotations. There are many questions hidden behind this automation:

• When is the service pulled?
• How is load balancing implemented?
• What are the principles and strategies of load balancing?

This article aims to deeply explore the principles of using Eureka to achieve load balancing, and provide us with a clearer understanding of the internal mechanism of service calls in microservice architecture. By answering these doubts, we will better understand how service discovery and load balancing operate, and lay a solid foundation for building a high-performance, stable distributed system.

1. What is Ribbon?

Ribbon is a load balancer based on HTTP and TCP clients. In a microservice architecture, service invocation usually involves load balancing, that is, selecting one of multiple service providers for invocation. Ribbon provides a simple yet effective load balancing solution.

Ribbon was originally developed by Netflix and later became part of the Spring Cloud project. Its main function is to achieve balanced traffic distribution between service consumers and providers, ensuring that each service provider can receive appropriate requests and avoiding service overload or resource waste.

Specifically, Ribbon implements the following functions:

1. Load balancing algorithm: Ribbon supports a variety of load balancing algorithms, such as polling, random, weighted polling, etc., allowing service consumers to choose appropriate load balancing strategies based on actual scenarios.

2. Automatic discovery of service instances: Ribbon integrates with service registration centers such as Eureka to automatically obtain a list of available service instances.

3. Failover and retry mechanism: Ribbon has failover and retry functions, which can automatically switch to other healthy instances when the service provider fails, improving the stability and availability of the system.

In Spring Cloud, Ribbon, as a load balancing client component, intercepts microservice call requests and dynamically selects target service instances to distribute the requested load and achieve fine-grained control of service calls.

2. The principle of Ribbon to achieve load balancing

2.1 Load balancing process

The flow chart of Ribbon's load balancing is as follows:

Here is a detailed description of this process:

1. First, the service consumer initiates a request. After the Ribbon load balancer receives the request, it obtains the service name in the request path, for example userservice.
2. The load balancer then uses the obtained service name to pull the corresponding service from Eureka Service.
3. In actual production, a service generally has multiple instances, so what is pulled is a service list, which contains the IP and port numbers of all normal instances of this service.
4. After obtaining this list, the load balancer uses the currently adopted load balancing policy to select an appropriate service, and then accesses the service.

This process ensures that service requests can be reasonably distributed to multiple instances, thereby achieving load balancing.

2.2 Ribbon source code analysis for load balancing

First, a class used by Ribbon to implement load balancing is called a LoadBalancerInterceptorload balancing interceptor. You can view its source code through IDEA:

Found that it implements an ClientHttpRequestInterceptorinterface, the client HTTP request interceptor:

It will intercept RestTemplateHTTP requests that occur. ClientHttpRequestInterceptorIt is an interface and contains a interceptmethod. Therefore, LoadBalancerInterceptoras a class that implements this interface, it must also override interceptthe method. At this time, we can set a breakpoint in this method for debugging to track Running of the code:

1. request.getURI()Get the request address:
   

2. originalUri.getHost()The host name in the requested address is obtained. What is obtained at this time is the name of the service, that is userservice:

3. After finding the name of the service, the next thing to do is to pull the corresponding service from EurekaServer. Then this method hands the obtained service name to a (Ribbon load balancing client) for RibbonLoadBalancerClientprocessing .

4. Continue to debug the code and enter executethe method:

5. Continuing down, you will get an LoadBalancerobject:
   
   the name of this object is called "Dynamic Service List Balancer". Looking at the content of this object, you can find that the number of services in the service list is 3, and these three services are obtained Three user-serviceservices registered with EurekaServer.
   

Therefore, getLoadBalancerthe function of the method is to find the service list in EurekaServer based on the service name. After finding the service list, we can boldly guess that the next step is to perform load balancing operations.

6. At this point, we enter getServerthe method:
   

7. Next, the method is called chooseServerand enters this method:

8. Then enter chooseServerthe method, and finally find rule.choosethe method:

Check the rule object at this time and find that it is an interface:

Since it is an interface, there is an implementation class:

The implementation class discovered at this time is the load balancing rule. The general rules include randomization, polling, etc.

9. Finally, through the default rules, the service of port 8082 is selected.

Next, you can use the real IP and port number instead userservice, and then access the specified service.

The above is the source code analysis of Ribbon's implementation of load balancing. Through debugging methods, we deeply explored the implementation process of service discovery and load balancing, which helped us better understand the operation methods of service discovery and load balancing.

3. Ribbon load balancing strategy

3.1 Load balancing strategy

Through the above source code analysis, it is not difficult to find that Ribbon's load balancing rules are defined by an interface called IRule, and each sub-interface is a rule.

The inheritance system of the IRule interface is shown in the figure below:

The load balancing strategy of Ribbon can be summarized as follows:

Built-in load balancing rule class	Rule description
RoundRobinRule	Simply poll the service list to select a server. It is Ribbon's default load balancing rule.
AvailabilityFilteringRule	Ignore servers with short circuits and high concurrency
WeightedResponseTimeRule	Give each server a weight value. The longer the server response time, the smaller the weight of the server. This rule will randomly select servers, and this weight value will affect server selection.
ZoneAvoidanceRule	Server selection is based on servers available in the region. Use Zone to classify servers. This Zone can be understood as a computer room, a rack, etc. Then poll multiple services in the Zone.
BestAvailableRule	Ignore those servers that are short-circuited and choose servers with lower concurrency numbers.
RandomRule	Randomly select an available server
RetryRule	Selection logic of retry mechanism

Ribbon provides these built-in load balancing rules and also supports custom load balancing rules. In practical applications, it is very important to choose an appropriate load balancing strategy based on business characteristics. The following demonstrates changes to the Ribbon load balancing policy.

3.2 Demonstrate changes to Ribbon load balancing strategy

Load balancing rules can be modified by defining IRule implementations in two ways:

1. Code method: In order-servicethe startup OrderApplicationclass, define a new one IRuleand @Beanregister it in the Spring container using annotations:

@Bean
public IRule randomRule(){
    
    
    return new RandomRule();
}

2. Configuration file method: In order-servicethe application.ymlfile, adding new configuration can also modify the rules:

# 修改 Ribbon 负载均衡策略
userservice:
  ribbon:
    NFLoadBalancerRuleClassName: com.netflix.loadbalancer.RandomRule # 负载均衡规则

For example, the following is the result of using the access order again after modifying the load balancing policy order-service. It can be found that the service is no longer selected in a polling manner user-service, but in a random manner.

4. Ribbon’s Hungry Loading

4.1 View Ribbon’s lazy loading

When we restart order-servicethe service and then access the order in the browser, we can find the following phenomenon:

When order-servicethe service is started, it can be found that it takes more than 300 milliseconds to access the service for the first time:

Then, after visiting it multiple times, you can find that the time consumption has become more than ten millimeters:


From the above phenomenon, you can find that Ribbon adopts the lazy loading mode by default, just like the lazy mode of the singleton mode. LoadBalanceClientInstances will be created only when required , and the request time will be very long.

4.2 Ribbon’s hungry loading mode

In order to solve the above-mentioned time-consuming problem of lazy loading, Ribbon also provides a hungry loading mode. Hungry loading will be created when the project is started, reducing the time of the first visit.

Enable hungry loading through the following configuration:

Restart order-servicethe service at this time:

When you start the service, you will find that there are more logs:

the content of this log is LoadBalanceClientthe log generated by loading the instance.

When you access order-servicethe service for the first time again, you will find that the time consumed has become shorter: