About Ribbon, this article is enough

Today we look at a very important component in microservices: Ribbon. It plays a very important role in a distributed network as a load balancer. Before introducing Ribbon, I have to talk about the relatively remote term load balancing. Why is it said that it is remote, because in the interview, the most talked about is message queue and cache to improve system performance and support high concurrency. Few people will ask about load balancing. The reason is that the selection and construction of load balancing components are general It's all done by the operation and maintenance team or architects, and developers really rarely come into contact with it. But it doesn't matter, we not only have CRUD, but also architectural thinking.

To put it simply, load balancing is to distribute network traffic (load) to different network servers (it can be distributed evenly or unevenly), and the system can realize horizontal horizontal expansion of services. So if you were asked to design a load balancing component, how would you design it? We need to consider these factors:

• How to obtain and synchronize the server list? Involves interacting with the registry.
• How to distribute the load? Involved in allocation strategy.
• How to intercept the client request and then choose the server to forward? Involved in request interception.

With these questions in mind, let's learn how to design a load balancer from the principle of load balancing + Ribbon architecture, which I believe will bring you some inspiration.

1. Load balancing

1.1 Concept

Two basic points of load balancing:

• Which server to choose to handle client requests;
• Forward client requests.

A core principle: maintain a list of service lists through hardware or software. When a user sends a request, the request will be sent to the load balancer, and then the address of a server will be selected from the available service list according to the load balancing algorithm, and the request will be forwarded to complete the load function.

1.2 Characteristics of load balancing

Features of load balancing

• High performance : the traffic can be allocated automatically according to different allocation rules;
• Scalability : It is easy to increase the number of devices or links in the cluster;
• High reliability : A failure of a certain device or link in the system will not cause service interruption;
• Easy configurability : easy configuration and maintenance;
• Transparency : Users do not perceive how to perform load balancing, and do not need to care about load balancing.

1.3 Classification of load balancing

Load balancing technology can be classified according to software or hardware, and can also be divided into server-side load and client-side load balancing according to the location where the server list is stored.

1.3.1 Hardware load balancing

F5 is a common hardware load balancing product.

• Advantages: Stable performance, with many functions that software load balancing does not have, such as application switching, session switching, status monitoring, etc.
• Disadvantages: The equipment is expensive, the configuration is redundant, and it is not as flexible as the software load balancing, so it cannot meet the customization requirements.

1.3.2 Software load balancing

Nginx: Good performance, can load more than 1W. Working on the 7th layer of the network, it can do some shunting strategies for HTTP applications. Nginx can also be used as a static web page and image server. Nginx can only support HTTP, HTTPS and Email protocols.

LVS (Linux Virtual Server): It is a virtual server cluster system that uses IP address balancing technology and content request distribution technology to achieve load balancing. It is close to the network throughput and connection load capacity of hardware devices. Strong load resistance, it works on the network layer 4 and is only used for distribution. It has a complete dual-system hot backup solution, such as LVS+Keepalived. The software itself does not support regular expression processing, and cannot do dynamic and static separation.

1.3.3 Server load balancing

Both Nginx and F5 can be divided into server-side load balancing, and the back-end server address list is stored in the back-end server or stored on a dedicated Nginx server or F5. The source of the server's address list is through the registration center or through manual configuration.

1.3.4 Client load balancing

Finally it's the Ribbon's turn to come on stage. It belongs to the client load balancer, and the client maintains a server address list by itself. This maintenance work is done by Ribbon. Ribbon will read the list of service information from Eureka Server and store it in Ribbon. If the server is down, Ribbon will remove the down server information from the list. Ribbon has a variety of load balancing algorithms, and we can set our own rules to request to the specified server.

2. Balance strategy

Various load balancing classifications have been introduced above. Next, let's see how these load balancers use load balancing strategies to select servers to process client requests. Common balancing strategies are as follows.

2.1 Round Robin

Principle: If the servers are numbered from 0 to N, the round-robin balancing strategy will select a server sequentially from 0 as the server to process this request.

Scenario: It is suitable for all fathers to have the same software and hardware configuration, and the request frequency is relatively balanced.

2.2 Weighted Round Robin

Principle: According to the different processing capabilities of the server, assign different weights to the server, and then the request will be assigned to different servers according to the weight.

Scenario: The performance of servers is different, making full use of high-performance servers while taking care of low-performance servers.

2.3 Random Equilibrium (Random)

Principle: Randomly assign requests to different servers.

Scenario: Suitable for scenarios where the frequency of client requests is relatively random.

2.4 Response Speed ​​Balance (Response Time)

Principle: The load balancing device sends a probe request to each server to see which server responds faster.

Scenario: Suitable for scenarios where the response performance of the server is constantly changing.

Note: The response speed is between the load balancing device and the server.

3. Ribbon core components

Next is our highlight, let's take a look at Ribbon, a load balancing component in Spring Cloud. Ribbon mainly has five functional components: ServerList, Rule, Ping, ServerListFilter, ServerListUpdater.

3.1 Load Balancer LoadBalancer

Components for managing load balancing. It is created by loading the YMAL configuration file during initialization.

3.2 Service List ServerList

ServerList is mainly used to obtain address information of all services and store them locally. According to different methods of obtaining service information, it is divided into static storage and dynamic storage.

• Static storage : Obtain the service node list from the configuration file and store it locally;
• Dynamic storage : Obtain the service node list from the registration center and store it locally.

3.3 Service list filtering ServerListFilter

Filter the obtained service list according to the filtering rules.

• Filter service instances through Eureka's partition rules;
• Compare the number of communication failures and concurrent connections of service instances to eliminate unhealthy instances;
• Filter out service instances in the same region based on the region they belong to.

3.4 Service list update ServerListUpdater

Service list update means that Ribbon will obtain the latest registry information from the registry center. is an update operation defined by the interface ServerListUpdater. And it has two implementation classes, that is, there are two update methods:

• Update by scheduled task. It is done by this implementation class PollingServerListUpdater;
• Utilize Eureka's event listeners to update. It is done by this implementation class EurekaNotificationServerListUpdater.

3.5 Heartbeat detection Ping

The IPing interface class is used to detect which services are available. If not available, remove these services. The implementation classes mainly include these: PingUrl, PingConstant, NoOpPing, DummyPing, NIWSDiscoveryPing. Heartbeat detection strategy object IPingStrategy, the default implementation is polling detection.

3.6 Load Balancing Policy Rule

Ribbon's load balancing strategy is partly the same as the load balancing strategy mentioned before. Let's first look at the UML class diagram of the balancing strategy in the Ribbon source code.

As can be seen from the figure, there are mainly the following balancing strategies:

• Linear round-robin balancing (RoundRobinRule) : Request different servers in turn. The advantage is that there is no need to record the status of all current connections, and there is no state scheduling;
• Available service filtering load balancing (AvailabilityFilteringRule) : Filter services that are in circuit breaker state due to multiple access failures, and filter services whose number of concurrent connections exceeds the threshold, and then access the remaining service list according to the polling strategy. By default, a service instance is considered broken if the last three connections have failed. Then keep it for 30s and then enter the closed state of the loop. If the connection still fails at this time, the waiting time to enter the closed state will increase exponentially with the increase in the number of failures;
• Weighted Response Time Load Balancing (WeightedResponseTimeRule) : Automatically assign weights to each service according to the response time. The longer the response time, the lower the weight and the lower the probability of being selected;
• Zone-aware load balancing (ZoneAvoidanceRule) : It is more inclined to select services in the hosting area where the calling service is located, reducing latency and saving costs. The default strategy in Spring Cloud Ribbon;
• Retry load balancing (RetryRule) : Select a server through the round-robin balancing strategy. If the request fails or the response times out, you can choose to retry the current service node or choose other nodes;
• High Availability Balance (BestAvailableRule) : Ignore servers that fail requests, and try to find servers with lower concurrency. Note: This will double the pressure on the server cluster;
• Random load balancing (RandomRule) : Randomly select servers. It is suitable for scenarios with relatively large concurrency.

4. The principle of Ribbon intercepting requests

This article initially raised a question: How does the load balancer intercept client requests and then select a server for forwarding? Combined with the Ribbon core components introduced above, we can draw a schematic diagram to sort out the principle of Ribbon interception requests:

• Step 1: Ribbon intercepts all RestTemplates annotated with @loadBalance. RestTemplate is used to send HTTP requests.
• Step 2: Add Ribbon's default interceptor, LoadBalancerInterceptor, to the execution logic of RestTemplate. Every time RestTemplate sends an HTTP request, it will be intercepted by Ribbon.
• Step 3: After interception, Ribbon will create an instance of ILoadBalancer.
• Step 4: The ILoadBalancer instance will use RibbonClientConfiguration to complete the automatic configuration. It will configure IRule, IPing, ServerList.
• Step 5: Ribbon will select a service from the service list and forward the request to this service.

5. The principle of Ribbon initialization

When we analyze the Ribbon source code, we need to find a breakthrough, and the @LoadBalanced annotation is a better entry. First come a flow chart of Ribbon initialization:

The code for adding annotations is as follows:

@LoadBalanced
@Bean
public RestTemplate getRestTemplate() {
   return new RestTemplate();
}

• Step 1: Ribbon has an automatic configuration class LoadBalancerAutoConfiguration. When SpringBoot loads the automatic configuration class, it will initialize the Ribbon.
• Step 2: When we add annotations to RestTemplate or AsyncRestTemplate, the Ribbon will collect the RestTemplate and AsyncRestTemplate annotated with @LoadBalanced during initialization, and put them into a List.
• Step 3: Then the RestTemplateCustomizer in the Ribbon will customize each RestTemplate, that is, add an interceptor: LoadBalancerInterceptor.
• Step 4: Obtain the service list from the Eureka registry and store it in the Ribbon.
• Step 5: Load the YMAL configuration file, configure the load balancing configuration, and create an ILoadbalancer instance.

6. Ribbon synchronization service list principle

After Ribbon obtains the full registry from Eureka for the first time, it will obtain the registry at regular intervals. The schematic diagram is as follows:

We mentioned before that ServerListUpdater, the core component of Ribbon, is used to synchronize the registry. It has an implementation class PollingServerListUpdater, which is specially used for timing synchronization. By default, a Runnable thread is executed after 1s, and then the Runnable thread is executed every 30s. This Runnable thread is to get the Eureka registry.

7. The principle of Eureka heartbeat detection

We know that the Eureka registry uses the heartbeat detection mechanism to determine whether the service is available. If it is not available, the service may be removed. Why is it possible? Because Eureka has a self-protection mechanism, if the threshold of the self-protection mechanism is reached, it will not be automatically removed in the future. Here we can review Eureka's self-protection mechanism and service removal mechanism.

• Eureka heartbeat mechanism : each service automatically sends a heartbeat to Eureka Server every 30s, and Eureka Server updates the last heartbeat time of this service. If the heartbeat is not received within 180 s (version 1.7.2), the task service is faulty;
• Eureka self-protection mechanism : If the actual number of heartbeats in the last minute is smaller than the expected number of heartbeats, the self-protection mechanism will be triggered and no instance will be removed. Expected number of heartbeats: number of service instances * 2 * 0.85;
• Eureka service removal mechanism : instead of removing the service instance at one time, a maximum of 15% is randomly removed each time. If the removal is not complete, remove it after 1 minute.

After talking about Eureka's heartbeat mechanism and service removal mechanism, let's look at Ribbon's heartbeat mechanism.

8. The principle of Ribbon heartbeat detection

Ribbon's heartbeat detection principle is not the same as that of Eureka. Ribbon does not detect whether the service is alive by sending a heartbeat to the Ribbon from each service or sending a heartbeat to each service from the Ribbon. Let's take a picture first to see Ribbon's heartbeat detection mechanism:

The principle of Ribbon heartbeat detection: Traverse the Server List cached locally to see if the status of each service is UP. The specific code is the isAlive method.

Core code:

isAlive = status.equals(InstanceStatus.UP);

So how often is it tested? By default, the following PingTask scheduling tasks are executed every 30s, and the isAlive method is executed for each service to determine the status. 

9. Ribbon common configuration items

9.1 Disable Eureka

# 禁用 Eureka
ribbon.eureka.enabled=false

The service registration list is obtained from Eureka by default. If you do not want to use Eureka, you can disable it. Then we need to manually configure the list of services.

9.2 Configure service list

ribbon-config-passjava.ribbon.listOfServers=localhost:8081,localhost:8083

This configuration is for a specific service, and the prefix is ​​the service name. After configuration, you can use the service name to call the interface as before.

9.3 Other configuration items

10. Summary

This article provides an in-depth explanation of the Ribbon architecture principle of the load balancing component in Spring Cloud microservices, which is divided into several major parts:

• Ribbon's six core components
• How Ribbon intercepts requests and forwards them
• The principle of Ribbon initialization
• The principle of how Ribbon synchronizes the Eureka registry
• The principles of Eureka and Ribbon heartbeat detection
• Common configuration items for Ribbon