Spring cloud ribbon source code analysis

Ribbon is a load balancing project open sourced by Netflix. It belongs to the second type mentioned above. It is a client load balancer that runs on the client. Let's first look at three sequence diagrams, namely RestTemplate, Feign, and Zuul using Ribbon's call chain.

As can be seen from the above three figures, the ILoadBalancer#chooseServer() method is ultimately called to filter service instances, and in spring cloud, ZoneAwareLoadBalancer is used by default.

ILoadBalancer

Let's first take a look at the ILoadBalancer interface, which defines a series of abstract operations that client load balancing needs to use

public interface ILoadBalancer {
    public void addServers(List<Server> newServers);
    public Server chooseServer(Object key);
    public void markServerDown(Server server);
    public List<Server> getReachableServers();
    public List<Server> getAllServers();
}

• addServers(List)
  adds service instances to the instance list maintained in the load balancer

• chooseServer(Object)
  selects a specific service instance from the load balancer through a certain strategy

• markServerDown(Server) is
  used to notify and identify that a specific instance in the load balancer has stopped serving, otherwise the load balancer will think that the service instances are in normal service before obtaining the service instance list next time

• getReachableServers()
  returns a list of currently serviceable instances

• getAllServers()
  returns a list of all known service instances, including normal service and stopped service instances. The Server object definition involved
  in this interface is a traditional server node, in which some basic information of the server node is stored, such as host\port, etc.

For the implementation class ZoneAwareLoadBalancer of this interface, sort out the structure as shown in the following figure

It can be seen that the BaseLoadBalancer class implements basic load balancing, and DynamicServerListLoadBalancer and ZoneAwareLoadBalancer have made some extensions on its basis.

AbstractLoadBalancer

AbstractLoadBalancer is an abstract implementation of the ILoadBalancer interface.

public abstract class AbstractLoadBalancer implements ILoadBalancer {
    public enum ServerGroup{
        ALL,
        STATUS_UP,
        STATUS_NOT_UP        
    }
    public Server chooseServer() {
        return chooseServer(null);
    }
    public abstract List<Server> getServerList(ServerGroup serverGroup);
    public abstract LoadBalancerStats getLoadBalancerStats();    
}

A grouping enumeration class ServerGroup about service instances is defined in this abstract class, which contains three different types.
ALL: All service instances
STATUS_UP: Instances in service
STATUS_NOT_UP: Instances out of service

It also implements a chooseServer() method, which calls the implementation of chooseServer(Object key) in the interface, where the parameter key is null, indicating that the conditional judgment of the key is ignored when selecting a service instance.
Then define two abstract methods

• getServerList(ServerGroup):
  Returns a list of different service instances according to the grouping type.

• getLoadBalancerStats():
  Returns the LoadBalancerStatus object. The LoadBalancerStats object is used to store the current attributes and statistics of each service instance in the load balancer. We can observe the operation of the load balancer based on this information. They are also an important basis for formulating load balancing strategies, such as It is used in both AvailabilityPredicate and ZoneAvoiadancePredicate.

BaseLoadBalancer

BaseLoadBalancer is the basic implementation class of Ribbon load balancer, which defines a lot of basic content related to load balancer.

• Define and maintain two lists of service instance Server objects,
  one is to store all service instance lists
  , the other is to store normal service instance lists

• Defines the LoadBalanceStats object that stores the attributes and statistics of each service instance of the load system server

• Implements a series of basic operation methods that the load balancer defined by the ILoadBalancer interface should have
  addServers(List newServers)
  chooseServer(Object key)
  markServerDown(Server server)
  getReachableServers()
  getAllServers()

• Defines the IPing object that checks whether the service instance is in normal service.
  IPing is used to initiate a "ping" to the server to determine whether the server has a response, so as to determine whether the server is available. It has a boolean isAlive(Server) method. The implementation class of IPing has the following types of
  PingUrl
  to actually ping a url to determine whether it is alive

  PingConstant
  returns whether a service is available or not, and returns true by default, that is, it is available

  NoOpPing does
  not ping, it returns true directly, and it can be used

  DummyPing
  returns true directly

  NIWSDiscoveryPing judges
  according to the InstanceStatus of InstanceInfo of DiscoveryEnabledServer. If it is InstanceStatus.UP, it is available, otherwise it is unavailable

• Defines the execution strategy of the execution strategy object IPingStrategy
  ping to check the operation of the service instance. The default is linear polling in BaseLoadBalancer, which can be extended to parallel by itself.

• The IRule object of the load balancing processing rules is defined.
  Looking at the chooseServer(Object key) method of this class, we can know that the load balancer actually delegates the task logic selected by the service instance to the choose() method of the IRule instance to implement, and defaults to Initialize RoundRobinRule (linear polling) as the implementation object of IRule. The implementation class of IRule has
  BestAvailableRule
  to select the server with the minimum number of requests

  RandomRule
  randomly selected

  RoundRobinRule
  linear polling

  RetryRule retries
  based on polling

  WeightedResponseTimeRule
  assigns Weight according to the response time. The higher the Weight, the more likely it is to be selected.

  * ZoneAvoidanceRule*
  polls the selection based on the server's zone and availability

• Start the ping task
  In the construction method of BaseLoadBalancer, a task to regularly check whether the server is in normal service will be started. The default execution interval is 10 seconds. The execution interval can also be set through IClientConfig.

  DefaultClientConfigImpl config = new DefaultClientConfigImpl();
  …
  config.setProperty(IClientConfigKey.Keys.NFLoadBalancerPingInterval, 15); // 注意单位s
  …
  ILoadBalancer lb = new BaseLoadBalancer(config);

DynamicServerListLoadBalancer

DynamicServerListLoadBalancer inherits from BaseLoadBalancer, which extends the base load balancer. In this load balancer, the dynamic update of the service instance list during operation is realized; at the same time, it also has the function of filtering the service instance list, that is to say, a batch of service instance lists can be selected and obtained through the filter.

• ServerList
  is the newly added service list operation object of DynamicServerListLoadBalancer. It defines two methods,
  getInitialListOfServers(),
  to obtain the list of initialized service instances.

  getUpdatedListOfServers()
  is used to get the list of updated service instances

  The DomainExtractingServerList is used in DynamicServerListLoadBalancer. The ServerList also defines a ServerList list internally. The specific implementation of the DomainExtractingServerList for the above two methods is delegated to the internally defined ServerList object.
  The internally defined ServerList object is actually DiscoveryEnabledNIWSServerList. The implementation logic mainly relies on EurekaClient to obtain a list of service instances (instanceInfo) from the registry, and then traverses these service instances, converting the instances whose status is UP (normal service) into a DiscoveryEnabledServer object, and finally Returns organized as a list.
  Return the List to the DomainExtractingServerList class, convert the DiscoverEnabledServer object returned by DiscoveryEnabledNIWSServerList into its subclass object DomainExtractingServer, and initialize some properties in the constructor of this class, such as id, zone, isAliveFlag, readyToServer, etc.

• The ServerListUpdater
  interface implements the update of ServerList, which can be called "service updater". It defines a series of methods to control it

  public interface ServerListUpdater {
      //内部接口
      public interface UpdateAction {
      //实现对ServerList的更新操作
          void doUpdate();
      }
      //启动服务更新器
      void start(UpdateAction updateAction);
      //停止服务更新器
      void stop();
      //返回最近的更新时间戳
      String getLastUpdate();
      //返回上一次更新到现在的时间间隔，单位为ms
      long getDurationSinceLastUpdateMs();
      //返回错过的更新周期数
      int getNumberMissedCycles();
      //返回核心线程数
      int getCoreThreads();
  }

  Its implementation class has two default strategies of
  PollingServerListUpdater
  DynamicServerListLoadBalancer, which updates the service list by means of timed tasks

  EurekaNotificationServerListUpdater
  It needs to use Eureka's event listener to drive the update operation of the service list

• ServerListFilter
  This interface is very simple. There is only one method, List getFilteredListOfServers(List). It is mainly used to filter the service instance list. By passing in the service instance list, it returns the filtered service instance list according to some rules.
  

  ZoneAffinityServerListFilter
  This filter implements the filtering of service instances based on the "Zone Affinity" method. It compares the zone (Zone) of the instance providing the service with the zone (Zone) where the consumer is located, and filters out those Instances that are not in the same region.
  It should be noted that the filter also uses the shouldEnableZoneAffinity(List) method to determine whether to enable the "zone awareness" function. It uses the getZoneSnapshot method of LoadBalancerStats to obtain these filtered basic indicators of the same area instance (including the instance number, number of circuit breakers disconnected, number of clearing requests, average load of instances, etc.), according to a series of algorithms, the following evaluation values ​​are obtained and compared with the thresholds. If one of the conditions is met, "area awareness" will not be enabled. List of filtered service instances. The purpose is to provide services by relying on normal instances in other regions to ensure high availability when a regional failure occurs in the cluster.
  1､Percentage of faulty instances (number of disconnected circuit breakers/number of instances)>=0.8
  2､Average load of instances >=0.6
  3､Number of available instances (number of instances - number of disconnected circuit breakers)<2

  DefaultNIWSServerListFilterThis
  filter completely inherits ZoneAffinityServerListFilter

  ServerListSubsetFilter
  This filter is suitable for systems with large server clusters (hundreds or more), because it can produce a subset list of "region-aware" results, and it can also compare the number of communication failures and concurrent connections by service instances. number to determine whether the service is healthy to selectively remove those instances that are relatively unhealthy from the service instance list

  ZonePreferenceServerListFilter
  uses this filter by default when spring cloud integrates eureka and ribbon. It implements filtering out service instances in the same zone by configuring or the zone to which the eureka instance has no data.
  First, obtain the list of service instances filtered by "zone awareness" through the parent class, then traverse the list, and take out the preset zone according to the consumer configuration for filtering. If the filtered result is empty, return to the parent class directly. , otherwise return the filtered result.
  The consumer's preset zone can be configured through eureka.instance.metadataMap.zone. The initialization code is in the org.springframework.cloud.netflix.ribbon.eureka.EurekaRibbonClientConfiguration#preprocess() method. If eureka.instance.metadataMap.zone If not set, the default region and zone will be read.

ZoneAwareLoadBalancer

ZoneAwareLoadBalancer is an extension to DynamicServerListLoadBalancer. There is no zone area in DynamicServerListLoadBalancer, and the main extended function of ZoneAwareLoadBalancer is to add zone area filtering.
1. Rewrite the setServerListForZones(Map) method to create BaseLoadBalancer by zone. If ZoneAwareLoadBalancer#IRule is empty, AvailabilityFilteringRule is used by default, otherwise ZoneAwareLoadBalancer#IRule is used.
2. Rewrite the chooseServer(Object) method. When the number of instance zones maintained in the load balancer is greater than 1, the following strategy will be executed, otherwise, the method of the parent class will be executed.
3. Create zone snapshots based on LoadBalancerStats for subsequent algorithms.
4. According to the statistical data in the zone snapshot, the selection of the availability zone is realized.
5. When the returned set of available zones is not empty and the number is less than the total number of zones, a zone is randomly selected.
6. After determining the zone, obtain the load balancer corresponding to the zone, and call the chooseServer(Object) method to select a specific service instance.