table of Contents
2. Hard load balancing and soft load balancing
Third, the difference between client-side load balancing and server-side load balancing
Seven, load balancing strategy
1. What is load balancing?
Load balancing is a way to make the system highly available, solve network pressure, and improve processing capabilities; what we often say about load balancing refers to server-side load balancing, which is divided into hard load balancing and load balancing. The load balancing module maintains a list of available servers, and uses heartbeat checking to eliminate faulty server nodes to ensure that all the server nodes in the list can be accessed normally; when the client sends a request to the load balancing device, It will take out a server address from the list according to a certain algorithm (polling, random, etc.), and then forward it.
2. Hard load balancing and soft load balancing
Hard load balancing: mainly through the installation of dedicated load balancing equipment between server nodes, such as F5, A10, etc.;
Soft load balancing: It is mainly to complete the work of request distribution by installing software with load balancing functions or modules on the server, such as Nginx.
The architecture diagram is as follows:
Third, the difference between client-side load balancing and server-side load balancing
1. The location of service list maintenance is different; in client load balancing, all clients must maintain a list of services that they want to access, and the list comes from the service registry. In server-side load balancing, the list is maintained in the module of the load balancing device.
Four, RestTemplate
By configuring the RestTemplate object in the startup class and passing the @loadBalanced annotation, you can enable client load balancing.
@EnableDiscoveryClient
@SpringBootApplication
public class RibbonConsumerApplication {
@Bean
@LoadBalanced
RestTemplate restTemplate(){
return new RestTemplate();
};
public static void main(String[] args) {
SpringApplication.run(RibbonConsumerApplication.class, args);
}
}In RestTemplate, different request methods have different call implementation methods:
(1), GET request
@RestController
public class ConsumerController {
@Autowired
RestTemplate restTemplate;
@RequestMapping(value = "/ribbon-consumer",method = RequestMethod.GET)
public String helloConsumer(){
return restTemplate.getForEntity("http://indexService/index",String.class).getBody();
}
@RequestMapping(value = "/ribbon-consumer1",method = RequestMethod.GET)
public String helloConsumer1(){
return restTemplate.getForObject("http://indexService/index",String.class);
}
}
GET request has two functions that can be called: getForEntity and getForObject.
The getForEntity function returns ResponseEntity, which is Spring's encapsulation of the HTTP request response, storing information such as the request status, and then returning the required type through the getBody() method; such as the object type and so on. For specific information, please refer to the document.
The getForObject function is more convenient to use. There are three different overload implementations, which are equivalent to further encapsulation of the getForEntity function, and we can directly return the object type or other types.
(2) POST request
@RequestMapping(value = "/ribbon-consumer2",method = RequestMethod.POST)
public String helloConsumer2(){
User user = new User();
return restTemplate.postForEntity("http://indexService/user",user,String.class).getBody();
}
@RequestMapping(value = "/ribbon-consumer3",method = RequestMethod.POST)
public String helloConsumer3(){
User user = new User();
return restTemplate.postForObject("http://indexService/user",user,String.class);
}
@RequestMapping(value = "/ribbon-consumer4",method = RequestMethod.POST)
public URI helloConsumer4(){
User user = new User();
URI uri = restTemplate.postForLocation("http://indexService/user", user);
return uri;
}POST request has three functions that can be called: postForEntity, postForObject and postForLocation.
The postForEntity and postForObject functions are similar to the get request and have three different overloading methods; the postForLocation function returns the URI of the new resource, without specifying the return type.
(3) PUT and DELETE requests are not analyzed
Five, source code analysis
1. How does Ribbon implement client load balancing through RestTemplate?
We can find LoadBalancerClient through the source code of @LoadBalanced, its only implementation class RibbonLoadBalancerClient, which rewrites some methods, as shown in the figure:
The ServiceInstance choose(String serviceId) method selects a corresponding service instance from the load balancer according to the incoming service name serviceId.
The <T> T execute(String serviceId, LoadBalancerRequest<T> request) method uses the service strength selected from the load balancer to execute the request content.
Among them, the URI reconstructURI (ServiceInstance instance, URI original) method constructs a suitable URI for the system.
We can also view the automatic configuration of the load balancer through the LoadBalancerAutoConfiguration class. The source code is as follows:
@Configuration
@ConditionalOnClass(RestTemplate.class)
@ConditionalOnBean(LoadBalancerClient.class)
@EnableConfigurationProperties(LoadBalancerRetryProperties.class)
public class LoadBalancerAutoConfiguration {
@LoadBalanced
@Autowired(required = false)
private List<RestTemplate> restTemplates = Collections.emptyList();
@Autowired(required = false)
private List<LoadBalancerRequestTransformer> transformers = Collections.emptyList();
@Bean
public SmartInitializingSingleton loadBalancedRestTemplateInitializerDeprecated(
final ObjectProvider<List<RestTemplateCustomizer>> restTemplateCustomizers) {
return () -> restTemplateCustomizers.ifAvailable(customizers -> {
for (RestTemplate restTemplate : LoadBalancerAutoConfiguration.this.restTemplates) {
for (RestTemplateCustomizer customizer : customizers) {
customizer.customize(restTemplate);
}
}
});
}
@Bean
@ConditionalOnMissingBean
public LoadBalancerRequestFactory loadBalancerRequestFactory(
LoadBalancerClient loadBalancerClient) {
return new LoadBalancerRequestFactory(loadBalancerClient, this.transformers);
}
@Configuration
@ConditionalOnMissingClass("org.springframework.retry.support.RetryTemplate")
static class LoadBalancerInterceptorConfig {
@Bean
public LoadBalancerInterceptor ribbonInterceptor(
LoadBalancerClient loadBalancerClient,
LoadBalancerRequestFactory requestFactory) {
return new LoadBalancerInterceptor(loadBalancerClient, requestFactory);
}
@Bean
@ConditionalOnMissingBean
public RestTemplateCustomizer restTemplateCustomizer(
final LoadBalancerInterceptor loadBalancerInterceptor) {
return restTemplate -> {
List<ClientHttpRequestInterceptor> list = new ArrayList<>(
restTemplate.getInterceptors());
list.add(loadBalancerInterceptor);
restTemplate.setInterceptors(list);
};
}
}
/**
* Auto configuration for retry mechanism.
*/
@Configuration
@ConditionalOnClass(RetryTemplate.class)
public static class RetryAutoConfiguration {
@Bean
@ConditionalOnMissingBean
public LoadBalancedRetryFactory loadBalancedRetryFactory() {
return new LoadBalancedRetryFactory() {
};
}
}
/**
* Auto configuration for retry intercepting mechanism.
*/
@Configuration
@ConditionalOnClass(RetryTemplate.class)
public static class RetryInterceptorAutoConfiguration {
@Bean
@ConditionalOnMissingBean
public RetryLoadBalancerInterceptor ribbonInterceptor(
LoadBalancerClient loadBalancerClient,
LoadBalancerRetryProperties properties,
LoadBalancerRequestFactory requestFactory,
LoadBalancedRetryFactory loadBalancedRetryFactory) {
return new RetryLoadBalancerInterceptor(loadBalancerClient, properties,
requestFactory, loadBalancedRetryFactory);
}
@Bean
@ConditionalOnMissingBean
public RestTemplateCustomizer restTemplateCustomizer(
final RetryLoadBalancerInterceptor loadBalancerInterceptor) {
return restTemplate -> {
List<ClientHttpRequestInterceptor> list = new ArrayList<>(
restTemplate.getInterceptors());
list.add(loadBalancerInterceptor);
restTemplate.setInterceptors(list);
};
}
}
}
Ribbon to achieve automated configuration needs to meet two conditions: @ConditionalOnClass(RestTemplate.class) RestTemplate class must exist in the current project; @ConditionalOnBean(LoadBalancerClient.class) In the Bean project, there must be an implementation Bean of LoadBalancerClient.
This class mainly does three things:
1. Created a bean of loadBalancerInterceptor, which is used to intercept requests initiated by the client to achieve client load balancing.
2. Created a bean of RestTemplateCustomizer, used to add loadBalancerInterceptor interceptor to RestTemplate.
3. Maintain a list of RestTemplate objects decorated by @LoadBalanced annotation, and initialize it, and add a loadBalancerInterceptor interceptor to resttemplate that requires client load balancing by calling an instance of restTemplateCustomizer.
Next, let's take a look at how LoadBalancerInterceptor turns resttemplate into client load balancing. The source code is as follows:
public class LoadBalancerInterceptor implements ClientHttpRequestInterceptor {
private LoadBalancerClient loadBalancer;
private LoadBalancerRequestFactory requestFactory;
public LoadBalancerInterceptor(LoadBalancerClient loadBalancer,
LoadBalancerRequestFactory requestFactory) {
this.loadBalancer = loadBalancer;
this.requestFactory = requestFactory;
}
public LoadBalancerInterceptor(LoadBalancerClient loadBalancer) {
// for backwards compatibility
this(loadBalancer, new LoadBalancerRequestFactory(loadBalancer));
}
@Override
public ClientHttpResponse intercept(final HttpRequest request, final byte[] body,
final ClientHttpRequestExecution execution) throws IOException {
final URI originalUri = request.getURI();
String serviceName = originalUri.getHost();
Assert.state(serviceName != null,
"Request URI does not contain a valid hostname: " + originalUri);
return this.loadBalancer.execute(serviceName,
this.requestFactory.createRequest(request, body, execution));
}
}
Analyzing the source code, you can see that the implementation of LoadBalancerClient is injected into the interceptor. When a RestTemplate object modified by the @LoadBalanced annotation initiates an HTTP request, it will be intercepted by the intercept function in the LoadBalancerInterceptor class. String serviceName = originalUri.getHost(); Get the service name, and then call the excute function to select an instance based on the server name and initiate a request.
RibbonLoadBalancerClient as a specific implementation class, we can view the execute function in the source code, as follows:
public <T> T execute(String serviceId, LoadBalancerRequest<T> request, Object hint)
throws IOException {
ILoadBalancer loadBalancer = getLoadBalancer(serviceId);
Server server = getServer(loadBalancer, hint);
if (server == null) {
throw new IllegalStateException("No instances available for " + serviceId);
}
RibbonServer ribbonServer = new RibbonServer(serviceId, server,
isSecure(server, serviceId),
serverIntrospector(serviceId).getMetadata(server));
return execute(serviceId, ribbonServer, request);
}
@Override
public <T> T execute(String serviceId, ServiceInstance serviceInstance,
LoadBalancerRequest<T> request) throws IOException {
Server server = null;
if (serviceInstance instanceof RibbonServer) {
server = ((RibbonServer) serviceInstance).getServer();
}
if (server == null) {
throw new IllegalStateException("No instances available for " + serviceId);
}
RibbonLoadBalancerContext context = this.clientFactory
.getLoadBalancerContext(serviceId);
RibbonStatsRecorder statsRecorder = new RibbonStatsRecorder(context, server);
try {
T returnVal = request.apply(serviceInstance);
statsRecorder.recordStats(returnVal);
return returnVal;
}
// catch IOException and rethrow so RestTemplate behaves correctly
catch (IOException ex) {
statsRecorder.recordStats(ex);
throw ex;
}
catch (Exception ex) {
statsRecorder.recordStats(ex);
ReflectionUtils.rethrowRuntimeException(ex);
}
return null;
}It obtains the service instance through Server server = getServer(loadBalancer, hint). The source code of getServer is as follows:
protected Server getServer(ILoadBalancer loadBalancer, Object hint) {
if (loadBalancer == null) {
return null;
}
// Use 'default' on a null hint, or just pass it on?
return loadBalancer.chooseServer(hint != null ? hint : "default");
}Through analysis, we can see that when obtaining a specific service instance, the choose function in RibbonLoadBalancerClient is not used, but the chooseServer function defined in the ILoadBalancer interface is used. The ILoadBalancer interface source code is as follows:
public interface ILoadBalancer {
/**
* Initial list of servers.
* This API also serves to add additional ones at a later time
* The same logical server (host:port) could essentially be added multiple times
* (helpful in cases where you want to give more "weightage" perhaps ..)
*
* @param newServers new servers to add
*/
public void addServers(List<Server> newServers);
/**
* Choose a server from load balancer.
*
* @param key An object that the load balancer may use to determine which server to return. null if
* the load balancer does not use this parameter.
* @return server chosen
*/
public Server chooseServer(Object key);
/**
* To be called by the clients of the load balancer to notify that a Server is down
* else, the LB will think its still Alive until the next Ping cycle - potentially
* (assuming that the LB Impl does a ping)
*
* @param server Server to mark as down
*/
public void markServerDown(Server server);
/**
* @deprecated 2016-01-20 This method is deprecated in favor of the
* cleaner {@link #getReachableServers} (equivalent to availableOnly=true)
* and {@link #getAllServers} API (equivalent to availableOnly=false).
*
* Get the current list of servers.
*
* @param availableOnly if true, only live and available servers should be returned
*/
@Deprecated
public List<Server> getServerList(boolean availableOnly);
/**
* @return Only the servers that are up and reachable.
*/
public List<Server> getReachableServers();
/**
* @return All known servers, both reachable and unreachable.
*/
public List<Server> getAllServers();
}Which defines a series of abstract operations:
addServers: Add instances to the list of instances maintained in the load balancer;
chooseServer: Select a service instance from the list through a certain strategy;
markServerDown: used to notify and identify that an instance in the load balancer has stopped service;
getServerList: Get a list of service instances;
getAllServers: Get a list of all service instances, including services in normal and stopped states.
getReachableServers: Get the list of instances currently being served.
The implementation structure of this interface is shown in the figure:
It can be seen from the RibbonClientConfiguration configuration class that SpringCloud uses ZoneAwareLoadBalancer by default to implement the load balancer when integrating Ribbon.
@Bean
@ConditionalOnMissingBean
public ILoadBalancer ribbonLoadBalancer(IClientConfig config,
ServerList<Server> serverList, ServerListFilter<Server> serverListFilter,
IRule rule, IPing ping, ServerListUpdater serverListUpdater) {
if (this.propertiesFactory.isSet(ILoadBalancer.class, name)) {
return this.propertiesFactory.get(ILoadBalancer.class, config, name);
}
return new ZoneAwareLoadBalancer<>(config, rule, ping, serverList,
serverListFilter, serverListUpdater);
}Next, continue to look at the execute function in RibbonLoadBalancerClient. After obtaining the instance, wrap its content into a RibbonServer object, and use the object to call back the apply function of LoadBalancerRequest in the request interceptor, sending the request like a service instance.
Part is not perfect.
Six, load balancer
Here we mainly conduct a simple analysis of the implementation class of the ILoadBalancer interface:
(1) The source code of AbstractLoadBalancer is as follows:
public abstract class AbstractLoadBalancer implements ILoadBalancer {
public enum ServerGroup{
ALL,
STATUS_UP,
STATUS_NOT_UP
}
/**
* delegate to {@link #chooseServer(Object)} with parameter null.
*/
public Server chooseServer() {
return chooseServer(null);
}
/**
* List of servers that this Loadbalancer knows about
*
* @param serverGroup Servers grouped by status, e.g., {@link ServerGroup#STATUS_UP}
*/
public abstract List<Server> getServerList(ServerGroup serverGroup);
/**
* Obtain LoadBalancer related Statistics
*/
public abstract LoadBalancerStats getLoadBalancerStats();
}
The enumerated ServerGroup includes three types: ALL service instances, STATUS_UP normal service instances, and STATUS_NOT_UP service stopped instances.
The getServerList function obtains a list of service instances based on the type.
The getLoadBalancerStats function defines the method to obtain the LoadBalancerStats object.
(Two), BaseLoadBalancer source code is too much to show
The BaseLoadBalancer class is the basic implementation class of the Ribbon load balancer, which defines the content of the load balancer.
(3) There are too many sources of DynamicServerListLoadBalancer to show
DynamicServerListLoadBalancer is an extension of the BaseLoadBalancer class. It is the service list of the load balancer that can dynamically update the list at runtime, and has the function of filtering the list of service instances.
(4) The source code of ZoneAwareLoadBalancer is as follows:
/*
*
* Copyright 2013 Netflix, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package com.netflix.loadbalancer;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.common.annotations.VisibleForTesting;
import com.netflix.client.ClientFactory;
import com.netflix.client.config.IClientConfig;
import com.netflix.config.DynamicBooleanProperty;
import com.netflix.config.DynamicDoubleProperty;
import com.netflix.config.DynamicPropertyFactory;
/**
* Load balancer that can avoid a zone as a whole when choosing server.
*<p>
* The key metric used to measure the zone condition is Average Active Requests,
which is aggregated per rest client per zone. It is the
total outstanding requests in a zone divided by number of available targeted instances (excluding circuit breaker tripped instances).
This metric is very effective when timeout occurs slowly on a bad zone.
<p>
The LoadBalancer will calculate and examine zone stats of all available zones. If the Average Active Requests for any zone has reached a configured threshold, this zone will be dropped from the active server list. In case more than one zone has reached the threshold, the zone with the most active requests per server will be dropped.
Once the the worst zone is dropped, a zone will be chosen among the rest with the probability proportional to its number of instances.
A server will be returned from the chosen zone with a given Rule (A Rule is a load balancing strategy, for example {@link AvailabilityFilteringRule})
For each request, the steps above will be repeated. That is to say, each zone related load balancing decisions are made at real time with the up-to-date statistics aiding the choice.
* @author awang
*
* @param <T>
*/
public class ZoneAwareLoadBalancer<T extends Server> extends DynamicServerListLoadBalancer<T> {
private ConcurrentHashMap<String, BaseLoadBalancer> balancers = new ConcurrentHashMap<String, BaseLoadBalancer>();
private static final Logger logger = LoggerFactory.getLogger(ZoneAwareLoadBalancer.class);
private volatile DynamicDoubleProperty triggeringLoad;
private volatile DynamicDoubleProperty triggeringBlackoutPercentage;
private static final DynamicBooleanProperty ENABLED = DynamicPropertyFactory.getInstance().getBooleanProperty("ZoneAwareNIWSDiscoveryLoadBalancer.enabled", true);
void setUpServerList(List<Server> upServerList) {
this.upServerList = upServerList;
}
public ZoneAwareLoadBalancer() {
super();
}
@Deprecated
public ZoneAwareLoadBalancer(IClientConfig clientConfig, IRule rule,
IPing ping, ServerList<T> serverList, ServerListFilter<T> filter) {
super(clientConfig, rule, ping, serverList, filter);
}
public ZoneAwareLoadBalancer(IClientConfig clientConfig, IRule rule,
IPing ping, ServerList<T> serverList, ServerListFilter<T> filter,
ServerListUpdater serverListUpdater) {
super(clientConfig, rule, ping, serverList, filter, serverListUpdater);
}
public ZoneAwareLoadBalancer(IClientConfig niwsClientConfig) {
super(niwsClientConfig);
}
@Override
protected void setServerListForZones(Map<String, List<Server>> zoneServersMap) {
super.setServerListForZones(zoneServersMap);
if (balancers == null) {
balancers = new ConcurrentHashMap<String, BaseLoadBalancer>();
}
for (Map.Entry<String, List<Server>> entry: zoneServersMap.entrySet()) {
String zone = entry.getKey().toLowerCase();
getLoadBalancer(zone).setServersList(entry.getValue());
}
// check if there is any zone that no longer has a server
// and set the list to empty so that the zone related metrics does not
// contain stale data
for (Map.Entry<String, BaseLoadBalancer> existingLBEntry: balancers.entrySet()) {
if (!zoneServersMap.keySet().contains(existingLBEntry.getKey())) {
existingLBEntry.getValue().setServersList(Collections.emptyList());
}
}
}
@Override
public Server chooseServer(Object key) {
if (!ENABLED.get() || getLoadBalancerStats().getAvailableZones().size() <= 1) {
logger.debug("Zone aware logic disabled or there is only one zone");
return super.chooseServer(key);
}
Server server = null;
try {
LoadBalancerStats lbStats = getLoadBalancerStats();
Map<String, ZoneSnapshot> zoneSnapshot = ZoneAvoidanceRule.createSnapshot(lbStats);
logger.debug("Zone snapshots: {}", zoneSnapshot);
if (triggeringLoad == null) {
triggeringLoad = DynamicPropertyFactory.getInstance().getDoubleProperty(
"ZoneAwareNIWSDiscoveryLoadBalancer." + this.getName() + ".triggeringLoadPerServerThreshold", 0.2d);
}
if (triggeringBlackoutPercentage == null) {
triggeringBlackoutPercentage = DynamicPropertyFactory.getInstance().getDoubleProperty(
"ZoneAwareNIWSDiscoveryLoadBalancer." + this.getName() + ".avoidZoneWithBlackoutPercetage", 0.99999d);
}
Set<String> availableZones = ZoneAvoidanceRule.getAvailableZones(zoneSnapshot, triggeringLoad.get(), triggeringBlackoutPercentage.get());
logger.debug("Available zones: {}", availableZones);
if (availableZones != null && availableZones.size() < zoneSnapshot.keySet().size()) {
String zone = ZoneAvoidanceRule.randomChooseZone(zoneSnapshot, availableZones);
logger.debug("Zone chosen: {}", zone);
if (zone != null) {
BaseLoadBalancer zoneLoadBalancer = getLoadBalancer(zone);
server = zoneLoadBalancer.chooseServer(key);
}
}
} catch (Exception e) {
logger.error("Error choosing server using zone aware logic for load balancer={}", name, e);
}
if (server != null) {
return server;
} else {
logger.debug("Zone avoidance logic is not invoked.");
return super.chooseServer(key);
}
}
@VisibleForTesting
BaseLoadBalancer getLoadBalancer(String zone) {
zone = zone.toLowerCase();
BaseLoadBalancer loadBalancer = balancers.get(zone);
if (loadBalancer == null) {
// We need to create rule object for load balancer for each zone
IRule rule = cloneRule(this.getRule());
loadBalancer = new BaseLoadBalancer(this.getName() + "_" + zone, rule, this.getLoadBalancerStats());
BaseLoadBalancer prev = balancers.putIfAbsent(zone, loadBalancer);
if (prev != null) {
loadBalancer = prev;
}
}
return loadBalancer;
}
private IRule cloneRule(IRule toClone) {
IRule rule;
if (toClone == null) {
rule = new AvailabilityFilteringRule();
} else {
String ruleClass = toClone.getClass().getName();
try {
rule = (IRule) ClientFactory.instantiateInstanceWithClientConfig(ruleClass, this.getClientConfig());
} catch (Exception e) {
throw new RuntimeException("Unexpected exception creating rule for ZoneAwareLoadBalancer", e);
}
}
return rule;
}
@Override
public void setRule(IRule rule) {
super.setRule(rule);
if (balancers != null) {
for (String zone: balancers.keySet()) {
balancers.get(zone).setRule(cloneRule(rule));
}
}
}
}
It is actually an extension of DynamicServerListLoadBalancer.
Seven, load balancing strategy
Ribbon implements many selection strategies. The following figure shows the various implementations of the IRule interface:
From the diagram, we can find that the AbstractLoadBalancerRule class is an abstract class for load balancing, which defines the load balancer ILoadBalancer object, which can obtain service information as a basis when implementing specific service selection strategies.
public abstract class AbstractLoadBalancerRule implements IRule, IClientConfigAware {
private ILoadBalancer lb;
@Override
public void setLoadBalancer(ILoadBalancer lb){
this.lb = lb;
}
@Override
public ILoadBalancer getLoadBalancer(){
return lb;
}
}RandomRule : Random strategy, that is, a service is randomly selected from the list of service instances. Its specific implementation is as follows: rewrite the choose (Object key) function in the IRule interface, and call the choose (ILoadBalancer lb, Object key) under this class, adding a load balancer object to the parameters. First obtain the available instance list upList and all the instance list allList through the load balancer object. If the sizes of the two collections are both zero, then no service is available. The ThreadLocalRandom.current().nextInt(serverCount) function gets a random value, and Use this random value as the index of upList to return a specific instance, and return this instance if it is available and ready.
public class RandomRule extends AbstractLoadBalancerRule {
/**
* Randomly choose from all living servers
*/
@edu.umd.cs.findbugs.annotations.SuppressWarnings(value = "RCN_REDUNDANT_NULLCHECK_OF_NULL_VALUE")
public Server choose(ILoadBalancer lb, Object key) {
if (lb == null) {
return null;
}
Server server = null;
while (server == null) {
if (Thread.interrupted()) {
return null;
}
List<Server> upList = lb.getReachableServers();
List<Server> allList = lb.getAllServers();
int serverCount = allList.size();
if (serverCount == 0) {
/*
* No servers. End regardless of pass, because subsequent passes
* only get more restrictive.
*/
return null;
}
int index = chooseRandomInt(serverCount);
server = upList.get(index);
if (server == null) {
/*
* The only time this should happen is if the server list were
* somehow trimmed. This is a transient condition. Retry after
* yielding.
*/
Thread.yield();
continue;
}
if (server.isAlive()) {
return (server);
}
// Shouldn't actually happen.. but must be transient or a bug.
server = null;
Thread.yield();
}
return server;
}
protected int chooseRandomInt(int serverCount) {
return ThreadLocalRandom.current().nextInt(serverCount);
}
@Override
public Server choose(Object key) {
return choose(getLoadBalancer(), key);
}
@Override
public void initWithNiwsConfig(IClientConfig clientConfig) {
// TODO Auto-generated method stub
}
}
RoundRobinRule : Select each service instance in turn according to the linear polling method. Its internal structure is similar to RandomRule. We can see that it adds a count variable. If no service is selected for more than 10 times, then the attempt is ended and No available alive servers after 10 tries from load balancer are printed. The difference between it and RandomRule is that linear polling is implemented by the AtomicInteger nextServerCyclicCounter object, and increment is achieved by calling the int nextServerIndex = incrementAndGetModulo(serverCount); function each time an instance is selected.
public class RoundRobinRule extends AbstractLoadBalancerRule {
private AtomicInteger nextServerCyclicCounter;
private static final boolean AVAILABLE_ONLY_SERVERS = true;
private static final boolean ALL_SERVERS = false;
private static Logger log = LoggerFactory.getLogger(RoundRobinRule.class);
public RoundRobinRule() {
nextServerCyclicCounter = new AtomicInteger(0);
}
public RoundRobinRule(ILoadBalancer lb) {
this();
setLoadBalancer(lb);
}
public Server choose(ILoadBalancer lb, Object key) {
if (lb == null) {
log.warn("no load balancer");
return null;
}
Server server = null;
int count = 0;
while (server == null && count++ < 10) {
List<Server> reachableServers = lb.getReachableServers();
List<Server> allServers = lb.getAllServers();
int upCount = reachableServers.size();
int serverCount = allServers.size();
if ((upCount == 0) || (serverCount == 0)) {
log.warn("No up servers available from load balancer: " + lb);
return null;
}
int nextServerIndex = incrementAndGetModulo(serverCount);
server = allServers.get(nextServerIndex);
if (server == null) {
/* Transient. */
Thread.yield();
continue;
}
if (server.isAlive() && (server.isReadyToServe())) {
return (server);
}
// Next.
server = null;
}
if (count >= 10) {
log.warn("No available alive servers after 10 tries from load balancer: "
+ lb);
}
return server;
}
/**
* Inspired by the implementation of {@link AtomicInteger#incrementAndGet()}.
*
* @param modulo The modulo to bound the value of the counter.
* @return The next value.
*/
private int incrementAndGetModulo(int modulo) {
for (;;) {
int current = nextServerCyclicCounter.get();
int next = (current + 1) % modulo;
if (nextServerCyclicCounter.compareAndSet(current, next))
return next;
}
}
@Override
public Server choose(Object key) {
return choose(getLoadBalancer(), key);
}
@Override
public void initWithNiwsConfig(IClientConfig clientConfig) {
}
}
RetryRule : An instance selection strategy with a retry mechanism. From the source code, we can see that an IRule object is defined and an instance of RoundRobinRule is used. In the choose(ILoadBalancer lb, Object key) function, the strategy of repeatedly trying the internally defined strategy is implemented. If a specific instance can be selected during the period, it will return. If it has not been selected, it will end until the set threshold is reached. Then return null.
public class RetryRule extends AbstractLoadBalancerRule {
IRule subRule = new RoundRobinRule();
long maxRetryMillis = 500;
public RetryRule() {
}
public RetryRule(IRule subRule) {
this.subRule = (subRule != null) ? subRule : new RoundRobinRule();
}
public RetryRule(IRule subRule, long maxRetryMillis) {
this.subRule = (subRule != null) ? subRule : new RoundRobinRule();
this.maxRetryMillis = (maxRetryMillis > 0) ? maxRetryMillis : 500;
}
public void setRule(IRule subRule) {
this.subRule = (subRule != null) ? subRule : new RoundRobinRule();
}
public IRule getRule() {
return subRule;
}
public void setMaxRetryMillis(long maxRetryMillis) {
if (maxRetryMillis > 0) {
this.maxRetryMillis = maxRetryMillis;
} else {
this.maxRetryMillis = 500;
}
}
public long getMaxRetryMillis() {
return maxRetryMillis;
}
@Override
public void setLoadBalancer(ILoadBalancer lb) {
super.setLoadBalancer(lb);
subRule.setLoadBalancer(lb);
}
/*
* Loop if necessary. Note that the time CAN be exceeded depending on the
* subRule, because we're not spawning additional threads and returning
* early.
*/
public Server choose(ILoadBalancer lb, Object key) {
long requestTime = System.currentTimeMillis();
long deadline = requestTime + maxRetryMillis;
Server answer = null;
answer = subRule.choose(key);
if (((answer == null) || (!answer.isAlive()))
&& (System.currentTimeMillis() < deadline)) {
InterruptTask task = new InterruptTask(deadline
- System.currentTimeMillis());
while (!Thread.interrupted()) {
answer = subRule.choose(key);
if (((answer == null) || (!answer.isAlive()))
&& (System.currentTimeMillis() < deadline)) {
/* pause and retry hoping it's transient */
Thread.yield();
} else {
break;
}
}
task.cancel();
}
if ((answer == null) || (!answer.isAlive())) {
return null;
} else {
return answer;
}
}
@Override
public Server choose(Object key) {
return choose(getLoadBalancer(), key);
}
@Override
public void initWithNiwsConfig(IClientConfig clientConfig) {
}
}WeightedResponseTimeRule : It is an extension of RoundRobinRule, adding weights based on the running conditions of the instances, and selecting instances based on the weights, which can achieve a better distribution effect. It mainly has three cores: timing tasks, weight calculations and instance selection.
Timed tasks:
When WeightedResponseTimeRule is initialized, it will start a timed task through serverWeightTimer.schedule(new DynamicServerWeightTask(), 0, serverWeightTaskTimerInterval); to calculate the weight for each service, which is executed every 30 seconds by default.
class DynamicServerWeightTask extends TimerTask {
public void run() {
ServerWeight serverWeight = new ServerWeight();
try {
serverWeight.maintainWeights();
} catch (Exception e) {
logger.error("Error running DynamicServerWeightTask for {}", name, e);
}
}
}Weight calculation:
Through the source code, it is found that private volatile List<Double> accumulatedWeights = new ArrayList<Double>(); is an object used to store weights. The position of each weight value in the list corresponds to all the service instances in the list of service instances maintained by the load balancer. The position of the instance in the list.
public void maintainWeights() {
ILoadBalancer lb = getLoadBalancer();
if (lb == null) {
return;
}
if (!serverWeightAssignmentInProgress.compareAndSet(false, true)) {
return;
}
try {
logger.info("Weight adjusting job started");
AbstractLoadBalancer nlb = (AbstractLoadBalancer) lb;
LoadBalancerStats stats = nlb.getLoadBalancerStats();
if (stats == null) {
// no statistics, nothing to do
return;
}
//计算所有实例的平均响应时间的总和
double totalResponseTime = 0;
// find maximal 95% response time
for (Server server : nlb.getAllServers()) {
// 如果服务实例的状态不在缓存中那么这里会进行自动加载
ServerStats ss = stats.getSingleServerStat(server);
totalResponseTime += ss.getResponseTimeAvg();
}
// weight for each server is (sum of responseTime of all servers - responseTime)
// so that the longer the response time, the less the weight and the less likely to be chosen
Double weightSoFar = 0.0;
// create new list and hot swap the reference
List<Double> finalWeights = new ArrayList<Double>();
for (Server server : nlb.getAllServers()) {
ServerStats ss = stats.getSingleServerStat(server);
double weight = totalResponseTime - ss.getResponseTimeAvg();
weightSoFar += weight;
finalWeights.add(weightSoFar);
}
setWeights(finalWeights);
} catch (Exception e) {
logger.error("Error calculating server weights", e);
} finally {
serverWeightAssignmentInProgress.set(false);
}
}Example selection: choose function
@Override
public Server choose(ILoadBalancer lb, Object key) {
if (lb == null) {
return null;
}
Server server = null;
while (server == null) {
// get hold of the current reference in case it is changed from the other thread
List<Double> currentWeights = accumulatedWeights;
if (Thread.interrupted()) {
return null;
}
List<Server> allList = lb.getAllServers();
int serverCount = allList.size();
if (serverCount == 0) {
return null;
}
int serverIndex = 0;
// last one in the list is the sum of all weights
double maxTotalWeight = currentWeights.size() == 0 ? 0 : currentWeights.get(currentWeights.size() - 1);
// No server has been hit yet and total weight is not initialized
// fallback to use round robin
if (maxTotalWeight < 0.001d || serverCount != currentWeights.size()) {
server = super.choose(getLoadBalancer(), key);
if(server == null) {
return server;
}
} else {
// generate a random weight between 0 (inclusive) to maxTotalWeight (exclusive)
double randomWeight = random.nextDouble() * maxTotalWeight;
// pick the server index based on the randomIndex
int n = 0;
for (Double d : currentWeights) {
if (d >= randomWeight) {
serverIndex = n;
break;
} else {
n++;
}
}
server = allList.get(serverIndex);
}
if (server == null) {
/* Transient. */
Thread.yield();
continue;
}
if (server.isAlive()) {
return (server);
}
// Next.
server = null;
}
return server;
}
ClientConfigEnabledRoundRobinRule : Special strategy. We generally inherit this strategy and will not use it directly. BestAvailableRule, PredicateBasedRule, AvailabilityFilteringRule and ZoneAvoidanceRule are all directly or indirectly inherited from this strategy. Many advanced strategies are implemented based on ClientConfigEnabledRoundRobinRule. (Related strategies will not be studied for the time being)
Eight, configuration
(1) Automatic configuration
Reference document
(Two), parameter configuration
There are two ways to configure Ribbon parameters: global configuration or specified client configuration
Global configuration: configure according to the ribbon.<key> = <value> format, where key represents the parameter name, and value represents the parameter value.
Specify client configuration: configure according to the format of <client>.ribbon.<key> = <value>, client represents the client name
(Three), combined with Eureka
Reference document
(4) Retry mechanism
Reference document