Micro-channel public number: landlord of a small dark
road with the distant future, a better
learning is endless, we Come together!
When we encountered traffic at high concurrency surges, often referred to 降级, 熔断and 限流concepts. Let me briefly explain the definition of these three concepts.
Related concepts
Downgrade
downgrade service is downgraded, when our servers pressure surge, in order to ensure the availability of core functions, selectively reduced availability of some features, or directly turn off the feature. For example, in buying two-eleven in the morning, when the flow pressure is great, in order to ensure the normal payment orders, orders in the morning around 1-2 o'clock modification feature is turned off.
Fuse
downgrade generally refers to our own system broke down and downgrade. Generally it refers to the fusing of the external interface dependency failure severed and external interface relationship.
A service such as yours where a service function dependent on B, then B service issues arise, and return slowly. The more extensive system, the call chain downstream will be longer, and if in a very long call chain, one service for some reason, the response time is too long, or completely unresponsive, it can be able to entire distributed systems collapse. This function may slow down because of all the features A service inside this case, severe cases can lead to service an avalanche phenomenon.
In a distributed system, in order to ensure the availability and consistency of the overall service, many systems will be used to retry mechanism , in some jitter failure occurred because the network reasons of such measures can be adopted.
But in some cases, not suitable for a retry mechanism , but further undermined the performance of the system. For example, the downstream system because the request is too large, resulting in the CPU has been played, or the database connection pool is filled, this time the system downstream of the upstream system call information obtaining nonconducting will retry, retry is in this case resulting in the collapse of the system downstream.
In a distributed system, most of the service but also because of the continuing avalanche caused retry, retry this may be the frame-level auto-retry, retry logic there may be code level, there may be an active user Retry.
Then we can use fuse mode to address this phenomenon. A typical fuse has three states:
1) Close
the fuse is off by default. Fuse itself with counting function, whenever an error occurs once, the counter will accumulate to a certain number of fuse opens, a timer is turned on simultaneously, once the time will be switched to the 半开启state.
2) Open
any request will be denied directly in an open state and a throw exception information.
3) is half on
the fuse section allows the request if these requests are successful, then that error does not exist, it will be switched to the closed state and the count is reset. If there is any request for a failure occurs, it will be restored to the 开启state, and resets the timer, a break time administering the system.
Rambling: how to explain to his girlfriend what is the fuse?
Limiting
this readily appreciated by evaluation of the system is only allowed to enter a certain number of requests, the remaining requests rejected. For example, in the system spike, spike to 100 parts of a commercial, I only allow incoming requests before 2000, the remaining direct interception reject.
Now that we have explained the three concepts, then the next first began to explain dubbo是如何降级的.
Dubbo service degradation
Dubbo error can temporarily block a non-critical services by service degradation function, and define the return policy after the downgrade. We can write rules covering the dynamic configuration to the registry:
RegistryFactory registryFactory = ExtensionLoader.getExtensionLoader(RegistryFactory.class).getAdaptiveExtension();
Registry registry = registryFactory.getRegistry(URL.valueOf("zookeeper://10.20.153.10:2181"));
registry.register(URL.valueOf("override://0.0.0.0/com.foo.BarService?category=configurators&dynamic=false&application=foo&mock=force:return+null"));
复制代码It can also be defined by the configuration file.
<dubbo:reference id="iUser" interface="com.dubbosample.iface.IUser" timeout="1000" check="false" mock="return null">
复制代码It also has several forms:
<dubbo:reference mock="true" .../>
<dubbo:reference mock="com.xxxx" .../>
<dubbo:reference mock="return null" .../>
<dubbo:reference mock="throw xxx" .../>
<dubbo:reference mock="force:return .." .../>
<dubbo:reference mock="force:throw ..." .../>
复制代码其中，最主要的两种形式是：
1） mock='force:return+null'表示消费对该服务的方法调用都直接返回null值，不发起远程调用。用来屏蔽不重要服务不可用时对调用方的影响。
2） 还可以改为mock=fail:return+null表示消费方对该服务的方法调用在失败后，再返回null。用来容忍不重要服务不稳定时对调用方的影响。
具体代码
阅读过源码的知道Dubbo的远程调用是从一个代理Proxy开始的，首先将运行时参数存储在数组中，然后调用InvocationHandler接口来实现类的invoke方法。下面是一个动态生成的一个代理类例子。
public class proxy0 implements ClassGenerator.DC, EchoService, DemoService {
// 方法数组
public static Method[] methods;
private InvocationHandler handler;
public proxy0(InvocationHandler invocationHandler) {
this.handler = invocationHandler;
}
public proxy0() {
}
public String sayHello(String string) {
// 将参数存储到 Object 数组中
Object[] arrobject = new Object[]{string};
// 调用 InvocationHandler 实现类的 invoke 方法得到调用结果
Object object = this.handler.invoke(this, methods[0], arrobject);
// 返回调用结果
return (String)object;
}
public Object $echo(Object object) {
Object[] arrobject = new Object[]{object};
Object object2 = this.handler.invoke(this, methods[1], arrobject);
return object2;
}
}
复制代码InvokerInvocationHandler中的invoker成员变量为MockClusterInvoker，它来处理服务降级的逻辑。
public class InvokerInvocationHandler implements InvocationHandler {
private final Invoker<?> invoker;
public InvokerInvocationHandler(Invoker<?> handler) {
this.invoker = handler;
}
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
String methodName = method.getName();
Class<?>[] parameterTypes = method.getParameterTypes();
// 拦截定义在 Object 类中的方法（未被子类重写），比如 wait/notify
if (method.getDeclaringClass() == Object.class) {
return method.invoke(invoker, args);
}
// 如果 toString、hashCode 和 equals 等方法被子类重写了，这里也直接调用
if ("toString".equals(methodName) && parameterTypes.length == 0) {
return invoker.toString();
}
if ("hashCode".equals(methodName) && parameterTypes.length == 0) {
return invoker.hashCode();
}
if ("equals".equals(methodName) && parameterTypes.length == 1) {
return invoker.equals(args[0]);
}
// 将 method 和 args 封装到 RpcInvocation 中，并执行后续的调用
return invoker.invoke(new RpcInvocation(method, args)).recreate();
}
}
复制代码在MockClusterInvoker中，从no mock(正常情况),force:direct mock(屏蔽),fail-mock(容错)三种情况我们也可以看出,普通情况是直接调用,容错的情况是调用失败后,返回一个设置的值.而屏蔽就很暴力了,直接连调用都不调用,就直接返回一个之前设置的值.
从下面的注释中可以看出，如果没有降级，会执行this.invoker.invoke(invocation)方法进行远程调动，默认类是FailoverClusterInvoker，它会执行集群模块的逻辑，主要是调用Directory#list方法获取所有该服务提供者的地址列表，然后将多个服务提供者聚合成一个Invoker， 并调用 Router 的 route 方法进行路由，过滤掉不符合路由规则的 Invoker。当 FailoverClusterInvoker 拿到 Directory 返回的 Invoker 列表后，它会通过 LoadBalance 从 Invoker 列表中选择一个 Invoker。最后 FailoverClusterInvoker 会将参数传给 LoadBalance 选择出的 Invoker 实例的 invoke 方法，进行真正的远程调用。
public class MockClusterInvoker<T> implements Invoker<T> {
private final Invoker<T> invoker;
public Result invoke(Invocation invocation) throws RpcException {
Result result = null;
// 获取 mock 配置值
String value = directory.getUrl().getMethodParameter(invocation.getMethodName(), Constants.MOCK_KEY, Boolean.FALSE.toString()).trim();
if (value.length() == 0 || value.equalsIgnoreCase("false")) {
// 无 mock 逻辑，直接调用其他 Invoker 对象的 invoke 方法，
// 比如 FailoverClusterInvoker
result = this.invoker.invoke(invocation);
} else if (value.startsWith("force")) {
// force:xxx 直接执行 mock 逻辑，不发起远程调用
result = doMockInvoke(invocation, null);
} else {
// fail:xxx 表示消费方对调用服务失败后，再执行 mock 逻辑，不抛出异常
try {
// 调用其他 Invoker 对象的 invoke 方法
result = this.invoker.invoke(invocation);
} catch (RpcException e) {
if (e.isBiz()) {
throw e;
} else {
// 调用失败，执行 mock 逻辑
result = doMockInvoke(invocation, e);
}
}
}
return result;
}
// 省略其他方法
}
复制代码