一、前沿
前面分析了 Dubbo SPI、provider服务导出、consumer服务引用的源码，接下来就是分析服务调用的过程了，在学习服务调用之前，最好先学习 集群容错 模块，它包含四个部分，分别是 服务目录 Directory、服务路由 Router、集群 Cluster 和 负载均衡 LoadBalance
Dubbo 服务调用过程比较复杂，总体上来说包含如下步骤：
发送请求 -> 编解码 -> 服务降级 -> 过滤器链处理 -> 序列化 -> 线程派发 -> 响应请求 -> 结果编码 -> 结果解码
本文将着重讲解 发送请求、编解码、线程派发和响应请求部分代码，至于服务降级、过滤器链和序列化部分代码后续再讲解
二、服务调用过程
我们先通过下面一张图了解 Dubbo 服务调用过程，如图：
从上述图中我们可以看出调用过程：
1）、consumer 通过代理对象 Proxy 发起远程调用，接着通过网络客户端 Client 将编码后的请求发送给 provider，也就是 Server 端
2）、provider 收到请求后先对数据包解码，然后将解码后的请求发送至分发器 Dispatcher
3）、Dispatcher 分发器将请求派发到指定的线程池上，最后由线程池调用具体的服务
图中的过程虽然没有描述provider响应数据的发送与consumer接收数据的过程，但是后面会通过源码来详细分析
三、服务调用源码
3.1 consumer调用方式
Dubbo 支持同步和异步两种调用方式，其中异步调用还可细分为“有返回值”的异步调用和“无返回值”的异步调用。所谓“无返回值”异步调用是指consumer只管调用，不需要关心调用结果，此时 Dubbo 会直接返回一个空的 AsyncRpcResult。若要使用异步特性，需要consumer方手动进行配置。默认情况下，Dubbo 使用同步调用方式
我们以 dubbo-demo 中的 consumer debug运行调试代码查看服务调用过程，如下图：
上述debug调用栈中可以看出调用过程，main（Application） -> sayHello（proxy0）-> invoke（InvokerInvocationHandler），在 consumer服务引用 中我们知道 proxy0.class 代理类的代码如下：
package org.apache.dubbo.common.bytecode;
import com.alibaba.dubbo.rpc.service.EchoService;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import org.apache.dubbo.demo.DemoService;
public class proxy0
implements ClassGenerator.DC, EchoService, DemoService
{
public static Method[] methods;
private InvocationHandler handler;
public String sayHello(String paramString)
{
// 将运行时参数存储到数组中
Object[] arrayOfObject = new Object[1];
arrayOfObject[0] = paramString;
// 调用 InvocationHandler 实现类的 invoke 方法得到调用结果
Object localObject = this.handler.invoke(this, methods[0], arrayOfObject);
// 返回调用结果
return (String)localObject;
}
// 回声测试方法
public Object $echo(Object paramObject)
{
Object[] arrayOfObject = new Object[1];
arrayOfObject[0] = paramObject;
Object localObject = this.handler.invoke(this, methods[1], arrayOfObject);
return (Object)localObject;
}
public proxy0()
{
}
public proxy0(InvocationHandler paramInvocationHandler)
{
this.handler = paramInvocationHandler;
}
}proxy0 实现了 DemoService，实现了 sayHello 方法具体业务，DemoService 调用 sayHello 的时会先调用了 proxy0（DemoService的实现类） 的 sayHello，而 sayHello 则会调用 InvokerInvocationHandler 的 invoke 方法
接下来看 InvokerInvocationHandler 源码，如下：
package org.apache.dubbo.rpc.proxy;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import org.apache.dubbo.common.logger.Logger;
import org.apache.dubbo.common.logger.LoggerFactory;
import org.apache.dubbo.rpc.Invoker;
import org.apache.dubbo.rpc.RpcInvocation;
/**
* InvokerHandler
*/
public class InvokerInvocationHandler implements InvocationHandler {
private static final Logger logger = LoggerFactory.getLogger(InvokerInvocationHandler.class);
private final Invoker<?> invoker;
public InvokerInvocationHandler(Invoker<?> handler) {
this.invoker = handler;
}
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
// 获取方法名
String methodName = method.getName();
// 获取方法参数
Class<?>[] parameterTypes = method.getParameterTypes();
if (method.getDeclaringClass() == Object.class) {
// 定义在 Object 类中的方法（未被子类重写），比如 wait/notify等，直接调用
return method.invoke(invoker, args);
}
// 如果 toString、hashCode 和 equals 等方法被子类重写了，这里也直接调用
if ("toString".equals(methodName) && parameterTypes.length == 0) {
return invoker.toString();
}
if ("hashCode".equals(methodName) && parameterTypes.length == 0) {
return invoker.hashCode();
}
if ("equals".equals(methodName) && parameterTypes.length == 1) {
return invoker.equals(args[0]);
}
// 将 method 和 args 封装到 RpcInvocation 中，调用 MockClusterInvoker 的 invoke方法
// 最后调用 recreate 方法返回结果数据
return invoker.invoke(new RpcInvocation(method, args)).recreate();
}
}InvokerInvocationHandler 的 invoke 方法中逻辑很简单，注释已经说得很明白，这里就不在说了
下面我们看最终调用的 Invoker 的 invoke 方法，代码如下：
// 1、MockClusterInvoker 的 invoke 方法
@Override
public Result invoke(Invocation invocation) throws RpcException {
Result result = null;
// 获取 mock 配置
String value = directory.getUrl().getMethodParameter(invocation.getMethodName(), MOCK_KEY, Boolean.FALSE.toString()).trim();
if (value.length() == 0 || "false".equalsIgnoreCase(value)) {
//no mock
// 没有 mock 配置 或者 mock 为 false，直接调用 AbstractClusterInvoker 的 invoke 方法
result = this.invoker.invoke(invocation);
} else if (value.startsWith("force")) {
if (logger.isWarnEnabled()) {
logger.warn("force-mock: " + invocation.getMethodName() + " force-mock enabled , url : " + directory.getUrl());
}
//force:direct mock
// force:xxx 直接执行 mock 逻辑，不发起远程调用，这里就是服务降级策略
result = doMockInvoke(invocation, null);
} else {
//fail-mock
// consumer 调用失败后，再执行 mock 逻辑，不抛出异常
try {
result = this.invoker.invoke(invocation);
//fix:#4585
if(result.getException() != null && result.getException() instanceof RpcException){
RpcException rpcException= (RpcException)result.getException();
if(rpcException.isBiz()){
throw rpcException;
}else {
// 调用失败，执行 mock 逻辑，即服务降级
result = doMockInvoke(invocation, rpcException);
}
}
} catch (RpcException e) {
if (e.isBiz()) {
throw e;
}
if (logger.isWarnEnabled()) {
logger.warn("fail-mock: " + invocation.getMethodName() + " fail-mock enabled , url : " + directory.getUrl(), e);
}
result = doMockInvoke(invocation, e);
}
}
return result;
}
// 2、AbstractClusterInvoker 的 invoke 方法
@Override
public Result invoke(final Invocation invocation) throws RpcException {
// 校验 Invoker 是否销毁了
checkWhetherDestroyed();
// binding attachments into invocation.
// 将 RpcContext 中的 attachments 参数绑定到 RpcInvocation 中
Map<String, String> contextAttachments = RpcContext.getContext().getAttachments();
if (contextAttachments != null && contextAttachments.size() != 0) {
((RpcInvocation) invocation).addAttachments(contextAttachments);
}
// 获取 Invoker 列表
List<Invoker<T>> invokers = list(invocation);
// 初始化负载均衡策略，RandomLoadBalance 为默认的负载均衡策略
// invokers 不为空时，则取第一个 invoker 的 url 中 loadbalance 参数设置的负载均衡策略
LoadBalance loadbalance = initLoadBalance(invokers, invocation);
// 如果请求是异步的，需要设置 id 参数到 RpcInvocation 的 Attachment 中
RpcUtils.attachInvocationIdIfAsync(getUrl(), invocation);
// 调用具体的 Invoker 实现类，dubbo 中默认是 FailoverClusterInvoker，故这里调用 FailoverClusterInvoker 的 doInvoke 方法
return doInvoke(invocation, invokers, loadbalance);
}FailoverClusterInvoker 的 doInvoke 方法在 dubbo集群 中讲解了，这里直接从 FailoverClusterInvoker 的 doInvoke 方法调用目标 Invoker 的 invoke 方法讲起，debug调试栈如下图所示：
栈信息内容如下：
doInvoke:96, DubboInvoker (org.apache.dubbo.rpc.protocol.dubbo)
invoke:155, AbstractInvoker (org.apache.dubbo.rpc.protocol)
invoke:52, AsyncToSyncInvoker (org.apache.dubbo.rpc.protocol)
invoke:92, MonitorFilter (org.apache.dubbo.monitor.support)
invoke:82, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:54, FutureFilter (org.apache.dubbo.rpc.protocol.dubbo.filter)
invoke:82, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:58, ConsumerContextFilter (org.apache.dubbo.rpc.filter)
invoke:82, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:157, ProtocolFilterWrapper$CallbackRegistrationInvoker (org.apache.dubbo.rpc.protocol)
invoke:78, ListenerInvokerWrapper (org.apache.dubbo.rpc.listener)
invoke:56, InvokerWrapper (org.apache.dubbo.rpc.protocol)
doInvoke:91, FailoverClusterInvoker (org.apache.dubbo.rpc.cluster.support)
invoke:256, AbstractClusterInvoker (org.apache.dubbo.rpc.cluster.support)
invoke:81, MockClusterInvoker (org.apache.dubbo.rpc.cluster.support.wrapper)
invoke:61, InvokerInvocationHandler (org.apache.dubbo.rpc.proxy)
sayHello:-1, proxy0 (org.apache.dubbo.common.bytecode)
main:27, Application (org.apache.dubbo.demo.consumer)调用目标 Invoker 的 invoke 方法过程源码如下：
// 1、InvokerWrapper 的 invoke 方法
@Override
public Result invoke(Invocation invocation) throws RpcException {
// 调用 ListenerInvokerWrapper 的 invoke 方法
return invoker.invoke(invocation);
}
// 2、ListenerInvokerWrapper 的 invoke
@Override
public Result invoke(Invocation invocation) throws RpcException {
// 调用 ProtocolFilterWrapper 的回调 CallbackRegistrationInvoker 中的 invoke 方法
return invoker.invoke(invocation);
}
/**
* Register callback for each filter may be better, just like {@link java.util.concurrent.CompletionStage}, each callback
* registration generates a new CompletionStage whose status is determined by the original CompletionStage.
*
* If bridging status between filters is proved to not has significant performance drop, consider revert to the following commit:
* https://github.com/apache/dubbo/pull/4127
*/
static class CallbackRegistrationInvoker<T> implements Invoker<T> {
private final Invoker<T> filterInvoker;
private final List<Filter> filters;
public CallbackRegistrationInvoker(Invoker<T> filterInvoker, List<Filter> filters) {
this.filterInvoker = filterInvoker;
this.filters = filters;
}
// 3、ProtocolFilterWrapper 的内部类 CallbackRegistrationInvoker 中的 invoke 方法
@Override
public Result invoke(Invocation invocation) throws RpcException {
// 调用 ProtocolFilterWrapper 的 lastInvoker 中的 invoke 方法
Result asyncResult = filterInvoker.invoke(invocation);
asyncResult = asyncResult.whenCompleteWithContext((r, t) -> {
for (int i = filters.size() - 1; i >= 0; i--) {
Filter filter = filters.get(i);
// onResponse callback
if (filter instanceof ListenableFilter) {
Filter.Listener listener = ((ListenableFilter) filter).listener();
if (listener != null) {
if (t == null) {
listener.onResponse(r, filterInvoker, invocation);
} else {
listener.onError(t, filterInvoker, invocation);
}
}
} else {
filter.onResponse(r, filterInvoker, invocation);
}
}
});
return asyncResult;
}
@Override
public Class<T> getInterface() {
return filterInvoker.getInterface();
}
@Override
public URL getUrl() {
return filterInvoker.getUrl();
}
@Override
public boolean isAvailable() {
return filterInvoker.isAvailable();
}
@Override
public void destroy() {
filterInvoker.destroy();
}
}
private static <T> Invoker<T> buildInvokerChain(final Invoker<T> invoker, String key, String group) {
Invoker<T> last = invoker;
List<Filter> filters = ExtensionLoader.getExtensionLoader(Filter.class).getActivateExtension(invoker.getUrl(), key, group);
if (!filters.isEmpty()) {
for (int i = filters.size() - 1; i >= 0; i--) {
final Filter filter = filters.get(i);
final Invoker<T> next = last;
last = new Invoker<T>() {
@Override
public Class<T> getInterface() {
return invoker.getInterface();
}
@Override
public URL getUrl() {
return invoker.getUrl();
}
@Override
public boolean isAvailable() {
return invoker.isAvailable();
}
// 4、ProtocolFilterWrapper 的 Invoker 中的 invoke 方法
@Override
public Result invoke(Invocation invocation) throws RpcException {
Result asyncResult;
try {
// 按照filter中设置的 order 顺序，order 越小优先级越高，order 默认是 0，顺序调用所有 Filter 的 invoke 方法
asyncResult = filter.invoke(next, invocation);
} catch (Exception e) {
// onError callback
if (filter instanceof ListenableFilter) {
Filter.Listener listener = ((ListenableFilter) filter).listener();
if (listener != null) {
listener.onError(e, invoker, invocation);
}
}
throw e;
}
return asyncResult;
}
@Override
public void destroy() {
invoker.destroy();
}
@Override
public String toString() {
return invoker.toString();
}
};
}
}
return new CallbackRegistrationInvoker<>(last, filters);
}
// 5、ConsumerContextFilter 的 invoke 方法
@Override
public Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException {
RpcContext.getContext()
.setInvoker(invoker)
.setInvocation(invocation)
.setLocalAddress(NetUtils.getLocalHost(), 0)
.setRemoteAddress(invoker.getUrl().getHost(),
invoker.getUrl().getPort());
if (invocation instanceof RpcInvocation) {
((RpcInvocation) invocation).setInvoker(invoker);
}
try {
RpcContext.removeServerContext();
// 调用 ProtocolFilterWrapper 的 Invoker 的 invoke 方法
return invoker.invoke(invocation);
} finally {
RpcContext.removeContext();
}
}
// 6、FutureFilter 的 invoke 方法
@Override
public Result invoke(final Invoker<?> invoker, final Invocation invocation) throws RpcException {
fireInvokeCallback(invoker, invocation);
// need to configure if there's return value before the invocation in order to help invoker to judge if it's
// necessary to return future.
// 调用 ProtocolFilterWrapper 的 Invoker 的 invoke 方法
return invoker.invoke(invocation);
}
/**
* The invocation interceptor,it will collect the invoke data about this invocation and send it to monitor center
*
* @param invoker service
* @param invocation invocation.
* @return {@link Result} the invoke result
* @throws RpcException
*/
// 7、MonitorFilter 的 invoke 方法
@Override
public Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException {
if (invoker.getUrl().hasParameter(MONITOR_KEY)) {
invocation.setAttachment(MONITOR_FILTER_START_TIME, String.valueOf(System.currentTimeMillis()));
getConcurrent(invoker, invocation).incrementAndGet(); // count up
}
// 调用 AsyncToSyncInvoker 的 invoke 方法
return invoker.invoke(invocation); // proceed invocation chain
}
// 8、AsyncToSyncInvoker 的 invoke 方法
@Override
public Result invoke(Invocation invocation) throws RpcException {
// 调用 AbstractInvoker 的 invoke 方法
Result asyncResult = invoker.invoke(invocation);
try {
if (InvokeMode.SYNC == ((RpcInvocation) invocation).getInvokeMode()) {
asyncResult.get(Integer.MAX_VALUE, TimeUnit.MILLISECONDS);
}
} catch (InterruptedException e) {
throw new RpcException("Interrupted unexpectedly while waiting for remoting result to return! method: " + invocation.getMethodName() + ", provider: " + getUrl() + ", cause: " + e.getMessage(), e);
} catch (ExecutionException e) {
Throwable t = e.getCause();
if (t instanceof TimeoutException) {
throw new RpcException(RpcException.TIMEOUT_EXCEPTION, "Invoke remote method timeout. method: " + invocation.getMethodName() + ", provider: " + getUrl() + ", cause: " + e.getMessage(), e);
} else if (t instanceof RemotingException) {
throw new RpcException(RpcException.NETWORK_EXCEPTION, "Failed to invoke remote method: " + invocation.getMethodName() + ", provider: " + getUrl() + ", cause: " + e.getMessage(), e);
}
} catch (Throwable e) {
throw new RpcException(e.getMessage(), e);
}
return asyncResult;
}
// 9、调用 AbstractInvoker 的 invoke 方法
@Override
public Result invoke(Invocation inv) throws RpcException {
// if invoker is destroyed due to address refresh from registry, let's allow the current invoke to proceed
if (destroyed.get()) {
// 判断invoker是否被销毁了
logger.warn("Invoker for service " + this + " on consumer " + NetUtils.getLocalHost() + " is destroyed, "
+ ", dubbo version is " + Version.getVersion() + ", this invoker should not be used any longer");
}
RpcInvocation invocation = (RpcInvocation) inv;
// 设置invoker
invocation.setInvoker(this);
if (CollectionUtils.isNotEmptyMap(attachment)) {
// RpcInvocation 中设置 attachment 中的参数
invocation.addAttachmentsIfAbsent(attachment);
}
Map<String, String> contextAttachments = RpcContext.getContext().getAttachments();
if (CollectionUtils.isNotEmptyMap(contextAttachments)) {
/**
* invocation.addAttachmentsIfAbsent(context){@link RpcInvocation#addAttachmentsIfAbsent(Map)}should not be used here,
* because the {@link RpcContext#setAttachment(String, String)} is passed in the Filter when the call is triggered
* by the built-in retry mechanism of the Dubbo. The attachment to update RpcContext will no longer work, which is
* a mistake in most cases (for example, through Filter to RpcContext output traceId and spanId and other information).
*/
// RpcInvocation 中设置 contextAttachments 中的参数
invocation.addAttachments(contextAttachments);
}
// 设置同步、异步有返回值或者异步无返回值类型
invocation.setInvokeMode(RpcUtils.getInvokeMode(url, invocation));
// 如果是异步，RpcInvocation 中设置 id 参数
RpcUtils.attachInvocationIdIfAsync(getUrl(), invocation);
try {
// 调用具体 Invoker 实现，我们使用的 dubbo 协议，故这里调用的是 DubboInvoker 的 doInvoke 方法
return doInvoke(invocation);
} catch (InvocationTargetException e) { // biz exception
Throwable te = e.getTargetException();
if (te == null) {
return AsyncRpcResult.newDefaultAsyncResult(null, e, invocation);
} else {
if (te instanceof RpcException) {
((RpcException) te).setCode(RpcException.BIZ_EXCEPTION);
}
return AsyncRpcResult.newDefaultAsyncResult(null, te, invocation);
}
} catch (RpcException e) {
if (e.isBiz()) {
return AsyncRpcResult.newDefaultAsyncResult(null, e, invocation);
} else {
throw e;
}
} catch (Throwable e) {
return AsyncRpcResult.newDefaultAsyncResult(null, e, invocation);
}
}最后的实现逻辑在 DubboInvoker 的 doInvoke 方法，代码如下：
// DubboInvoker 的 doInvoke 方法
@Override
protected Result doInvoke(final Invocation invocation) throws Throwable {
RpcInvocation inv = (RpcInvocation) invocation;
final String methodName = RpcUtils.getMethodName(invocation);
// 设置 path 和 version 到 attachment 中
inv.setAttachment(PATH_KEY, getUrl().getPath());
inv.setAttachment(VERSION_KEY, version);
ExchangeClient currentClient;
// 获取 client
if (clients.length == 1) {
currentClient = clients[0];
} else {
currentClient = clients[index.getAndIncrement() % clients.length];
}
try {
// isOneway 为 true，表示“单向”通信
boolean isOneway = RpcUtils.isOneway(getUrl(), invocation);
int timeout = getUrl().getMethodPositiveParameter(methodName, TIMEOUT_KEY, DEFAULT_TIMEOUT);
if (isOneway) {
// 异步无返回值
boolean isSent = getUrl().getMethodParameter(methodName, Constants.SENT_KEY, false);
// 发送请求
currentClient.send(inv, isSent);
// 直接返回空值的 AsyncRpcResult
return AsyncRpcResult.newDefaultAsyncResult(invocation);
} else {
// 异步有返回值
AsyncRpcResult asyncRpcResult = new AsyncRpcResult(inv);
// 发送请求，并得到一个 DefaultFuture（继承 CompletableFuture） 实例
CompletableFuture<Object> responseFuture = currentClient.request(inv, timeout);
// 关注返回结果
asyncRpcResult.subscribeTo(responseFuture);
// save for 2.6.x compatibility, for example, TraceFilter in Zipkin uses com.alibaba.xxx.FutureAdapter
FutureContext.getContext().setCompatibleFuture(responseFuture);
return asyncRpcResult;
}
} catch (TimeoutException e) {
throw new RpcException(RpcException.TIMEOUT_EXCEPTION, "Invoke remote method timeout. method: " + invocation.getMethodName() + ", provider: " + getUrl() + ", cause: " + e.getMessage(), e);
} catch (RemotingException e) {
throw new RpcException(RpcException.NETWORK_EXCEPTION, "Failed to invoke remote method: " + invocation.getMethodName() + ", provider: " + getUrl() + ", cause: " + e.getMessage(), e);
}
}上述主要讲述了 dubbo 对同步请求和异步请求调用的处理逻辑，如下：
1）、同步请求时调用返回一个 DefaultFuture ，框架立刻直接调用 DefaultFuture 的 get 方法获取请求结果
2）、异步无返回结果时，异步请求时立即返回一个空值的 AsyncRpcResult 请求结果
3）、异步有返回结果时，异步请求时调用返回一个 DefaultFuture ，由用户线程调用 DefaultFuture 的 get 方法获取请求结果
接着我们看下存储请求返回结果的 DefaultFuture 的源码，如下：
/**
* DefaultFuture.
*/
public class DefaultFuture extends CompletableFuture<Object> {
private static final Logger logger = LoggerFactory.getLogger(DefaultFuture.class);
private static final Map<Long, Channel> CHANNELS = new ConcurrentHashMap<>();
private static final Map<Long, DefaultFuture> FUTURES = new ConcurrentHashMap<>();
public static final Timer TIME_OUT_TIMER = new HashedWheelTimer(
new NamedThreadFactory("dubbo-future-timeout", true),
30,
TimeUnit.MILLISECONDS);
// invoke id.
private final Long id;
private final Channel channel;
private final Request request;
private final int timeout;
private final long start = System.currentTimeMillis();
private volatile long sent;
private Timeout timeoutCheckTask;
private DefaultFuture(Channel channel, Request request, int timeout) {
this.channel = channel;
this.request = request;
// 获取请求 id，这个 id 相当重要，它标记是哪个请求的结果
this.id = request.getId();
// 超时时间
this.timeout = timeout > 0 ? timeout : channel.getUrl().getPositiveParameter(TIMEOUT_KEY, DEFAULT_TIMEOUT);
// put into waiting map.
// 请求对应的异步结果存储到map中，key->requestId, value->DefaultFuture
FUTURES.put(id, this);
// 请求对应的请求通道存储到map中，key->requestId, value->Channel
CHANNELS.put(id, channel);
}
/**
* check time out of the future
* 请求返回的超时检查
*/
private static void timeoutCheck(DefaultFuture future) {
TimeoutCheckTask task = new TimeoutCheckTask(future.getId());
future.timeoutCheckTask = TIME_OUT_TIMER.newTimeout(task, future.getTimeout(), TimeUnit.MILLISECONDS);
}
/**
* init a DefaultFuture
* 1.init a DefaultFuture
* 2.timeout check
*
* @param channel channel
* @param request the request
* @param timeout timeout
* @return a new DefaultFuture
*/
public static DefaultFuture newFuture(Channel channel, Request request, int timeout) {
final DefaultFuture future = new DefaultFuture(channel, request, timeout);
// timeout check
timeoutCheck(future);
return future;
}
public static DefaultFuture getFuture(long id) {
// 获取请求对应的请求结果
return FUTURES.get(id);
}
public static boolean hasFuture(Channel channel) {
return CHANNELS.containsValue(channel);
}
public static void sent(Channel channel, Request request) {
DefaultFuture future = FUTURES.get(request.getId());
if (future != null) {
// 获取请求时间
future.doSent();
}
}
/**
* close a channel when a channel is inactive
* directly return the unfinished requests.
*
* @param channel channel to close
*/
public static void closeChannel(Channel channel) {
for (Map.Entry<Long, Channel> entry : CHANNELS.entrySet()) {
if (channel.equals(entry.getValue())) {
DefaultFuture future = getFuture(entry.getKey());
if (future != null && !future.isDone()) {
Response disconnectResponse = new Response(future.getId());
disconnectResponse.setStatus(Response.CHANNEL_INACTIVE);
disconnectResponse.setErrorMessage("Channel " +
channel +
" is inactive. Directly return the unFinished request : " +
future.getRequest());
DefaultFuture.received(channel, disconnectResponse);
}
}
}
}
public static void received(Channel channel, Response response) {
// 接收请求返回结果
received(channel, response, false);
}
public static void received(Channel channel, Response response, boolean timeout) {
try {
DefaultFuture future = FUTURES.remove(response.getId());
if (future != null) {
Timeout t = future.timeoutCheckTask;
if (!timeout) {
// decrease Time
// 取消超时检查任务
t.cancel();
}
// 执行接收请求结果处理
future.doReceived(response);
} else {
logger.warn("The timeout response finally returned at "
+ (new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS").format(new Date()))
+ ", response " + response
+ (channel == null ? "" : ", channel: " + channel.getLocalAddress()
+ " -> " + channel.getRemoteAddress()));
}
} finally {
CHANNELS.remove(response.getId());
}
}
@Override
public boolean cancel(boolean mayInterruptIfRunning) {
Response errorResult = new Response(id);
errorResult.setStatus(Response.CLIENT_ERROR);
errorResult.setErrorMessage("request future has been canceled.");
this.doReceived(errorResult);
FUTURES.remove(id);
CHANNELS.remove(id);
return true;
}
public void cancel() {
this.cancel(true);
}
private void doReceived(Response res) {
if (res == null) {
// 返回结果为null，抛出异常
throw new IllegalStateException("response cannot be null");
}
if (res.getStatus() == Response.OK) {
// 请求结果状态成功，处理结果值
this.complete(res.getResult());
} else if (res.getStatus() == Response.CLIENT_TIMEOUT || res.getStatus() == Response.SERVER_TIMEOUT) {
// 客户端请求超时或者服务端响应超时，抛出 TimeoutException
this.completeExceptionally(new TimeoutException(res.getStatus() == Response.SERVER_TIMEOUT, channel, res.getErrorMessage()));
} else {
// 请求中的其他异常，抛出 RemotingException
this.completeExceptionally(new RemotingException(channel, res.getErrorMessage()));
}
}
private long getId() {
return id;
}
private Channel getChannel() {
return channel;
}
private boolean isSent() {
return sent > 0;
}
public Request getRequest() {
return request;
}
private int getTimeout() {
return timeout;
}
private void doSent() {
sent = System.currentTimeMillis();
}
private String getTimeoutMessage(boolean scan) {
long nowTimestamp = System.currentTimeMillis();
return (sent > 0 ? "Waiting server-side response timeout" : "Sending request timeout in client-side")
+ (scan ? " by scan timer" : "") + ". start time: "
+ (new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS").format(new Date(start))) + ", end time: "
+ (new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS").format(new Date())) + ","
+ (sent > 0 ? " client elapsed: " + (sent - start)
+ " ms, server elapsed: " + (nowTimestamp - sent)
: " elapsed: " + (nowTimestamp - start)) + " ms, timeout: "
+ timeout + " ms, request: " + request + ", channel: " + channel.getLocalAddress()
+ " -> " + channel.getRemoteAddress();
}
private static class TimeoutCheckTask implements TimerTask {
private final Long requestID;
TimeoutCheckTask(Long requestID) {
this.requestID = requestID;
}
@Override
public void run(Timeout timeout) {
DefaultFuture future = DefaultFuture.getFuture(requestID);
if (future == null || future.isDone()) {
// future 为null或者请求完成，直接返回
return;
}
// create exception response.
Response timeoutResponse = new Response(future.getId());
// set timeout status.
timeoutResponse.setStatus(future.isSent() ? Response.SERVER_TIMEOUT : Response.CLIENT_TIMEOUT);
timeoutResponse.setErrorMessage(future.getTimeoutMessage(true));
// handle response.
DefaultFuture.received(future.getChannel(), timeoutResponse, true);
}
}
}到这里关于 dubbo 的调用方式的代码逻辑就分析完了，下面来分析请求数据的发送与接收，以及响应数据的发送与接收过程
3.2 consumer 发送请求
3.2.1 发送请求
我们debug调试看具体的请求过程，如下图：
请求栈具体内容：
send:147, NettyChannel (org.apache.dubbo.remoting.transport.netty4)
send:177, AbstractClient (org.apache.dubbo.remoting.transport)
send:53, AbstractPeer (org.apache.dubbo.remoting.transport)
request:118, HeaderExchangeChannel (org.apache.dubbo.remoting.exchange.support.header)
request:84, HeaderExchangeClient (org.apache.dubbo.remoting.exchange.support.header)
request:80, ReferenceCountExchangeClient (org.apache.dubbo.rpc.protocol.dubbo)
doInvoke:105, DubboInvoker (org.apache.dubbo.rpc.protocol.dubbo)
invoke:163, AbstractInvoker (org.apache.dubbo.rpc.protocol)
invoke:54, AsyncToSyncInvoker (org.apache.dubbo.rpc.protocol)
invoke:94, MonitorFilter (org.apache.dubbo.monitor.support)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:56, FutureFilter (org.apache.dubbo.rpc.protocol.dubbo.filter)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:60, ConsumerContextFilter (org.apache.dubbo.rpc.filter)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:161, ProtocolFilterWrapper$CallbackRegistrationInvoker (org.apache.dubbo.rpc.protocol)
invoke:80, ListenerInvokerWrapper (org.apache.dubbo.rpc.listener)
invoke:58, InvokerWrapper (org.apache.dubbo.rpc.protocol)
doInvoke:91, FailoverClusterInvoker (org.apache.dubbo.rpc.cluster.support)
invoke:256, AbstractClusterInvoker (org.apache.dubbo.rpc.cluster.support)
invoke:81, MockClusterInvoker (org.apache.dubbo.rpc.cluster.support.wrapper)
invoke:61, InvokerInvocationHandler (org.apache.dubbo.rpc.proxy)
sayHello:-1, proxy0 (org.apache.dubbo.common.bytecode)
main:27, Application (org.apache.dubbo.demo.consumer)从上图可以看出，经过多次调用后，才将请求数据送至 Netty 的 NioClientSocketChannel。这样做的原因是通过 Exchange 层为框架引入 Request 和 Response 语义，这一点会在接下来的源码分析过程中会看到。首先从 ReferenceCountExchangeClient 的源码分析，整个过程代码如下：
// ReferenceCountExchangeClient 内部仅实现了一个引用计数的功能,它定义了一个引用计数变量 referenceCount，
// 每当该对象被引用一次 referenceCount 都会进行自增。每当 close 方法被调用时，referenceCount 进行自减。
// 1、ReferenceCountExchangeClient 的 request 方法
@Override
public CompletableFuture<Object> request(Object request, int timeout) throws RemotingException {
// 调用 HeaderExchangeClient 的 request 方法
return client.request(request, timeout);
}
// HeaderExchangeClient 中主要封装了一些关于心跳检测逻辑
// 2、HeaderExchangeClient 的 request 方法
@Override
public CompletableFuture<Object> request(Object request, int timeout) throws RemotingException {
// 调用 HeaderExchangeChannel 的 request 方法
return channel.request(request, timeout);
}
// 3、HeaderExchangeChannel 的 request 方法
@Override
public CompletableFuture<Object> request(Object request, int timeout) throws RemotingException {
if (closed) {
throw new RemotingException(this.getLocalAddress(), null, "Failed to send request " + request + ", cause: The channel " + this + " is closed!");
}
// create request.
// 创建 org.apache.dubbo.remoting.exchange.Request 对象
Request req = new Request();
req.setVersion(Version.getProtocolVersion());
// 设置双向通信标志为 true
req.setTwoWay(true);
// 这里的 request 变量类型为 RpcInvocation
req.setData(request);
// 创建 DefaultFuture 对象
DefaultFuture future = DefaultFuture.newFuture(channel, req, timeout);
try {
// 调用 AbstractPeer 的 send 方法发送请求
channel.send(req);
} catch (RemotingException e) {
future.cancel();
throw e;
}
// 返回 DefaultFuture
return future;
}
// 4、AbstractPeer 的 send 方法
@Override
public void send(Object message) throws RemotingException {
// 调用 AbstractClient 的 send 发送请求
send(message, url.getParameter(Constants.SENT_KEY, false));
}
// 5、AbstractClient 的 send 方法
@Override
public void send(Object message, boolean sent) throws RemotingException {
if (needReconnect && !isConnected()) {
connect();
}
// 获取 Channel，getChannel 是一个抽象方法，具体由子类实现，dubbo默认使用的是 netty，
// 因此这里调用的是 org.apache.dubbo.remoting.transport.netty4.NettyClient 的 getChannel 方法
Channel channel = getChannel();
//TODO Can the value returned by getChannel() be null? need improvement.
if (channel == null || !channel.isConnected()) {
throw new RemotingException(this, "message can not send, because channel is closed . url:" + getUrl());
}
// 调用 NettyChannel 的 send 方法发送请求
channel.send(message, sent);
}
// 6、NettyClient 的 getChannel 方法
@Override
protected org.apache.dubbo.remoting.Channel getChannel() {
Channel c = channel;
if (c == null || !c.isActive()) {
return null;
}
// 获取或者增加一个 NettyChannel 对象
return NettyChannel.getOrAddChannel(c, getUrl(), this);
}
/**
* Send message by netty and whether to wait the completion of the send.
*
* @param message message that need send.
* @param sent whether to ack async-sent
* @throws RemotingException throw RemotingException if wait until timeout or any exception thrown by method body that surrounded by try-catch.
*/
// 7、NettyChannel 的 send 方法
@Override
public void send(Object message, boolean sent) throws RemotingException {
// whether the channel is closed
// NettyChannel 是否关闭
super.send(message, sent);
boolean success = true;
int timeout = 0;
try {
// 发送消息（包含请求和响应消息）
ChannelFuture future = channel.writeAndFlush(message);
// sent 的值源于 <dubbo:method sent="true/false" /> 中 sent 的配置值，有两种配置值：
// 1. true: 等待消息发出，消息发送失败将抛出异常
// 2. false: 不等待消息发出，将消息放入 IO 队列，即刻返回
// 默认情况下 sent = false；
if (sent) {
// wait timeout ms
timeout = getUrl().getPositiveParameter(TIMEOUT_KEY, DEFAULT_TIMEOUT);
success = future.await(timeout);
}
Throwable cause = future.cause();
if (cause != null) {
throw cause;
}
} catch (Throwable e) {
throw new RemotingException(this, "Failed to send message " + message + " to " + getRemoteAddress() + ", cause: " + e.getMessage(), e);
}
if (!success) {
// 消息发送失败时抛出异常
throw new RemotingException(this, "Failed to send message " + message + " to " + getRemoteAddress()
+ "in timeout(" + timeout + "ms) limit");
}
}
经过多层调用，到这里请求数据的发送过程就结束了。
在 Netty 中，出站数据在发出之前还需要进行编码操作，接下来我们来分析一下请求数据的编码逻辑。
3.2.2 请求编码
在分析请求编码逻辑之前，我们先来看一下 Dubbo 数据包结构，如下图：
Dubbo 数据包分为 head（消息头）和 body（消息体），作用分别如下：
head：存储一些元信息，比如魔数（Magic），数据包类型（Request/Response）、消息体长度（Data Length）等
body：存储具体的调用消息，比如方法名称，参数列表等
下面使用表格简单列举一下 head 的内容：
| 偏移量(Bit) | 字段 | 取值 |
|---|---|---|
| 0 ~ 7 | 魔数高位 | 0xda00 |
| 8 ~ 15 | 魔数低位 | 0xbb |
| 16 | 数据包类型 | 0 - Response, 1 - Request |
| 17 | 调用方式 | 仅在第16位被设为1的情况下有效，0 - 单向调用，1 - 双向调用 |
| 18 | 事件标识 | 0 - 当前数据包是请求或响应包，1 - 当前数据包是心跳包 |
| 19 ~ 23 | 序列化器编号 | 2 - Hessian2Serialization 3 - JavaSerialization 4 - CompactedJavaSerialization 6 - FastJsonSerialization 7 - NativeJavaSerialization 8 - KryoSerialization 9 - FstSerialization |
| 24 ~ 31 | 状态 | 20 - OK 30 - CLIENT_TIMEOUT 31 - SERVER_TIMEOUT 40 - BAD_REQUEST 50 - BAD_RESPONSE ...... |
| 32 ~ 95 | 请求编号 | 共8字节，运行时生成 |
| 96 ~ 127 | 消息体长度 | 运行时计算 |
了解了 Dubbo 数据包格式，接下来我们就可以分析编码过程了。调用栈信息如下图：
具体的完整调用栈信息：
encode:68, ExchangeCodec (org.apache.dubbo.remoting.exchange.codec)
encode:40, DubboCountCodec (org.apache.dubbo.rpc.protocol.dubbo)
encode:70, NettyCodecAdapter$InternalEncoder (org.apache.dubbo.remoting.transport.netty4)
write:107, MessageToByteEncoder (io.netty.handler.codec)
invokeWrite0:738, AbstractChannelHandlerContext (io.netty.channel)
invokeWrite:730, AbstractChannelHandlerContext (io.netty.channel)
write:816, AbstractChannelHandlerContext (io.netty.channel)
write:723, AbstractChannelHandlerContext (io.netty.channel)
write:304, IdleStateHandler (io.netty.handler.timeout)
invokeWrite0:738, AbstractChannelHandlerContext (io.netty.channel)
invokeWrite:730, AbstractChannelHandlerContext (io.netty.channel)
write:816, AbstractChannelHandlerContext (io.netty.channel)
write:723, AbstractChannelHandlerContext (io.netty.channel)
write:106, ChannelDuplexHandler (io.netty.channel)
write:87, NettyClientHandler (org.apache.dubbo.remoting.transport.netty4)
invokeWrite0:738, AbstractChannelHandlerContext (io.netty.channel)
invokeWrite:730, AbstractChannelHandlerContext (io.netty.channel)
access$1900:38, AbstractChannelHandlerContext (io.netty.channel)
write:1081, AbstractChannelHandlerContext$AbstractWriteTask (io.netty.channel)
write:1128, AbstractChannelHandlerContext$WriteAndFlushTask (io.netty.channel)
run:1070, AbstractChannelHandlerContext$AbstractWriteTask (io.netty.channel)
safeExecute$$$capture:163, AbstractEventExecutor (io.netty.util.concurrent)
safeExecute:-1, AbstractEventExecutor (io.netty.util.concurrent)
- Async stack trace
addTask:-1, SingleThreadEventExecutor (io.netty.util.concurrent)
execute:765, SingleThreadEventExecutor (io.netty.util.concurrent)
safeExecute:1007, AbstractChannelHandlerContext (io.netty.channel)
write:825, AbstractChannelHandlerContext (io.netty.channel)
writeAndFlush:794, AbstractChannelHandlerContext (io.netty.channel)
writeAndFlush:831, AbstractChannelHandlerContext (io.netty.channel)
writeAndFlush:1071, DefaultChannelPipeline (io.netty.channel)
writeAndFlush:300, AbstractChannel (io.netty.channel)
send:136, NettyChannel (org.apache.dubbo.remoting.transport.netty4)
send:181, AbstractClient (org.apache.dubbo.remoting.transport)
send:55, AbstractPeer (org.apache.dubbo.remoting.transport)
request:124, HeaderExchangeChannel (org.apache.dubbo.remoting.exchange.support.header)
request:87, HeaderExchangeClient (org.apache.dubbo.remoting.exchange.support.header)
request:85, ReferenceCountExchangeClient (org.apache.dubbo.rpc.protocol.dubbo)
doInvoke:105, DubboInvoker (org.apache.dubbo.rpc.protocol.dubbo)
invoke:163, AbstractInvoker (org.apache.dubbo.rpc.protocol)
invoke:54, AsyncToSyncInvoker (org.apache.dubbo.rpc.protocol)
invoke:94, MonitorFilter (org.apache.dubbo.monitor.support)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:56, FutureFilter (org.apache.dubbo.rpc.protocol.dubbo.filter)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:60, ConsumerContextFilter (org.apache.dubbo.rpc.filter)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:161, ProtocolFilterWrapper$CallbackRegistrationInvoker (org.apache.dubbo.rpc.protocol)
invoke:80, ListenerInvokerWrapper (org.apache.dubbo.rpc.listener)
invoke:58, InvokerWrapper (org.apache.dubbo.rpc.protocol)
doInvoke:91, FailoverClusterInvoker (org.apache.dubbo.rpc.cluster.support)
invoke:256, AbstractClusterInvoker (org.apache.dubbo.rpc.cluster.support)
invoke:81, MockClusterInvoker (org.apache.dubbo.rpc.cluster.support.wrapper)
invoke:61, InvokerInvocationHandler (org.apache.dubbo.rpc.proxy)
sayHello:-1, proxy0 (org.apache.dubbo.common.bytecode)
main:27, Application (org.apache.dubbo.demo.consumer)
上图可知，编码逻辑入口在 ExchangeCodec 类中，代码如下：
// 1、ExchangeCodec 的 encode 方法
@Override
public void encode(Channel channel, ChannelBuffer buffer, Object msg) throws IOException {
if (msg instanceof Request) {
// Request 请求数据编码
encodeRequest(channel, buffer, (Request) msg);
} else if (msg instanceof Response) {
// Response 返回结果编码
encodeResponse(channel, buffer, (Response) msg);
} else {
super.encode(channel, buffer, msg);
}
}
// 2、ExchangeCodec 的 encodeRequest 方法，实现了 request 编码
protected void encodeRequest(Channel channel, ChannelBuffer buffer, Request req) throws IOException {
// 获取序列化方式，默认是 Hessian2Serialization
Serialization serialization = getSerialization(channel);
// header.
// 创建消息头字节数组，长度为 16
byte[] header = new byte[HEADER_LENGTH];
// set magic number.
// 设置魔数
Bytes.short2bytes(MAGIC, header);
// set request and serialization flag.
// 设置数据包类型（Request/Response）和序列化器编号
header[2] = (byte) (FLAG_REQUEST | serialization.getContentTypeId());
// 设置通信方式(单向/双向)
if (req.isTwoWay()) {
header[2] |= FLAG_TWOWAY;
}
// 设置事件标识
if (req.isEvent()) {
header[2] |= FLAG_EVENT;
}
// set request id.
// 设置请求编号id，8个字节，从第4个字节开始设置
Bytes.long2bytes(req.getId(), header, 4);
// encode request data.
// 获取 buffer 当前的写位置
int savedWriteIndex = buffer.writerIndex();
// 更新 writerIndex，为消息头预留 16 个字节的空间
buffer.writerIndex(savedWriteIndex + HEADER_LENGTH);
ChannelBufferOutputStream bos = new ChannelBufferOutputStream(buffer);
// 创建序列化器，这里默认的是 Hessian2ObjectOutput
ObjectOutput out = serialization.serialize(channel.getUrl(), bos);
if (req.isEvent()) {
// 对事件数据进行序列化操作
encodeEventData(channel, out, req.getData());
} else {
// 调用 DubboCodec 的 encodeResponseData 方法对请求数据进行序列化操作
encodeRequestData(channel, out, req.getData(), req.getVersion());
}
out.flushBuffer();
if (out instanceof Cleanable) {
((Cleanable) out).cleanup();
}
bos.flush();
bos.close();
// 获取写入的字节数，也就是消息体长度
int len = bos.writtenBytes();
// 校验消息体长度，默认支持 8M 数据，超出 8M 抛出异常
checkPayload(channel, len);
// 将消息体长度写入到消息头中
Bytes.int2bytes(len, header, 12);
// write
// 将 buffer 指针移动到 savedWriteIndex，为写消息头做准备
buffer.writerIndex(savedWriteIndex);
// 从 savedWriteIndex 下标处写入消息头
buffer.writeBytes(header); // write header.
// 设置新的 writerIndex，writerIndex = 原写下标 + 消息头长度 + 消息体长度
buffer.writerIndex(savedWriteIndex + HEADER_LENGTH + len);
}
// 3、DubboCodec 的 encodeResponseData 方法
@Override
protected void encodeRequestData(Channel channel, ObjectOutput out, Object data, String version) throws IOException {
RpcInvocation inv = (RpcInvocation) data;
// 依次序列化 dubbo version、path、version
out.writeUTF(version);
out.writeUTF(inv.getAttachment(PATH_KEY));
out.writeUTF(inv.getAttachment(VERSION_KEY));
// 序列化调用方法名
out.writeUTF(inv.getMethodName());
// 将参数类型转换为字符串，并进行序列化
out.writeUTF(ReflectUtils.getDesc(inv.getParameterTypes()));
Object[] args = inv.getArguments();
if (args != null) {
for (int i = 0; i < args.length; i++) {
// 对运行时参数进行序列化
out.writeObject(encodeInvocationArgument(channel, inv, i));
}
}
// 序列化 attachments 对象
out.writeObject(inv.getAttachments());
}上述编码过程如下：
1）、通过位运算将消息头写入到 header 数组中
2）、序列化 Request 中的 data 字段数据，序列化后的数据存储到 ChannelBuffer 中
3）、获取序列化后的数据长度 len，并校验 len 是否超出 8M，没有超出长度限制的话将 len 写入到 header 指定位置
4）、将 header 字节数组数据 写入到 ChannelBuffer 中
至此，consumer 发送请求的过程就分析完了，接下来就要分析 provider 是如何接收请求的
3.3 provider 接收请求
默认情况下 Dubbo 使用 Netty 作为底层的通信框架。Netty 检测到有数据入站后，首先会通过解码器对数据进行解码，并将解码后的数据传递给下一个入站处理器的指定方法。所以在进行后续的分析之前，我们先来看一下 request 解码过程
3.3.1 请求解码
consumer 发送请求之后，provider 会等着接收请求数据，provider 提供者 debug 调试栈如下图：
完整的调用栈信息如下：
decode:162, DecodeableRpcInvocation (org.apache.dubbo.rpc.protocol.dubbo)
decode:73, DecodeableRpcInvocation (org.apache.dubbo.rpc.protocol.dubbo)
decodeBody:149, DubboCodec (org.apache.dubbo.rpc.protocol.dubbo)
decode:82, ExchangeCodec (org.apache.dubbo.remoting.exchange.codec)
decode:52, DubboCountCodec (org.apache.dubbo.rpc.protocol.dubbo)
decode:90, NettyCodecAdapter$InternalDecoder (org.apache.dubbo.remoting.transport.netty4)
decodeRemovalReentryProtection:489, ByteToMessageDecoder (io.netty.handler.codec)
callDecode:428, ByteToMessageDecoder (io.netty.handler.codec)
channelRead:265, ByteToMessageDecoder (io.netty.handler.codec)
invokeChannelRead:362, AbstractChannelHandlerContext (io.netty.channel)
invokeChannelRead:348, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:340, AbstractChannelHandlerContext (io.netty.channel)
channelRead:1434, DefaultChannelPipeline$HeadContext (io.netty.channel)
invokeChannelRead:362, AbstractChannelHandlerContext (io.netty.channel)
invokeChannelRead:348, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:965, DefaultChannelPipeline (io.netty.channel)
read:163, AbstractNioByteChannel$NioByteUnsafe (io.netty.channel.nio)
processSelectedKey:647, NioEventLoop (io.netty.channel.nio)
processSelectedKeysOptimized:582, NioEventLoop (io.netty.channel.nio)
processSelectedKeys:499, NioEventLoop (io.netty.channel.nio)
run:461, NioEventLoop (io.netty.channel.nio)
run:884, SingleThreadEventExecutor$5 (io.netty.util.concurrent)
run:30, FastThreadLocalRunnable (io.netty.util.concurrent)
run:748, Thread (java.lang)从栈过程可以看出，解码入口在 DubboCountCodec 的 encode 方法，整个过程代码如下：
// 1、DubboCountCodec 的 decode 方法
@Override
public Object decode(Channel channel, ChannelBuffer buffer) throws IOException {
// 获取读取位置
int save = buffer.readerIndex();
// 构建 MultiMessage 对象
MultiMessage result = MultiMessage.create();
do {
// 调用 ExchangeCodec 的 decode 方法对请求解码
Object obj = codec.decode(channel, buffer);
if (Codec2.DecodeResult.NEED_MORE_INPUT == obj) {
buffer.readerIndex(save);
break;
} else {
result.addMessage(obj);
logMessageLength(obj, buffer.readerIndex() - save);
save = buffer.readerIndex();
}
} while (true);
if (result.isEmpty()) {
return Codec2.DecodeResult.NEED_MORE_INPUT;
}
if (result.size() == 1) {
return result.get(0);
}
return result;
}
// 2、ExchangeCodec 的 decode 方法
@Override
public Object decode(Channel channel, ChannelBuffer buffer) throws IOException {
int readable = buffer.readableBytes();
// 创建消息头数组
byte[] header = new byte[Math.min(readable, HEADER_LENGTH)];
// 读取消息头数据
buffer.readBytes(header);
return decode(channel, buffer, readable, header);
}
// 3、ExchangeCodec 的 decode 方法
@Override
protected Object decode(Channel channel, ChannelBuffer buffer, int readable, byte[] header) throws IOException {
// check magic number.
// 检查魔数是否相等
if (readable > 0 && header[0] != MAGIC_HIGH
|| readable > 1 && header[1] != MAGIC_LOW) {
int length = header.length;
if (header.length < readable) {
header = Bytes.copyOf(header, readable);
buffer.readBytes(header, length, readable - length);
}
for (int i = 1; i < header.length - 1; i++) {
if (header[i] == MAGIC_HIGH && header[i + 1] == MAGIC_LOW) {
buffer.readerIndex(buffer.readerIndex() - header.length + i);
header = Bytes.copyOf(header, i);
break;
}
}
// 通过 telnet 命令行发送的数据包不包含消息头，所以这里
// 调用 TelnetCodec 的 decode 方法对数据包进行解码
return super.decode(channel, buffer, readable, header);
}
// check length.
// 检测可读数据量是否少于消息头长度，若小于则立即返回 DecodeResult.NEED_MORE_INPUT
if (readable < HEADER_LENGTH) {
return DecodeResult.NEED_MORE_INPUT;
}
// get data length.
// 从消息头中获取消息体长度
int len = Bytes.bytes2int(header, 12);
// 检测消息体长度是否超出限制 8M（默认，可以通过 payload 自行设置），超出则抛出异常
checkPayload(channel, len);
int tt = len + HEADER_LENGTH;
// 检测可读的字节数是否小于实际的字节数
if (readable < tt) {
return DecodeResult.NEED_MORE_INPUT;
}
// limit input stream.
ChannelBufferInputStream is = new ChannelBufferInputStream(buffer, len);
try {
// 调用 DubboCodec 的 decodeBody 解码
return decodeBody(channel, is, header);
} finally {
if (is.available() > 0) {
try {
if (logger.isWarnEnabled()) {
logger.warn("Skip input stream " + is.available());
}
StreamUtils.skipUnusedStream(is);
} catch (IOException e) {
logger.warn(e.getMessage(), e);
}
}
}
}
// 4、DubboCodec 的 decodeBody 方法
@Override
protected Object decodeBody(Channel channel, InputStream is, byte[] header) throws IOException {
// 获取消息头中的第三个字节，并通过逻辑与运算得到序列化器编号
byte flag = header[2], proto = (byte) (flag & SERIALIZATION_MASK);
// get request id.
// 获取消息头中的 request id 值
long id = Bytes.bytes2long(header, 4);
// 通过逻辑与运算得到调用类型，0 - Response，1 - Request
if ((flag & FLAG_REQUEST) == 0) {
// decode response.
// 解码 Response
Response res = new Response(id);
if ((flag & FLAG_EVENT) != 0) {
res.setEvent(true);
}
// get status.
byte status = header[3];
res.setStatus(status);
try {
if (status == Response.OK) {
Object data;
if (res.isHeartbeat()) {
ObjectInput in = CodecSupport.deserialize(channel.getUrl(), is, proto);
data = decodeHeartbeatData(channel, in);
} else if (res.isEvent()) {
ObjectInput in = CodecSupport.deserialize(channel.getUrl(), is, proto);
data = decodeEventData(channel, in);
} else {
DecodeableRpcResult result;
if (channel.getUrl().getParameter(DECODE_IN_IO_THREAD_KEY, DEFAULT_DECODE_IN_IO_THREAD)) {
result = new DecodeableRpcResult(channel, res, is,
(Invocation) getRequestData(id), proto);
result.decode();
} else {
result = new DecodeableRpcResult(channel, res,
new UnsafeByteArrayInputStream(readMessageData(is)),
(Invocation) getRequestData(id), proto);
}
data = result;
}
res.setResult(data);
} else {
ObjectInput in = CodecSupport.deserialize(channel.getUrl(), is, proto);
res.setErrorMessage(in.readUTF());
}
} catch (Throwable t) {
if (log.isWarnEnabled()) {
log.warn("Decode response failed: " + t.getMessage(), t);
}
res.setStatus(Response.CLIENT_ERROR);
res.setErrorMessage(StringUtils.toString(t));
}
return res;
} else {
// decode request.
// 解码 Request
// 创建 Request 对象
Request req = new Request(id);
req.setVersion(Version.getProtocolVersion());
// 通过逻辑与运算得到通信方式，并设置到 Request 对象中
req.setTwoWay((flag & FLAG_TWOWAY) != 0);
// 通过位运算检测数据包是否为事件类型
if ((flag & FLAG_EVENT) != 0) {
// 设置心跳事件到 Request 对象中
req.setEvent(true);
}
try {
Object data;
if (req.isHeartbeat()) {
// 对心跳包数据进行解码，即反序列化
ObjectInput in = CodecSupport.deserialize(channel.getUrl(), is, proto);
data = decodeHeartbeatData(channel, in);
} else if (req.isEvent()) {
// 对事件数据进行解码，即反序列化
ObjectInput in = CodecSupport.deserialize(channel.getUrl(), is, proto);
data = decodeEventData(channel, in);
} else {
// 对请求 Request 数据解码
DecodeableRpcInvocation inv;
// 根据 url 参数判断是否在 IO 线程上对消息体进行解码
if (channel.getUrl().getParameter(DECODE_IN_IO_THREAD_KEY, DEFAULT_DECODE_IN_IO_THREAD)) {
inv = new DecodeableRpcInvocation(channel, req, is, proto);
// 在当前线程，也就是 IO 线程上进行后续的解码工作。此工作完成后，可将调用方法名、attachment、以及调用参数解析出来
// 调用 DecodeableRpcInvocation 的 decode 方法解码
inv.decode();
} else {
// 仅创建 DecodeableRpcInvocation 对象，但不在当前线程上执行解码逻辑
inv = new DecodeableRpcInvocation(channel, req,
new UnsafeByteArrayInputStream(readMessageData(is)), proto);
}
data = inv;
}
// 设置 data 到 Request 对象中
req.setData(data);
} catch (Throwable t) {
if (log.isWarnEnabled()) {
log.warn("Decode request failed: " + t.getMessage(), t);
}
// bad request
// 编码异常，broken 属性设置为 true，并将异常信息设置到 Request 对象中
req.setBroken(true);
req.setData(t);
}
return req;
}
}
// 5、DecodeableRpcInvocation 的 decode 方法
@Override
public Object decode(Channel channel, InputStream input) throws IOException {
// 反序列化 url 数据
ObjectInput in = CodecSupport.getSerialization(channel.getUrl(), serializationType)
.deserialize(channel.getUrl(), input);
// 通过反序列化得到 dubbo version，并保存到 attachments 变量中
String dubboVersion = in.readUTF();
request.setVersion(dubboVersion);
setAttachment(DUBBO_VERSION_KEY, dubboVersion);
// 通过反序列化得到 path 和 version，并保存到 attachments 变量中
setAttachment(PATH_KEY, in.readUTF());
setAttachment(VERSION_KEY, in.readUTF());
// 通过反序列化得到 方法名，并设置到 RpcInvocation中
setMethodName(in.readUTF());
try {
Object[] args;
Class<?>[] pts;
// 通过反序列化得到参数类型字符串，比如 Ljava/lang/String;
String desc = in.readUTF();
if (desc.length() == 0) {
pts = DubboCodec.EMPTY_CLASS_ARRAY;
args = DubboCodec.EMPTY_OBJECT_ARRAY;
} else {
// 将 desc 解析为参数类型数组
pts = ReflectUtils.desc2classArray(desc);
args = new Object[pts.length];
for (int i = 0; i < args.length; i++) {
try {
// 解析运行时参数，即方法传入参数
args[i] = in.readObject(pts[i]);
} catch (Exception e) {
if (log.isWarnEnabled()) {
log.warn("Decode argument failed: " + e.getMessage(), e);
}
}
}
}
// 设置参数类型数组
setParameterTypes(pts);
// 通过反序列化得到原 attachment 的内容
Map<String, String> map = (Map<String, String>) in.readObject(Map.class);
if (map != null && map.size() > 0) {
Map<String, String> attachment = getAttachments();
if (attachment == null) {
attachment = new HashMap<String, String>();
}
// 将 map 与当前对象中的 attachment 集合进行合并
attachment.putAll(map);
setAttachments(attachment);
}
//decode argument ,may be callback
// 对 callback 类型的参数进行处理
for (int i = 0; i < args.length; i++) {
args[i] = decodeInvocationArgument(channel, this, pts, i, args[i]);
}
// 设置方法传入参数列表
setArguments(args);
} catch (ClassNotFoundException e) {
throw new IOException(StringUtils.toString("Read invocation data failed.", e));
} finally {
if (in instanceof Cleanable) {
((Cleanable) in).cleanup();
}
}
return this;
}3.3.2 调用服务
解码器将数据包解析成 Request 对象后，NettyServerHandler 的 channelRead 方法会收到这个对象，并将这个对象继续向下传递。这期间该对象会被依次传递给 AbstractPeer、MultiMessageHandler、HeartbeatHandler 以及 AllChannelHandler。最后由 AllChannelHandler 将该对象封装到 Runnable 实现类对象中，并将 Runnable 放入ThreadPoolExecutor（线程池）中执行后续的调用逻辑。整个调用栈如下图：
调用栈具体信息：
execute:1343, ThreadPoolExecutor (java.util.concurrent)
received:63, AllChannelHandler (org.apache.dubbo.remoting.transport.dispatcher.all)
received:88, HeartbeatHandler (org.apache.dubbo.remoting.exchange.support.header)
received:43, MultiMessageHandler (org.apache.dubbo.remoting.transport)
received:149, AbstractPeer (org.apache.dubbo.remoting.transport)
channelRead:94, NettyServerHandler (org.apache.dubbo.remoting.transport.netty4)
invokeChannelRead:362, AbstractChannelHandlerContext (io.netty.channel)
invokeChannelRead:348, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:340, AbstractChannelHandlerContext (io.netty.channel)
channelRead:286, IdleStateHandler (io.netty.handler.timeout)
invokeChannelRead:362, AbstractChannelHandlerContext (io.netty.channel)
invokeChannelRead:348, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:340, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:310, ByteToMessageDecoder (io.netty.handler.codec)
channelRead:284, ByteToMessageDecoder (io.netty.handler.codec)
invokeChannelRead:362, AbstractChannelHandlerContext (io.netty.channel)
invokeChannelRead:348, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:340, AbstractChannelHandlerContext (io.netty.channel)
channelRead:1434, DefaultChannelPipeline$HeadContext (io.netty.channel)
invokeChannelRead:362, AbstractChannelHandlerContext (io.netty.channel)
invokeChannelRead:348, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:965, DefaultChannelPipeline (io.netty.channel)
read:163, AbstractNioByteChannel$NioByteUnsafe (io.netty.channel.nio)
processSelectedKey:647, NioEventLoop (io.netty.channel.nio)
processSelectedKeysOptimized:582, NioEventLoop (io.netty.channel.nio)
processSelectedKeys:499, NioEventLoop (io.netty.channel.nio)
run:461, NioEventLoop (io.netty.channel.nio)
run:884, SingleThreadEventExecutor$5 (io.netty.util.concurrent)
run:30, FastThreadLocalRunnable (io.netty.util.concurrent)
run:748, Thread (java.lang)整个过程的源码如下：
// 1、NettyServerHandler 的 channelRead 方法
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
NettyChannel channel = NettyChannel.getOrAddChannel(ctx.channel(), url, handler);
try {
// 调用 AbstractPeer 的 received 方法接收数据
handler.received(channel, msg);
} finally {
NettyChannel.removeChannelIfDisconnected(ctx.channel());
}
}
// 2、AbstractPeer 的 received 方法
@Override
public void received(Channel ch, Object msg) throws RemotingException {
if (closed) {
return;
}
// 调用 MultiMessageHandler 的 received 方法接收数据
handler.received(ch, msg);
}
// 3、MultiMessageHandler 的 received 方法
@Override
public void received(Channel channel, Object message) throws RemotingException {
if (message instanceof MultiMessage) {
MultiMessage list = (MultiMessage) message;
for (Object obj : list) {
handler.received(channel, obj);
}
} else {
// 调用 HeartbeatHandler 的 received 方法接收数据
handler.received(channel, message);
}
}
// 4、HeartbeatHandler 的 received 方法
@Override
public void received(Channel channel, Object message) throws RemotingException {
setReadTimestamp(channel);
if (isHeartbeatRequest(message)) {
// 处理心跳请求数据
Request req = (Request) message;
if (req.isTwoWay()) {
Response res = new Response(req.getId(), req.getVersion());
res.setEvent(Response.HEARTBEAT_EVENT);
channel.send(res);
if (logger.isInfoEnabled()) {
int heartbeat = channel.getUrl().getParameter(Constants.HEARTBEAT_KEY, 0);
if (logger.isDebugEnabled()) {
logger.debug("Received heartbeat from remote channel " + channel.getRemoteAddress()
+ ", cause: The channel has no data-transmission exceeds a heartbeat period"
+ (heartbeat > 0 ? ": " + heartbeat + "ms" : ""));
}
}
}
return;
}
if (isHeartbeatResponse(message)) {
// 如果是心跳 的 Response 数据信息，不做处理直接返回
if (logger.isDebugEnabled()) {
logger.debug("Receive heartbeat response in thread " + Thread.currentThread().getName());
}
return;
}
// 调用 AllChannelHandler 的 received 方法接收数据
handler.received(channel, message);
}整个逻辑相当简单，就是不断地将 Request 对象向后传递，最后由 AllChannelHandler 的 received 方法处理，在讲解 AllChannelHandler 类之前，我们先看看 dubbo 的线程派发模型
3.3.2.1 线程派发模型
Dubbo 将底层通信框架中接收请求的线程称为 IO 线程。如果一些事件处理逻辑可以很快执行完，比如只在内存打一个标记，此时直接在 IO 线程上执行该段逻辑即可。但如果事件的处理逻辑比较耗时，比如该段逻辑会发起数据库查询或者 HTTP 请求。此时我们就不应该让事件处理逻辑在 IO 线程上执行，而是应该派发到线程池中去执行。原因也很简单，IO 线程主要用于接收请求，如果 IO 线程被占满，将导致它不能接收新的请求。使用线程池不仅可以降低性能损耗（线程的频繁创建和销毁），还可以达到异步并行执行的目的，提高处理请求的效率
以上就是线程派发的背景，下面我们再来通过 Dubbo 调用图，看一下线程派发器所处的位置
上图红框中的 Dispatcher 就是线程派发器。需要说明的是，Dispatcher 真实的职责创建具有线程派发能力的 ChannelHandler，比如 AllChannelHandler（默认） 和 MessageOnlyChannelHandler 等，其本身并不具备线程派发能力。Dubbo 支持 5 种不同的线程派发策略，下面通过一个表格列举一下。
| 策略 | 用途 |
|---|---|
| all | 所有消息都派发到线程池，包括请求，响应，连接事件，断开事件等 |
| direct | 所有消息都不派发到线程池，全部在 IO 线程上直接执行 |
| message | 只有请求和响应消息派发到线程池，其它消息均在 IO 线程上执行 |
| execution | 只有请求消息派发到线程池，不含响应。其它消息均在 IO 线程上执行 |
| connection | 在 IO 线程上，将连接断开事件放入队列，有序逐个执行，其它消息派发到线程池 |
默认配置下，Dubbo 使用 all 派发策略，即将所有的消息都派发到线程池中。下面我们来分析一下 AllChannelHandler 的代码，如下：
// 默认是 all 策略，即 AllDispatcher
@SPI(AllDispatcher.NAME)
public interface Dispatcher {
/**
* dispatch the message to threadpool.
*
* @param handler
* @param url
* @return channel handler
*/
@Adaptive({Constants.DISPATCHER_KEY, "dispather", "channel.handler"})
// The last two parameters are reserved for compatibility with the old configuration
ChannelHandler dispatch(ChannelHandler handler, URL url);
}
public class AllDispatcher implements Dispatcher {
public static final String NAME = "all";
@Override
public ChannelHandler dispatch(ChannelHandler handler, URL url) {
// 创建了 AllChannelHandler 对象返回
return new AllChannelHandler(handler, url);
}
}
public class AllChannelHandler extends WrappedChannelHandler {
public AllChannelHandler(ChannelHandler handler, URL url) {
super(handler, url);
}
// 处理连接事件
@Override
public void connected(Channel channel) throws RemotingException {
ExecutorService executor = getExecutorService();
try {
executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.CONNECTED));
} catch (Throwable t) {
throw new ExecutionException("connect event", channel, getClass() + " error when process connected event .", t);
}
}
// 处理断开连接事件
@Override
public void disconnected(Channel channel) throws RemotingException {
ExecutorService executor = getExecutorService();
try {
executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.DISCONNECTED));
} catch (Throwable t) {
throw new ExecutionException("disconnect event", channel, getClass() + " error when process disconnected event .", t);
}
}
// 处理请求和响应消息，这里的 message 变量类型可能是 Request，也可能是 Response
@Override
public void received(Channel channel, Object message) throws RemotingException {
ExecutorService executor = getExecutorService();
try {
// 将请求和响应消息派发到线程池中处理
executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.RECEIVED, message));
} catch (Throwable t) {
//TODO A temporary solution to the problem that the exception information can not be sent to the opposite end after the thread pool is full. Need a refactoring
//fix The thread pool is full, refuses to call, does not return, and causes the consumer to wait for time out
if(message instanceof Request && t instanceof RejectedExecutionException){
Request request = (Request)message;
if(request.isTwoWay()){
// 如果通信方式为双向通信，此时将 Server side ... threadpool is exhausted
// 错误信息封装到 Response 中，并返回给服务消费方。
String msg = "Server side(" + url.getIp() + "," + url.getPort() + ") threadpool is exhausted ,detail msg:" + t.getMessage();
Response response = new Response(request.getId(), request.getVersion());
response.setStatus(Response.SERVER_THREADPOOL_EXHAUSTED_ERROR);
response.setErrorMessage(msg);
channel.send(response);
return;
}
}
throw new ExecutionException(message, channel, getClass() + " error when process received event .", t);
}
}
// 处理异常信息
@Override
public void caught(Channel channel, Throwable exception) throws RemotingException {
ExecutorService executor = getExecutorService();
try {
executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.CAUGHT, exception));
} catch (Throwable t) {
throw new ExecutionException("caught event", channel, getClass() + " error when process caught event .", t);
}
}
}Request 对象被 AllChannelHandler 的 received 方法接收之后，被封装到了 ChannelEventRunnable 对象中，ChannelEventRunnable 是调用服务的起点，下面从这个类开始讲起。
3.3.2.2 调用服务
调用服务的真正逻辑从 ChannelEventRunnable 开始，debug 调试后得到整个调用栈，如下图：
调用栈具体信息：
sayHello:30, DemoServiceImpl (org.apache.dubbo.demo.provider)
invokeMethod:-1, Wrapper1 (org.apache.dubbo.common.bytecode)
doInvoke:53, JavassistProxyFactory$1 (org.apache.dubbo.rpc.proxy.javassist)
invoke:84, AbstractProxyInvoker (org.apache.dubbo.rpc.proxy)
invoke:56, DelegateProviderMetaDataInvoker (org.apache.dubbo.config.invoker)
invoke:58, InvokerWrapper (org.apache.dubbo.rpc.protocol)
invoke:55, ExceptionFilter (org.apache.dubbo.rpc.filter)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:94, MonitorFilter (org.apache.dubbo.monitor.support)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:48, TimeoutFilter (org.apache.dubbo.rpc.filter)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:81, TraceFilter (org.apache.dubbo.rpc.protocol.dubbo.filter)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:96, ContextFilter (org.apache.dubbo.rpc.filter)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:148, GenericFilter (org.apache.dubbo.rpc.filter)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:38, ClassLoaderFilter (org.apache.dubbo.rpc.filter)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:41, EchoFilter (org.apache.dubbo.rpc.filter)
invoke:84, ProtocolFilterWrapper$1 (org.apache.dubbo.rpc.protocol)
invoke:161, ProtocolFilterWrapper$CallbackRegistrationInvoker (org.apache.dubbo.rpc.protocol)
reply:157, DubboProtocol$1 (org.apache.dubbo.rpc.protocol.dubbo)
handleRequest:107, HeaderExchangeHandler (org.apache.dubbo.remoting.exchange.support.header)
received:207, HeaderExchangeHandler (org.apache.dubbo.remoting.exchange.support.header)
received:56, DecodeHandler (org.apache.dubbo.remoting.transport)
run:61, ChannelEventRunnable (org.apache.dubbo.remoting.transport.dispatcher)下面我们通过源码分析一下整个过程，代码如下：
// 1、ChannelEventRunnable 的 run 方法
@Override
public void run() {
// 检测通道状态，对于请求或响应消息，此时 state = RECEIVED
if (state == ChannelState.RECEIVED) {
// 此状态频繁处理，因此单独做出判断处理
try {
// 将 channel 和 message 传给 ChannelHandler 对象，调用 DecodeHandler 的 received 方法
handler.received(channel, message);
} catch (Exception e) {
logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel
+ ", message is " + message, e);
}
} else {
// 其他状态的处理，例如 连接、断开连接、发送、异常处理等
switch (state) {
case CONNECTED:
try {
handler.connected(channel);
} catch (Exception e) {
logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel, e);
}
break;
case DISCONNECTED:
try {
handler.disconnected(channel);
} catch (Exception e) {
logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel, e);
}
break;
case SENT:
try {
handler.sent(channel, message);
} catch (Exception e) {
logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel
+ ", message is " + message, e);
}
break;
case CAUGHT:
try {
handler.caught(channel, exception);
} catch (Exception e) {
logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel
+ ", message is: " + message + ", exception is " + exception, e);
}
break;
default:
logger.warn("unknown state: " + state + ", message is " + message);
}
}
}
// 2、DecodeHandler 的 received 方法
@Override
public void received(Channel channel, Object message) throws RemotingException {
if (message instanceof Decodeable) {
// 对 Decodeable 接口实现类对象进行解码
decode(message);
}
if (message instanceof Request) {
// 对 Request 的 data 字段进行解码
decode(((Request) message).getData());
}
if (message instanceof Response) {
// 对 Response 的 result 字段进行解码
decode(((Response) message).getResult());
}
// 调用 HeaderExchangeHandler 的 received 方法
handler.received(channel, message);
}
// 3、HeaderExchangeHandler 的 received 方法
@Override
public void received(Channel channel, Object message) throws RemotingException {
channel.setAttribute(KEY_READ_TIMESTAMP, System.currentTimeMillis());
final ExchangeChannel exchangeChannel = HeaderExchangeChannel.getOrAddChannel(channel);
try {
if (message instanceof Request) {
// handle request.
// 处理 Request 请求
Request request = (Request) message;
if (request.isEvent()) {
// 处理事件
handlerEvent(channel, request);
} else {
// 双向通信
if (request.isTwoWay()) {
// 调用本类方法处理请求
handleRequest(exchangeChannel, request);
} else {
// 如果是单向通信，仅向后调用指定服务即可，无需返回调用结果
handler.received(exchangeChannel, request.getData());
}
}
} else if (message instanceof Response) {
// 处理响应对象，服务消费方会执行此处逻辑，后面分析
handleResponse(channel, (Response) message);
} else if (message instanceof String) {
if (isClientSide(channel)) {
Exception e = new Exception("Dubbo client can not supported string message: " + message + " in channel: " + channel + ", url: " + channel.getUrl());
logger.error(e.getMessage(), e);
} else {
String echo = handler.telnet(channel, (String) message);
if (echo != null && echo.length() > 0) {
channel.send(echo);
}
}
} else {
handler.received(exchangeChannel, message);
}
} finally {
HeaderExchangeChannel.removeChannelIfDisconnected(channel);
}
}
// 4、HeaderExchangeHandler 的 handleRequest 方法
void handleRequest(final ExchangeChannel channel, Request req) throws RemotingException {
// 构建 Response 返回数据
Response res = new Response(req.getId(), req.getVersion());
// 检测请求是否合法，不合法则返回状态码为 BAD_REQUEST 的响应
if (req.isBroken()) {
Object data = req.getData();
String msg;
if (data == null) {
msg = null;
} else if (data instanceof Throwable) {
msg = StringUtils.toString((Throwable) data);
} else {
msg = data.toString();
}
res.setErrorMessage("Fail to decode request due to: " + msg);
res.setStatus(Response.BAD_REQUEST);
channel.send(res);
return;
}
// find handler by message class.
// 获取 data 字段值，也就是 RpcInvocation 对象
Object msg = req.getData();
try {
// 调用 DubboProtocol$ExchangeHandlerAdapter 的 reply 方法
CompletionStage<Object> future = handler.reply(channel, msg);
future.whenComplete((appResult, t) -> {
try {
if (t == null) {
// 请求成功
res.setStatus(Response.OK);
res.setResult(appResult);
} else {
// 请求服务异常
res.setStatus(Response.SERVICE_ERROR);
res.setErrorMessage(StringUtils.toString(t));
}
// 发送请求结果
channel.send(res);
} catch (RemotingException e) {
logger.warn("Send result to consumer failed, channel is " + channel + ", msg is " + e);
} finally {
// HeaderExchangeChannel.removeChannelIfDisconnected(channel);
}
});
} catch (Throwable e) {
// 若调用过程出现异常，则设置 SERVICE_ERROR，表示服务端异常
res.setStatus(Response.SERVICE_ERROR);
res.setErrorMessage(StringUtils.toString(e));
channel.send(res);
}
}
// 5、DubboProtocol 内部匿名类 ExchangeHandlerAdapter 的 reply 方法
private ExchangeHandler requestHandler = new ExchangeHandlerAdapter() {
@Override
public CompletableFuture<Object> reply(ExchangeChannel channel, Object message) throws RemotingException {
if (!(message instanceof Invocation)) {
throw new RemotingException(channel, "Unsupported request: "
+ (message == null ? null : (message.getClass().getName() + ": " + message))
+ ", channel: consumer: " + channel.getRemoteAddress() + " --> provider: " + channel.getLocalAddress());
}
Invocation inv = (Invocation) message;
// 获取 Invoker 实例
// 这里主要是从 exporterMap 查找与 serviceKey 相对应的 DubboExporter 对象，DubboExporter 的 getInvoker 方法获取到 Invoker 实例
// provider 导出服务时，将<serviceKey, DubboExporter> 映射关系存储到 exporterMap 集合中了
Invoker<?> invoker = getInvoker(channel, inv);
// need to consider backward-compatibility if it's a callback
if (Boolean.TRUE.toString().equals(inv.getAttachments().get(IS_CALLBACK_SERVICE_INVOKE))) {
String methodsStr = invoker.getUrl().getParameters().get("methods");
boolean hasMethod = false;
if (methodsStr == null || !methodsStr.contains(",")) {
hasMethod = inv.getMethodName().equals(methodsStr);
} else {
String[] methods = methodsStr.split(",");
for (String method : methods) {
if (inv.getMethodName().equals(method)) {
hasMethod = true;
break;
}
}
}
if (!hasMethod) {
logger.warn(new IllegalStateException("The methodName " + inv.getMethodName()
+ " not found in callback service interface ,invoke will be ignored."
+ " please update the api interface. url is:"
+ invoker.getUrl()) + " ,invocation is :" + inv);
return null;
}
}
RpcContext.getContext().setRemoteAddress(channel.getRemoteAddress());
// 通过 Invoker 调用具体的服务, 这期间会调用dubbo 的 所有Filter 过滤器链，按照顺序调用： EchoFilter、ClassLoaderFilter、GenericFilter、
// ContextFilter、TraceFilter、TimeoutFilter、MonitorFilter、ExceptionFilter
// 过滤器链执行之后，就调用到了关键的 AbstractProxyInvoker 类的 invoke 方法
Result result = invoker.invoke(inv);
// 将请求结果放入异步带返回结果的 Future 中
return result.completionFuture().thenApply(Function.identity());
}
// 6、AbstractProxyInvoker 的 invoke 方法
@Override
public Result invoke(Invocation invocation) throws RpcException {
try {
// 调用实现类 JavassistProxyFactory 的 doInvoke 方法执行方法调用
Object value = doInvoke(proxy, invocation.getMethodName(), invocation.getParameterTypes(), invocation.getArguments());
// 将调用结果封装到 AsyncRpcResult 中
CompletableFuture<Object> future = wrapWithFuture(value, invocation);
AsyncRpcResult asyncRpcResult = new AsyncRpcResult(invocation);
future.whenComplete((obj, t) -> {
AppResponse result = new AppResponse();
if (t != null) {
if (t instanceof CompletionException) {
result.setException(t.getCause());
} else {
result.setException(t);
}
} else {
result.setValue(obj);
}
asyncRpcResult.complete(result);
});
return asyncRpcResult;
} catch (InvocationTargetException e) {
if (RpcContext.getContext().isAsyncStarted() && !RpcContext.getContext().stopAsync()) {
logger.error("Provider async started, but got an exception from the original method, cannot write the exception back to consumer because an async result may have returned the new thread.", e);
}
return AsyncRpcResult.newDefaultAsyncResult(null, e.getTargetException(), invocation);
} catch (Throwable e) {
throw new RpcException("Failed to invoke remote proxy method " + invocation.getMethodName() + " to " + getUrl() + ", cause: " + e.getMessage(), e);
}
}
// 7、JavassistProxyFactory 的 doInvoke 方法
@Override
public <T> Invoker<T> getInvoker(T proxy, Class<T> type, URL url) {
// TODO Wrapper cannot handle this scenario correctly: the classname contains '$'
// 为目标类创建 Wrapper
final Wrapper wrapper = Wrapper.getWrapper(proxy.getClass().getName().indexOf('$') < 0 ? proxy.getClass() : type);
return new AbstractProxyInvoker<T>(proxy, type, url) {
// 创建匿名 Invoker 对象，并实现了 doInvoke 方法
@Override
protected Object doInvoke(T proxy, String methodName,
Class<?>[] parameterTypes,
Object[] arguments) throws Throwable {
// 调用 Wrapper 的 invokeMethod 方法，invokeMethod 方法中会调用目标方法
return wrapper.invokeMethod(proxy, methodName, parameterTypes, arguments);
}
};
}Wrapper 是一个抽象类，其中 invokeMethod 是一个抽象方法。Dubbo 会在运行时通过 Javassist 框架为 Wrapper 生成实现类，并实现 invokeMethod 方法，该方法最终会根据调用信息调用具体的服务。这个运行时生成的 Wrapper 实现类如何看到呢？只需要在 Wrapper 源码中 Class<?> wc = cc.toClass(); 这句话下面增加：
cc.getClassPool().get((Modifier.isPublic(c.getModifiers()) ? Wrapper.class.getName() : c.getName() + "$sw") + id).debugWriteFile("E:\\");如下图所示：
以 DemoServiceImpl 为例，Javassist 为其生成的代理类 Wrapper0，使用 ju-jdi 反编译文件，获取到代码如下:
package org.apache.dubbo.common.bytecode;
import java.lang.reflect.InvocationTargetException;
import java.util.Map;
import org.apache.dubbo.demo.DemoService;
public class Wrapper0 extends Wrapper
implements ClassGenerator.DC
{
public static String[] pns;
public static Map pts;
public static String[] mns;
public static String[] dmns;
public static Class[] mts0;
public String[] getPropertyNames()
{
return pns;
}
public boolean hasProperty(String paramString)
{
return pts.containsKey(paramString);
}
public Class getPropertyType(String paramString)
{
return (Class)pts.get(paramString);
}
public String[] getMethodNames()
{
return mns;
}
public String[] getDeclaredMethodNames()
{
return dmns;
}
public void setPropertyValue(Object paramObject1, String paramString, Object paramObject2)
{
try
{
DemoService localDemoService = (DemoService)paramObject1;
}
catch (Throwable localThrowable)
{
throw new IllegalArgumentException(localThrowable);
}
throw new NoSuchPropertyException("Not found property \"" + paramString + "\" field or setter method in class org.apache.dubbo.demo.DemoService.");
}
public Object getPropertyValue(Object paramObject, String paramString)
{
try
{
DemoService localDemoService = (DemoService)paramObject;
}
catch (Throwable localThrowable)
{
throw new IllegalArgumentException(localThrowable);
}
throw new NoSuchPropertyException("Not found property \"" + paramString + "\" field or setter method in class org.apache.dubbo.demo.DemoService.");
}
public Object invokeMethod(Object paramObject, String paramString, Class[] paramArrayOfClass, Object[] paramArrayOfObject)
throws InvocationTargetException
{
DemoService localDemoService;
try
{
// 类型转换
localDemoService = (DemoService)paramObject;
}
catch (Throwable localThrowable1)
{
throw new IllegalArgumentException(localThrowable1);
}
try
{
// 根据方法名和参数调用指定方法
if ((!"sayHello".equals(paramString)) || (paramArrayOfClass.length == 1))
return localDemoService.sayHello((String)paramArrayOfObject[0]);
}
catch (Throwable localThrowable2)
{
throw new InvocationTargetException(localThrowable2);
}
throw new NoSuchMethodException("Not found method \"" + paramString + "\" in class org.apache.dubbo.demo.DemoService.");
}
}到这里，服务的整个调用过程就分析完了。
3.4 provider 返回结果
provider 调用指定服务后，会将调用结果封装到 Response 对象中，并将该对象返回给 consumer。provider 也是通过 NettyChannel 的 send 方法将 Response 对象返回，这个方法在 3.2.1 节分析过，这里就不在重复分析了。我们仅需关注 Response 对象的编码过程即可，编码逻辑代码如下:
// 1、ExchangeCodec 的 encode 方法，这里 msg 类型是 Response
@Override
public void encode(Channel channel, ChannelBuffer buffer, Object msg) throws IOException {
if (msg instanceof Request) {
// Request 请求数据编码
encodeRequest(channel, buffer, (Request) msg);
} else if (msg instanceof Response) {
// Response 返回结果编码
encodeResponse(channel, buffer, (Response) msg);
} else {
super.encode(channel, buffer, msg);
}
}
// 2、ExchangeCodec 的 encodeResponse 方法，实现了 Response 编码
protected void encodeResponse(Channel channel, ChannelBuffer buffer, Response res) throws IOException {
int savedWriteIndex = buffer.writerIndex();
try {
// 获取序列化方式，默认是 Hessian2Serialization
Serialization serialization = getSerialization(channel);
// header.
// 创建消息头字节数组，长度为 16
byte[] header = new byte[HEADER_LENGTH];
// set magic number.
// 设置魔数
Bytes.short2bytes(MAGIC, header);
// set request and serialization flag.
// 设置序列化器编号
header[2] = serialization.getContentTypeId();
if (res.isHeartbeat()) {
header[2] |= FLAG_EVENT;
}
// set response status.
// 获取并设置响应状态
byte status = res.getStatus();
header[3] = status;
// set request id.
// 设置请求编号
Bytes.long2bytes(res.getId(), header, 4);
// 更新 writerIndex，为消息头预留 16 个字节的空间
buffer.writerIndex(savedWriteIndex + HEADER_LENGTH);
ChannelBufferOutputStream bos = new ChannelBufferOutputStream(buffer);
ObjectOutput out = serialization.serialize(channel.getUrl(), bos);
// encode response data or error message.
if (status == Response.OK) {
if (res.isHeartbeat()) {
// 对心跳响应结果进行序列化
encodeHeartbeatData(channel, out, res.getResult());
} else {
// 调用 DubboCodec 的 encodeResponseData 方法对调用结果进行序列化
encodeResponseData(channel, out, res.getResult(), res.getVersion());
}
} else {
// 对错误信息进行序列化
out.writeUTF(res.getErrorMessage());
}
out.flushBuffer();
if (out instanceof Cleanable) {
((Cleanable) out).cleanup();
}
bos.flush();
bos.close();
// 获取写入的字节数，也就是消息体长度
int len = bos.writtenBytes();
checkPayload(channel, len);
// 将消息体长度写入到消息头中
Bytes.int2bytes(len, header, 12);
// write
// 将 buffer 指针移动到 savedWriteIndex，为写消息头做准备
buffer.writerIndex(savedWriteIndex);
// write header.
// 从 savedWriteIndex 下标处写入消息头
buffer.writeBytes(header);
// 设置新的 writerIndex，writerIndex = 原写下标 + 消息头长度 + 消息体长度
buffer.writerIndex(savedWriteIndex + HEADER_LENGTH + len);
} catch (Throwable t) {
// clear buffer
buffer.writerIndex(savedWriteIndex);
// send error message to Consumer, otherwise, Consumer will wait till timeout.
if (!res.isEvent() && res.getStatus() != Response.BAD_RESPONSE) {
Response r = new Response(res.getId(), res.getVersion());
r.setStatus(Response.BAD_RESPONSE);
if (t instanceof ExceedPayloadLimitException) {
logger.warn(t.getMessage(), t);
try {
r.setErrorMessage(t.getMessage());
channel.send(r);
return;
} catch (RemotingException e) {
logger.warn("Failed to send bad_response info back: " + t.getMessage() + ", cause: " + e.getMessage(), e);
}
} else {
// FIXME log error message in Codec and handle in caught() of IoHanndler?
logger.warn("Fail to encode response: " + res + ", send bad_response info instead, cause: " + t.getMessage(), t);
try {
r.setErrorMessage("Failed to send response: " + res + ", cause: " + StringUtils.toString(t));
channel.send(r);
return;
} catch (RemotingException e) {
logger.warn("Failed to send bad_response info back: " + res + ", cause: " + e.getMessage(), e);
}
}
}
// Rethrow exception
if (t instanceof IOException) {
throw (IOException) t;
} else if (t instanceof RuntimeException) {
throw (RuntimeException) t;
} else if (t instanceof Error) {
throw (Error) t;
} else {
throw new RuntimeException(t.getMessage(), t);
}
}
}
// 3、DubboCodec 的 encodeResponseData 方法
@Override
protected void encodeResponseData(Channel channel, ObjectOutput out, Object data, String version) throws IOException {
Result result = (Result) data;
// currently, the version value in Response records the version of Request
// 检测当前协议版本是否支持带有 attachment 集合的 Response 对象
boolean attach = Version.isSupportResponseAttachment(version);
Throwable th = result.getException();
if (th == null) {
// 无异常信息
Object ret = result.getValue();
// 序列化响应类型
if (ret == null) {
out.writeByte(attach ? RESPONSE_NULL_VALUE_WITH_ATTACHMENTS : RESPONSE_NULL_VALUE);
} else {
out.writeByte(attach ? RESPONSE_VALUE_WITH_ATTACHMENTS : RESPONSE_VALUE);
// 序列化调用结果
out.writeObject(ret);
}
} else {
out.writeByte(attach ? RESPONSE_WITH_EXCEPTION_WITH_ATTACHMENTS : RESPONSE_WITH_EXCEPTION);
// 序列化异常信息
out.writeObject(th);
}
if (attach) {
// returns current version of Response to consumer side.
// 记录 Dubbo 协议版本
result.getAttachments().put(DUBBO_VERSION_KEY, Version.getProtocolVersion());
// 序列化 attachments 集合
out.writeObject(result.getAttachments());
}
}这个provider 编码 Response 对象逻辑和 consumer 编码 Request 对象逻辑几乎差不多，这里就不在做过多的解释了。解下来就是分析 consumer 是如何接收请求返回结果的
3.5 consumer 接收请求结果
consumer 收到请求结果数据后，首先要做的就是对响应数据进行解码，得到 Response 对象。然后再将该对象传递给下一个入站处理器，这个入站处理器就是 NettyHandler。接下来 NettyHandler 会将这个对象继续向下传递，最后 AllChannelHandler 的 received 方法会收到这个对象，并将这个对象派发到线程池中。这个过程和 provider 接收请求的过程是一样的，因此这里就不重复分析了。本节我们重点分析两个方面的内容，一是响应数据的解码过程，二是 Dubbo 如何将调用结果传递给用户线程的。下面先来分析 响应数据的解码过程
3.5.1 响应数据解码
provider 对请求数据的解码和 consumer 对响应数据的解码逻辑差不多，在 3.3.1 请求解码 中我们知道，解码逻辑主要实现在 DubboCodec 的 decodeBody 方法，debug 调用栈如下：
调用具体栈信息：
decode:80, DecodeableRpcResult (org.apache.dubbo.rpc.protocol.dubbo)
decode:127, DecodeableRpcResult (org.apache.dubbo.rpc.protocol.dubbo)
decodeBody:102, DubboCodec (org.apache.dubbo.rpc.protocol.dubbo)
decode:137, ExchangeCodec (org.apache.dubbo.remoting.exchange.codec)
decode:88, ExchangeCodec (org.apache.dubbo.remoting.exchange.codec)
decode:52, DubboCountCodec (org.apache.dubbo.rpc.protocol.dubbo)
decode:90, NettyCodecAdapter$InternalDecoder (org.apache.dubbo.remoting.transport.netty4)
decodeRemovalReentryProtection:489, ByteToMessageDecoder (io.netty.handler.codec)
callDecode:428, ByteToMessageDecoder (io.netty.handler.codec)
channelRead:265, ByteToMessageDecoder (io.netty.handler.codec)
invokeChannelRead:362, AbstractChannelHandlerContext (io.netty.channel)
invokeChannelRead:348, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:340, AbstractChannelHandlerContext (io.netty.channel)
channelRead:1434, DefaultChannelPipeline$HeadContext (io.netty.channel)
invokeChannelRead:362, AbstractChannelHandlerContext (io.netty.channel)
invokeChannelRead:348, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:965, DefaultChannelPipeline (io.netty.channel)
read:163, AbstractNioByteChannel$NioByteUnsafe (io.netty.channel.nio)
processSelectedKey:647, NioEventLoop (io.netty.channel.nio)
processSelectedKeysOptimized:582, NioEventLoop (io.netty.channel.nio)
processSelectedKeys:499, NioEventLoop (io.netty.channel.nio)
run:461, NioEventLoop (io.netty.channel.nio)
run:884, SingleThreadEventExecutor$5 (io.netty.util.concurrent)
run:30, FastThreadLocalRunnable (io.netty.util.concurrent)
run:748, Thread (java.lang)整个解码过程的代码如下：
// 1、DubboCodec 的 decodeBody 方法
@Override
protected Object decodeBody(Channel channel, InputStream is, byte[] header) throws IOException {
// 获取消息头中的第三个字节，并通过逻辑与运算得到序列化器编号
byte flag = header[2], proto = (byte) (flag & SERIALIZATION_MASK);
// get request id.
// 获取消息头中的 request id 值
long id = Bytes.bytes2long(header, 4);
// 通过逻辑与运算得到调用类型，0 - Response，1 - Request
if ((flag & FLAG_REQUEST) == 0) {
// decode response.
// 创建 Response 对象
Response res = new Response(id);
if ((flag & FLAG_EVENT) != 0) {
res.setEvent(true);
}
// get status.
// 响应状态
byte status = header[3];
res.setStatus(status);
try {
if (status == Response.OK) {
Object data;
if (res.isHeartbeat()) {
// 反序列化心跳数据
ObjectInput in = CodecSupport.deserialize(channel.getUrl(), is, proto);
data = decodeHeartbeatData(channel, in);
} else if (res.isEvent()) {
// 反序列化事件数据数据
ObjectInput in = CodecSupport.deserialize(channel.getUrl(), is, proto);
data = decodeEventData(channel, in);
} else {
DecodeableRpcResult result;
// 根据 url 参数决定是否在 IO 线程上执行解码逻辑
if (channel.getUrl().getParameter(DECODE_IN_IO_THREAD_KEY, DEFAULT_DECODE_IN_IO_THREAD)) {
result = new DecodeableRpcResult(channel, res, is,
(Invocation) getRequestData(id), proto);
// 调用 DecodeableRpcInvocation 的 decode 方法解码
result.decode();
} else {
// 创建 DecodeableRpcResult 对象
result = new DecodeableRpcResult(channel, res,
new UnsafeByteArrayInputStream(readMessageData(is)),
(Invocation) getRequestData(id), proto);
}
data = result;
}
res.setResult(data);
} else {
ObjectInput in = CodecSupport.deserialize(channel.getUrl(), is, proto);
res.setErrorMessage(in.readUTF());
}
} catch (Throwable t) {
if (log.isWarnEnabled()) {
log.warn("Decode response failed: " + t.getMessage(), t);
}
res.setStatus(Response.CLIENT_ERROR);
res.setErrorMessage(StringUtils.toString(t));
}
return res;
} else {
// 省略request解码
}
}
// 2、DecodeableRpcResult 的 decode 方法
@Override
public Object decode(Channel channel, InputStream input) throws IOException {
ObjectInput in = CodecSupport.getSerialization(channel.getUrl(), serializationType)
.deserialize(channel.getUrl(), input);
// 反序列化响应类型
byte flag = in.readByte();
// 响应类型对应不同的处理逻辑
switch (flag) {
// 返回值为空
case DubboCodec.RESPONSE_NULL_VALUE:
break;
// 有返回值
case DubboCodec.RESPONSE_VALUE:
// 解析出返回值，并存储
handleValue(in);
break;
// 返回内容有异常
case DubboCodec.RESPONSE_WITH_EXCEPTION:
handleException(in);
break;
// 返回值为空，且携带了 attachments 集合
case DubboCodec.RESPONSE_NULL_VALUE_WITH_ATTACHMENTS:
// 解析出 attachments，并存储
handleAttachment(in);
break;
// 有返回值，且携带了 attachments 集合
case DubboCodec.RESPONSE_VALUE_WITH_ATTACHMENTS:
// 解析出返回值，并存储
handleValue(in);
// 解析出 attachments，并存储
handleAttachment(in);
break;
// 返回内容有异常，且携带了 attachments 集合
case DubboCodec.RESPONSE_WITH_EXCEPTION_WITH_ATTACHMENTS:
// 解析出异常，并存储
handleException(in);
// 解析出 attachments，并存储
handleAttachment(in);
break;
default:
throw new IOException("Unknown result flag, expect '0' '1' '2' '3' '4' '5', but received: " + flag);
}
if (in instanceof Cleanable) {
((Cleanable) in).cleanup();
}
return this;
}响应数据的解码逻辑也比较简单，代码注释也详细，这里就不在多赘述了。下面重点看一下怎么向用户线程传递调用结果的
3.5.2 向用户线程传递调用结果
consumer 响应数据解码完成之后，通过 NettyClientHandler 的 channelRead 方法将响应对象派发到线程池上，线程池中的线程并非用户的调用线程，那么这里是如何将 Response 从线程池线程传递到用户线程上的呢？答案是：
用户线程先调用 DefaultFuture 的 getFuture 方法通过 requestId 获取到线程池中请求对应的 DefaultFuture 对象，然后调用 DefaultFuture（实际在CompletableFuture中实现） 的 get 方法获取请求结果，当请求结果还没有时，用户线程会阻塞等待指定的时间
debug 调用栈流程如下：
1）、接收请求结果消息并交给线程池处理
2）、线程池线程处理请求结果
调用栈具体信息：
// 1、接收请求结果消息并交给线程池处理
received:64, AllChannelHandler (org.apache.dubbo.remoting.transport.dispatcher.all)
received:92, HeartbeatHandler (org.apache.dubbo.remoting.exchange.support.header)
received:45, MultiMessageHandler (org.apache.dubbo.remoting.transport)
received:151, AbstractPeer (org.apache.dubbo.remoting.transport)
channelRead:79, NettyClientHandler (org.apache.dubbo.remoting.transport.netty4)
invokeChannelRead:362, AbstractChannelHandlerContext (io.netty.channel)
invokeChannelRead:348, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:340, AbstractChannelHandlerContext (io.netty.channel)
channelRead:286, IdleStateHandler (io.netty.handler.timeout)
invokeChannelRead:362, AbstractChannelHandlerContext (io.netty.channel)
invokeChannelRead:348, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:340, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:310, ByteToMessageDecoder (io.netty.handler.codec)
channelRead:284, ByteToMessageDecoder (io.netty.handler.codec)
invokeChannelRead:362, AbstractChannelHandlerContext (io.netty.channel)
invokeChannelRead:348, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:340, AbstractChannelHandlerContext (io.netty.channel)
channelRead:1434, DefaultChannelPipeline$HeadContext (io.netty.channel)
invokeChannelRead:362, AbstractChannelHandlerContext (io.netty.channel)
invokeChannelRead:348, AbstractChannelHandlerContext (io.netty.channel)
fireChannelRead:965, DefaultChannelPipeline (io.netty.channel)
read:163, AbstractNioByteChannel$NioByteUnsafe (io.netty.channel.nio)
processSelectedKey:647, NioEventLoop (io.netty.channel.nio)
processSelectedKeysOptimized:582, NioEventLoop (io.netty.channel.nio)
processSelectedKeys:499, NioEventLoop (io.netty.channel.nio)
run:461, NioEventLoop (io.netty.channel.nio)
run:884, SingleThreadEventExecutor$5 (io.netty.util.concurrent)
run:30, FastThreadLocalRunnable (io.netty.util.concurrent)
run:748, Thread (java.lang)
// 2、线程池线程处理请求结果
doReceived:199, DefaultFuture (org.apache.dubbo.remoting.exchange.support)
received:163, DefaultFuture (org.apache.dubbo.remoting.exchange.support)
received:149, DefaultFuture (org.apache.dubbo.remoting.exchange.support)
handleResponse:62, HeaderExchangeHandler (org.apache.dubbo.remoting.exchange.support.header)
received:215, HeaderExchangeHandler (org.apache.dubbo.remoting.exchange.support.header)
received:56, DecodeHandler (org.apache.dubbo.remoting.transport)
run:61, ChannelEventRunnable (org.apache.dubbo.remoting.transport.dispatcher)
runWorker:1149, ThreadPoolExecutor (java.util.concurrent)
run:624, ThreadPoolExecutor$Worker (java.util.concurrent)
run:748, Thread (java.lang)这里的请求结果处理是在 DefaultFuture 的 received 方法完成的，DefaultFuture 源码我们已经在 3.1 consumer调用方式中讲过了，这里就不在讲解了。
本文中多次强调 requestId（调用编号） ，但一直没有解释原因，这里简单说明一下。一般情况下，consumer 会并发调用多个服务，每个用户线程发送请求后，会调用不同 DefaultFuture 对象的 get 方法进行等待。 一段时间后，服务消费方的线程池会收到多个响应对象。这个时候要考虑一个问题，如何将每个响应对象传递给相应的 DefaultFuture 对象，且不出错。答案是通过 requestId（调用编号）。DefaultFuture 被创建时，会要求传入一个 Request 对象。此时 DefaultFuture 可从 Request 对象中获取调用编号，并将 <调用编号, DefaultFuture 对象> 映射关系存入到静态 Map 中，即 FUTURES。线程池中的线程在收到 Response 对象后，会根据 Response 对象中的调用编号到 FUTURES 集合中取出相应的 DefaultFuture 对象，然后再将 Response 对象设置到 DefaultFuture 对象中。最后再唤醒用户线程，这样用户线程即可从 DefaultFuture 对象中获取调用结果了。整个过程大致如下图：
四、总结
本文中我们讲解了 Dubbo 中的几种服务调用方式，以及从双向通信的角度对整个通信过程进行了详细的分析。 通信的顺序如下：
consumer 发送请求 -> request编码 -> 线程派发 -> provider 接收请求 -> request 解码 -> provider 调用服务 -> 响应请求 -> response 编码 -> consumer 接收请求响应结果 -> response解码