Dubbo added
Asynchronous to synchronous mechanism
1. Client execution: AsyncToSyncInvoker#invoke
After invoke#invoke is executed, a Result instance is returned, and the asyncResult.get(Integer.MAX_VALUE, TimeUnit.MILLISECONDS) method is called to block infinitely while waiting for the response result to return;
public class AsyncToSyncInvoker<T> implements Invoker<T> {
private Invoker<T> invoker;
public AsyncToSyncInvoker(Invoker<T> invoker) {
this.invoker = invoker;
}
@Override
public Result invoke(Invocation invocation) throws RpcException {
// 异步转同步
Result asyncResult = invoker.invoke(invocation); // AsyncRpcResult--->CompletableFuture
try {
// 如果invocation指定是同步的，则阻塞等待结果
if (InvokeMode.SYNC == ((RpcInvocation) invocation).getInvokeMode()) {
asyncResult.get(Integer.MAX_VALUE, TimeUnit.MILLISECONDS);
}
} catch (InterruptedException e) {
} catch (Throwable e) {
throw new RpcException(e.getMessage(), e);
}
return asyncResult;
}
Here Future is represented by AsyncToSyncInvoker#asyncResult;
2. Client execution: DubboInvoker#invoke
- Create an instance of AsyncRpcResult asyncRpcResult;
- Call Client to send data, return CompleteFuture instance responseFuture;
- asyncRpcResult.subscribeTo(responseFuture), when responseFuture is completed, asyncRpcResult will be notified;
- return asyncRpcResult;
That is, when responseFuture#complete is completed, asyncRpcResult#get will be notified to unblock.
public class DubboInvoker<T> extends AbstractInvoker<T> {
@Override
protected Result doInvoke(final Invocation invocation) throws Throwable {
RpcInvocation inv = (RpcInvocation) invocation;
ExchangeClient currentClient;
if (clients.length == 1) {
currentClient = clients[0];
} else {
// 轮询使用clients
currentClient = clients[index.getAndIncrement() % clients.length];
}
try {
// isOneway为true，表示请求不需要拿结果
boolean isOneway = RpcUtils.isOneway(getUrl(), invocation);
// 拿当前方法的所配置的超时时间，默认为1000,1秒
int timeout = getUrl().getMethodPositiveParameter(methodName, TIMEOUT_KEY, DEFAULT_TIMEOUT);
if (isOneway) {
//省略
} else {
AsyncRpcResult asyncRpcResult = new AsyncRpcResult(inv);
// 异步去请求，得到一个CompletableFuture
CompletableFuture<Object> responseFuture = currentClient.request(inv, timeout);
// responseFuture会完成后会调用asyncRpcResult中的方法，这里并不会阻塞，如果要达到阻塞的效果在外层使用asyncRpcResult去控制
asyncRpcResult.subscribeTo(responseFuture);
// save for 2.6.x compatibility, for example, TraceFilter in Zipkin uses com.alibaba.xxx.FutureAdapter
FutureContext.getContext().setCompatibleFuture(responseFuture);
return asyncRpcResult;
}
} catch (TimeoutException e) {
} catch (RemotingException e) {
}
}
}
Execute the client here to return responseFuture and record it as DubboInvoker#responseFuture
3. Client execution: HeaderExchangeChannel#request
Create a DefaultFuture#newFuture method, create an instance of DefaultFuture, set DefaultFuture as value, and put the key as the request ID into the FUTURES cache;
@Override
public CompletableFuture<Object> request(Object request, int timeout) throws RemotingException {
// create request.
Request req = new Request();
req.setVersion(Version.getProtocolVersion());
req.setTwoWay(true);
req.setData(request);
DefaultFuture future = DefaultFuture.newFuture(channel, req, timeout);
try {
channel.send(req);
} catch (RemotingException e) {
future.cancel();
throw e;
}
return future;
}
DefaultFuture#newFuture
Maintain a local cache FUTURES instance;
public class DefaultFuture extends CompletableFuture<Object> {
private static final Logger logger = LoggerFactory.getLogger(DefaultFuture.class);
private static final Map<Long, Channel> CHANNELS = new ConcurrentHashMap<>();
private static final Map<Long, DefaultFuture> FUTURES = new ConcurrentHashMap<>();
public static final Timer TIME_OUT_TIMER = new HashedWheelTimer(
new NamedThreadFactory("dubbo-future-timeout", true),
30,
TimeUnit.MILLISECONDS);
// invoke id.
private final Long id;
private final Channel channel;
private final Request request;
private final int timeout;
private final long start = System.currentTimeMillis();
private volatile long sent;
private Timeout timeoutCheckTask;
private DefaultFuture(Channel channel, Request request, int timeout) {
this.channel = channel;
this.request = request;
this.id = request.getId();
this.timeout = timeout > 0 ? timeout : channel.getUrl().getPositiveParameter(TIMEOUT_KEY, DEFAULT_TIMEOUT);
// put into waiting map.
FUTURES.put(id, this);
CHANNELS.put(id, channel);
}
}
So far, the relationship is as follows: AsyncToSyncInvoker#asyncResult => Subscription => DubboInvoker#responseResult => Put in cache => DEFUTURES#DefaultFuture;
4. Server execution: AbstractProxyInvoker#invoke
- Call the doInvoke method, pass in the proxy proxy instance, method name methodName, parameter type parameterTypes, method parameter arguments; return an Object value;
- Install the Object return value value wave as the CompleteFuture instance future;
- Create an instance of AsyncRpcResult asyncRpcResult
- When the method is executed, the setting will call the method future.whenComplete. In the callback method, create an AppResponse instance, which represents the content of the response, and set the content of the response according to whether an exception occurs; then call the method of asyncRpcResult#complete to set the content of the response result to asyncRpcResult
- return asyncRpcResult;
public abstract class AbstractProxyInvoker<T> implements Invoker<T> {
@Override
public Result invoke(Invocation invocation) throws RpcException {
try {
// 执行服务，得到一个接口，可能是一个CompletableFuture(表示异步调用)，可能是一个正常的服务执行结果（同步调用）
// 如果是同步调用会阻塞，如果是异步调用不会阻塞
Object value = doInvoke(proxy, invocation.getMethodName(), invocation.getParameterTypes(), invocation.getArguments());
// 将同步调用的服务执行结果封装为CompletableFuture类型
CompletableFuture<Object> future = wrapWithFuture(value, invocation);
// 异步RPC结果
AsyncRpcResult asyncRpcResult = new AsyncRpcResult(invocation);
//设置一个回调，如果是异步调用，那么服务执行完成后将执行这里的回调
// 不会阻塞
future.whenComplete((obj, t) -> {
AppResponse result = new AppResponse();
if (t != null) {
if (t instanceof CompletionException) {
result.setException(t.getCause());
} else {
result.setException(t);
}
} else {
result.setValue(obj);
}
// 将服务执行完之后的结果设置到异步RPC结果对象中
asyncRpcResult.complete(result);
});
// 返回异步RPC结果
return asyncRpcResult;
} catch (InvocationTargetException e) {
// 假设抛的NullPointException，那么会把这个异常包装为一个Result对象
return AsyncRpcResult.newDefaultAsyncResult(null, e.getTargetException(), invocation);
} catch (Throwable e) {
//抛出异常
}
private CompletableFuture<Object> wrapWithFuture (Object value, Invocation invocation) {
if (RpcContext.getContext().isAsyncStarted()) {
return ((AsyncContextImpl)(RpcContext.getContext().getAsyncContext())).getInternalFuture();
} else if (value instanceof CompletableFuture) {
return (CompletableFuture<Object>) value;
}
return CompletableFuture.completedFuture(value);
}
protected abstract Object doInvoke(T proxy, String methodName, Class<?>[] parameterTypes, Object[] arguments) throws Throwable;
The Future here is recorded as AbstractProxyInvoker#future, and asyncRpcResult returns the HeaderExchangeHandler#future of the previous layer
2. The server executes HeaderExchangeHandler#hadnleRequest
- Create a Response object instance; set the request ID, version number;
- Call the next handler#reply method to process the request and return a CompletionStage instance, which is the asyncRpcResult instance returned above.
- Call the CompletionStage#whenComplete method, if the request execution is completed, the callback method will be triggered;
[When (1) AbstractProxyInvoker#invoke calls the asyncRpcResult#complete method, the current whenComplete method will be triggered]
If an exception occurs during execution, , the return status is service call error SERVICE_ERROR, and the result result is set to exception information;
if no exception occurs during execution, the return status is Response.OK, and the return result is set to apResult, which represents the data returned by the request - Call the channel#send method to send data to the client;
void handleRequest(final ExchangeChannel channel, Request req) throws RemotingException {
// 请求id，请求版本
Response res = new Response(req.getId(), req.getVersion());
// 获取 data 字段值，也就是 RpcInvocation 对象，表示请求内容
// find handler by message class.
Object msg = req.getData();
try {
// 继续向下调用，分异步调用和同步调用，如果是同步则会阻塞，如果是异步则不会阻塞
CompletionStage<Object> future = handler.reply(channel, msg); // 异步执行服务
// 如果是同步调用则直接拿到结果，并发送到channel中去
// 如果是异步调用则会监听，直到拿到服务执行结果，然后发送到channel中去
future.whenComplete((appResult, t) -> {
try {
if (t == null) {
res.setStatus(Response.OK);
res.setResult(appResult);
} else {
// 服务执行过程中出现了异常，则把Throwable转成字符串，发送给channel中，也就是发送给客户端
res.setStatus(Response.SERVICE_ERROR);
res.setErrorMessage(StringUtils.toString(t));
}
channel.send(res);
} catch (RemotingException e) {
logger.warn("Send result to consumer failed, channel is " + channel + ", msg is " + e);
} finally {
// HeaderExchangeChannel.removeChannelIfDisconnected(channel);
}
});
} catch (Throwable e) {
res.setStatus(Response.SERVICE_ERROR);
res.setErrorMessage(StringUtils.toString(e));
channel.send(res);
}
}
Here, the server-side Future relationship AbstractProxyInvoker#future#complete => trigger => AbstractProxyInvoker#future#whenComplete callback, call HeaderExchangeHandler#future#complete =>
trigger HeaderExchangeHandler#future#whenComplete => channel#send returns the response data Response;
3. The client executes HeaderExchangeHandler#hadnleResponse
After the client sends the request, the server returns the response data. The data type received by the client is Response, and handleResponse will be called to process the response result;
static void handleResponse(Channel channel, Response response) throws RemotingException {
if (response != null && !response.isHeartbeat()) {
DefaultFuture.received(channel, response);
}
}
- When the client requests data, Netty's channel#send method sends data asynchronously, returns immediately after sending, and has no return value. In order to wait for the returned data, the AsyncToSyncInvoker asynchronous-to-synchronous executor will wait indefinitely;
- When the client calls the request method in the HeaderExchangeChannel, it will create a DefaultFuture instance and put it in the cache DEFUTURES, the key is the request ID, and the value is DefaultFuture;
- When the response result is returned, in DefaultFuture#received, the DefaultFuture instance will be obtained from the Futures cache according to the ID of the response. If the future is empty, it means that it has timed out. If it is not empty, call the future#doreceived method to process the return value;
- In doReceived, if the status is Response.OK, set the return value appResult to the future through Future#complete, which represents the return value of the blocked future, that is, the server execution result
- If a timeout occurs, call completeExceptionally to throw a timeout exception;
- If a business exception occurs, the remote call exception information will be displayed on the client as the return value;
- This is Dubbo's asynchronous-to-synchronous mechanism; the channel between Spring and Redis, and openFeign's asynchronous-to-synchronous mechanism are similar;
public class DefaultFuture extends CompletableFuture<Object> {
public static void received(Channel channel, Response response) {
received(channel, response, false);
}
public static void received(Channel channel, Response response, boolean timeout) {
try {
// response的id，
DefaultFuture future = FUTURES.remove(response.getId());
if (future != null) {
Timeout t = future.timeoutCheckTask;
if (!timeout) {
// decrease Time
t.cancel();
}
future.doReceived(response);
} else {
logger.warn("The timeout response finally returned at "
+ (new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS").format(new Date()))
+ ", response " + response
+ (channel == null ? "" : ", channel: " + channel.getLocalAddress()
+ " -> " + channel.getRemoteAddress()));
}
} finally {
CHANNELS.remove(response.getId());
}
}
private void doReceived(Response res) {
if (res.getStatus() == Response.OK) {
this.complete(res.getResult());
} else if (res.getStatus() == Response.CLIENT_TIMEOUT || res.getStatus() == Response.SERVER_TIMEOUT) {
this.completeExceptionally(new TimeoutException(res.getStatus() == Response.SERVER_TIMEOUT, channel, res.getErrorMessage()));
} else {
this.completeExceptionally(new RemotingException(channel, res.getErrorMessage()));
}
}
}
At this point, when the server sends Response data to the client, => DEFUTURES#DefaultFuture#complete method;
it means DubboInvoker#responseResult execution is completed, and it notifies =》AsyncToSyncInvoker#asyncResult#get to unblock;
Draw a picture as follows:
Dubbo exception handling
InvokerInvocationHandler#invoke
public class InvokerInvocationHandler implements InvocationHandler {
private static final Logger logger = LoggerFactory.getLogger(InvokerInvocationHandler.class);
private final Invoker<?> invoker;
public InvokerInvocationHandler(Invoker<?> handler) {
this.invoker = handler;
}
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
String methodName = method.getName();
// 这里的recreate方法很重要，他会调用AppResponse的recreate方法，
// 如果AppResponse对象中存在exception信息，则此方法中会throw这个异常
return invoker.invoke(new RpcInvocation(method, args)).recreate();
}
}
invoker.invoke(new RpcInvocation(method, args)) returns a Response object instance, which is the response result;
call the recreate method to obtain the response content AppResponse;
recreate#recreate
Call the getAppResponse method to obtain the response content AppResponse; then call the AppResponse#recreate method;
public class AsyncRpcResult extends AbstractResult {
@Override
public Object recreate() throws Throwable {
RpcInvocation rpcInvocation = (RpcInvocation) invocation;
FutureAdapter future = new FutureAdapter(this);
RpcContext.getContext().setFuture(future);
if (InvokeMode.FUTURE == rpcInvocation.getInvokeMode()) {
return future;
}
return getAppResponse().recreate();
}
}
AppResponse#recreate
Determine whether the exception of the response content exists;
get the exception type if it exists;
set the error stack information,
@Override
public Object recreate() throws Throwable {
if (exception != null) {
// fix issue#619
try {
// get Throwable class
Class clazz = exception.getClass();
while (!clazz.getName().equals(Throwable.class.getName())) {
clazz = clazz.getSuperclass();
}
// get stackTrace value
Field stackTraceField = clazz.getDeclaredField("stackTrace");
stackTraceField.setAccessible(true);
Object stackTrace = stackTraceField.get(exception);
if (stackTrace == null) {
exception.setStackTrace(new StackTraceElement[0]);
}
} catch (Exception e) {
// ignore
}
throw exception;
}
return result;
}
When the service provider executes the service, if an exception occurs, the framework will capture the exception. The logic of catching the exception is in AbstractProxyInvoker. After catching the exception, it will package the exception information into a normal AppResponse object, but the value attribute of AppResponse is not Value, the exception attribute has a value;
After that, the service provider will send the AppResponse object to the service consumer. The service consumer calls the recreate method of AppResponse in the InvokerInvocationHandler to get a new result. In the recreate method, it will fail whether the AppResponse object is normal, that is, whether there is an exception Information, if it exists, throw this exception directly, so that the exception that occurs during service execution is thrown at the service consumer.
So there is a problem here, if the exception class thrown by the service provider does not exist on the service consumer side, then the service consumer will not be able to throw this exception, so how does dubbo handle it?
This involves ExceptionFilter, which is a filter on the service provider side. It mainly identifies exceptions after the service provider finishes executing the service:
- If it is an exception that needs to be caught by the developer, ignore it and return the exception directly to the consumer
- If there is a statement in the currently executed method signature, ignore it and return the exception directly to the consumer
- If the thrown exception does not need to be caught by the developer, or there is no statement on the method, then the server will record an error log
- If the exception class and the interface class are in the same jar package, ignore it and return the exception directly to the consumer
- If the exception class is an exception that comes with JDK, ignore it and return the exception to the consumer directly
- If the exception class is an exception that comes with Dubbo, ignore it and return the exception to the consumer directly
- Otherwise, wrap the exception information into RuntimeException and override the exception attribute in the AppResponse object
Summarize
Consumer
- MockClusterInvoker.invoke(new RpcInvocation(method, args))：Mock逻辑
- AbstractClusterInvoker.invoke(invocation): Add the Attachments set in RpcContext to the invocation object, call the routing chain to filter out the suitable service Invoker from the service directory, and obtain the service balancing policy loadbalance
- FailoverClusterInvoker.doInvoke(invocation, invokers, loadbalance): Select an invoker according to the load balancing strategy, and then execute
- InvokerWrapper.invoke(invocation): did nothing
- CallbackRegistrationInvoker.invoke(invocation): Start to execute the Filter chain. After the execution is completed, the listener in the ListenableFilter will be obtained and the onResponse method of the listener will be executed.
- ConsumerContextFilter.invoke(invocation): Set LocalAddress, RemoteAddress, RemoteApplicationName parameters in RpcContext
- FutureFilter.invoke(invocation)：
- MonitorFilter.invoke(invocation): The number of executions of the method +1
- ListenerInvokerWrapper.invoke(invocation): did not do anything
- AsyncToSyncInvoker.invoke(invocation): Asynchronous to synchronous, it will first use the underlying Invoker to execute asynchronously, and then block the Integer.MAX_VALUE time until the result is obtained
- AbstractInvoker.invoke(invocation): It mainly calls the doInvoke method of DubboInvoker. If there is an exception in the doInvoker method, it will be packaged and packaged into AsyncRpcResult
- DubboInvoker.doInvoke(invocation): Polling a client from clients to send data. If it is configured not to care about the result, call the send method of ReferenceCountExchangeClient, otherwise call the request method of ReferenceCountExchangeClient
- ReferenceCountExchangeClient.request(Object request, int timeout): did nothing
- HeaderExchangeClient.request(Object request, int timeout): did not do anything
- HeaderExchangeChannel.request(Object request, int timeout): Construct a Request object, and construct a DefaultFuture object to block the timeout time to wait for the result. When constructing the DefaultFuture object, the DefaultFuture object and req id will be stored in FUTURES, FUTURES is a Map. When HeaderExchangeHandler receives the result, it will get the DefaultFuture object from the Map according to the id, and then return the Response.
- AbstractPeer.send(Object message): Get the send parameter from the url, the default is false
- AbstractClient.send(Object message, boolean sent): does nothing
- NettyChannel.send(Object message, boolean sent): Call writeAndFlush of NioSocketChannel to send data, and then judge if send is true, then block the timeout time specified in url, because if send is false, timeout time will be blocked in HeaderExchangeChannel
- NioSocketChannel.writeAndFlush(Object msg): The bottom-level Netty non-blocking sending data
Consumer summary
- The outermost layer is Mock logic, Mock before calling and after calling
- From the service directory, filter out some service Invokers (DubboInvoker) according to the currently invoked method and routing chain
- Load balance the service Invoker and select a service Invoker
- Execute the Filter chain
- AsyncToSyncInvoker completes asynchronous to synchronous, because the execution of DubboInvoker is asynchronous and non-blocking, so if it is a synchronous call, it will block here until the response result is obtained
- DubboInvoker starts an asynchronous non-blocking call
- HeaderExchangeChannel will block the timeout time to wait for the result. This timeout is the timeout configured by the user on the consumer side.
Server
- NettyServerHandler: Receive data
- MultiMessageHandler: Determine whether the received data is a MultiMessage, if so, obtain a single Message in the MultiMessage, and pass it to HeartbeatHandler for processing
- HeartbeatHandler: Determine whether it is a heartbeat message, and if not, pass the Message to AllChannelHandler
- AllChannelHandler: Encapsulate the received Message into a ChannelEventRunnable object and throw it to the thread pool for processing
- ChannelEventRunnable: In the run method of ChannelEventRunnable, DecodeHandler will be called to process Message
- DecodeHandler: According to the data format of the Dubbo protocol, parse the path, versio, method, method parameters, etc. of the current request, and then pass the parsed request to HeaderExchangeHandler
- HeaderExchangeHandler: To process Request data, first construct a Response object, then call the ExchangeHandlerAdapter to get a CompletionStage future, and then bind a callback function to the future through whenComplete. After the future is executed, you can get the execution result of the ExchangeHandlerAdapter from the callback function, and Set the execution result to the Response object and send it out through the channel.
- ExchangeHandlerAdapter: Find the Exporter object from the exported Exporter of the machine according to the service key corresponding to the current Request, get the Invoker from the Exporter, and then execute the invoke method. The Invoker is ProtocolFilterWrapper$CallbackRegistrationInvoker
- ProtocolFilterWrapper$CallbackRegistrationInvoker: Responsible for executing the filter chain, and calling back the onResponse or onError method of each filter after execution
- EchoFilter: Determine whether the current request is an echo test, if so, the filter chain will not continue to be executed (the service implementer Invoker will not call it)
- ClassLoaderFilter: Set the classloader of the current thread to the classloader corresponding to the service interface to be executed currently
- GenericFilter: wrap the information sent by the generalization call into an RpcInvocation object
- ContextFilter: Set the parameters of RpcContext.getContext()
- TraceFilter: Execute the invoke method of the next invoker first, and record the call information after the call is successful
- TimeoutFilter: There is no special processing when calling, just record the current time, when the onResponse method of TimeoutFilter is called back after the entire filter chain is executed, it will judge whether the call has exceeded the timeout
- MonitorFilter: Record the execution times of the current service
- ExceptionFilter: There is no special processing when calling. When the onResponse method is called back, different exceptions are processed. Explain Dubbo's exception handling in detail
- DelegateProviderMetaDataInvoker: The end of the filter chain, call the next Invoker
- AbstractProxyInvoker: When the service is exported, according to the service interface, the service implementation class object is generated, and its invoke method will execute the method of the service implementation class object to get the result