On the way to work, a question suddenly popped up: since dubbo describes the use of a single long connection between the consumer and the provider, if the consumer side is a web application with a high concurrency and multi-threading model, how about a single long connection? What about solving the problem of multi-threaded concurrent requests?
In fact, if you don't know much about sockets or multi-threaded programming, it's not easy to understand this problem. The traditional and simplest RPC method should be to create a corresponding thread for each remote call request. Let's not talk about the disadvantages of this method. At least the advantage is obvious, that is, simplicity. Where is the simplicity manifested?
One-to-one communication between the two parties (compared to NIO).
In layman's terms, the two sides of socket communication send and receive data without being interfered by other (threads), which is different from the "sticky packet problem" of data packets. In fact, it is equivalent to the scene of the telephone line:
Just imagine if multiple people are shouting into the same microphone at the same time, the other party's voice will be overlapping and cluttered.
For a single socket channel, if the sender is multi-threaded, if it is not controlled, the data in the channel will be messed up, and the receiver cannot distinguish the unit of the data, so it cannot process the request correctly.
At first glance, it seems that the single long connection described by the dubbo protocol and the client's multi-threaded concurrent requests are incompatible. But in fact, a little design, you can make them live in harmony.
How to solve the problem of sticky packets in socket? The most used is actually to define a fixed-length data packet header, which contains the length of the complete data packet, in order to complete the server-side unpacking work.
Then the method of solving the packet interference when multiple threads use a single long connection concurrent request is also a bit similar, that is to add an identification id to the packet header, and the server side should also carry this id when responding to the request, so that the client can receive the corresponding response data from multiple threads. clue.
In fact, if performance is not considered, dubbo can also create a corresponding server-side thread for each client thread, but this is unacceptable for massive high-concurrency scenarios~~
Then make up a picture:
Below we try to find traces from the code.
Tracking all the way, we came to this class: com.alibaba.dubbo.remoting.exchange.support.header.HeaderExchangeChannel.java, let's take a look at the requestmethods, probably around line 101:
public ResponseFuture request(Object request, int timeout) throws RemotingException { if (closed) { throw new RemotingException(this.getLocalAddress(), null, "Failed to send request " + request + ", cause: The channel " + this + " is closed!"); } // create request. Request req = new Request(); req.setVersion("2.0.0"); req.setTwoWay(true); req.setData(request); //这个future就是前面我们提到的：客户端并发请求线程阻塞的对象 DefaultFuture future = new DefaultFuture(channel, req, timeout); try{ channel.send(req); //非阻塞调用 }catch (RemotingException e) { future.cancel(); throw e; } return future; } Pay attention to the ResponseFutureobject returned by this method. The thread currently processing the client request will get the ResponseFutureobject after a series of calls, and finally the thread will block on the following method call of this object, as follows:
public Object get(int timeout) throws RemotingException {
if (timeout <= 0) {
timeout = Constants.DEFAULT_TIMEOUT;
}
if (! isDone()) { long start = System.currentTimeMillis(); lock.lock(); try { while (! isDone()) { //无限连 done.await(timeout, TimeUnit.MILLISECONDS); if (isDone() || System.currentTimeMillis() - start > timeout) { break; } } } catch (InterruptedException e) { throw new RuntimeException(e); } finally { lock.unlock(); } if (! isDone()) { throw new TimeoutException(sent > 0, channel, getTimeoutMessage(false)); } } return returnFromResponse(); } I have seen above that the request thread has been blocked, so how is it woken up? Looking at it again com.alibaba.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.java, in fact, all ChannelHandlerclasses that implement an interface are designed with the decorator pattern, so you can see code like this:
protected ChannelHandler wrapInternal(ChannelHandler handler, URL url) { return new MultiMessageHandler( new HeartbeatHandler( ExtensionLoader.getExtensionLoader(Dispather.class).getAdaptiveExtension().dispath(handler, url) )); } Now let's take a closer look at HeaderExchangeHandlerthe definition of the class, first look at the receivedmethods it defines, here is the code snippet:
public void received(Channel channel, Object message) throws RemotingException {
channel.setAttribute(KEY_READ_TIMESTAMP, System.currentTimeMillis());
ExchangeChannel exchangeChannel = HeaderExchangeChannel.getOrAddChannel(channel);
try {
if (message instanceof Request) { ..... } else if (message instanceof Response) { //这里就是作为消费者的dubbo客户端在接收到响应后，触发通知对应等待线程的起点 handleResponse(channel, (Response) message); } else if (message instanceof String) { ..... } else { handler.received(exchangeChannel, message); } } finally { HeaderExchangeChannel.removeChannelIfDisconnected(channel); } } We mainly look at the conditional branch in the middle, which is used to process the response message, that is to say, when the dubbo client receives the response from the server, it will execute this branch. It simply calls the handleResponsemethod, let's follow it to see :
static void handleResponse(Channel channel, Response response) throws RemotingException { if (response != null && !response.isHeartbeat()) { //排除心跳类型的响应 DefaultFuture.received(channel, response); } } Familiar figure: DefaultFuture, it implements the interface type we mentioned above . In fact, careful children's shoes should be able to see that this object is actually instantiated in the ResponseFutureabove method:requestDefaultFutrue
DefaultFuture future = new DefaultFuture(channel, req, timeout);
Then we can continue to look at DefaultFuture.receivedthe implementation details of the method:
public static void received(Channel channel, Response response) {
try {
DefaultFuture future = FUTURES.remove(response.getId());
if (future != null) { future.doReceived(response); } else { logger.warn("The timeout response finally returned at " + (new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS").format(new Date())) + ", response " + response + (channel == null ? "" : ", channel: " + channel.getLocalAddress() + " -> " + channel.getRemoteAddress())); } } finally { CHANNELS.remove(response.getId()); } } Leaving the role of the id we mentioned earlier, here we can see that it has come into play. Through id, DefaultFuture.FUTURESyou can get the specific DefaultFutureobject, which is the object we mentioned above that blocks the request thread. Well, after finding the target, call its doReceivedmethod, which is standard java multi-threaded programming knowledge:
private void doReceived(Response res) { lock.lock(); try { response = res; if (done != null) { done.signal(); } } finally { lock.unlock(); } if (callback != null) { invokeCallback(callback); } } This way we can confirm the two points marked by the green arrow on the left in the image above.
Next, let's take a look at how the two points mentioned by the green arrow on the right are implemented? In fact, dubbo relies on netty by default in the implementation of NIO, which means that the coolie that really sends and receives packets at both ends of the long connection is netty. Since I am not very familiar with netty, for the time being, we will directly use netty as a black box, just know that it can complete NIO communication well.
Reference link: https://blog.csdn.net/joeyon1985/article/details/51046548