Only in the network transmission of the original data as a sequence of bytes. What, then, our application requires these byte sequence of meaningful information. The application data is converted to network format, and the format for the network component converts the application data are called encoder and decoder, while having a single element both functions is called a codec.
1, & stick package unpacking
Based on the above analysis we know dubbo remote calls are the Nio Netty framework based on TCP / IP-Socket-based communication.
TCP is a "flow" of the agreement, the so-called stream is no limit to the string of data. You can imagine a river of water are contiguous, during which there is no dividing line. TCP does not know the specific meaning of the underlying upper layer service data, it will be divided based on the actual case of TCP packet buffer, so that the operations, a full TCP packet may be split into a plurality of packets transmitted. It is also possible to package a plurality of smaller packets into a larger packet transmission, which is called a TCP packet and unpacking sticky problems.
1.1 TCP stick package & unpacking description of the problem
Here will be described the TCP packet unpacking viscosity problem by the following diagram:
Suppose the client send two packets D1 and D2 to the server, the server since the time the read byte data is uncertain, it may be the following four cases.
- Twice server to read two separate data packets, respectively D1 and D2, and not stick package unpacking
- The server receives a packet two, D1 and D2 are bonded together, referred to as TCP packets adherend
- Twice server to read two data packets, first read into the interior portion of the complete packet D1 and D2 of the packet, the remainder of the second reading to the internal packet D2, which is referred to as TCP unpacking
- Twice server to read two data packets, the first read part to the package interior D1_1 - D1, the second reading to the packets D1 and D2 of the remaining packages inside D1_2 whole package.
If at this time the server receives the TCP sliding window is very small, and the data packet D1 and D2 is relatively large, is likely to occur fifth, that service will end points several times to complete packet reception D1 and D2, occur several times during the unpacking.
1.2 & solve the stick package unpacking
Because the underlying TCP upper layer service data can not be understood, it is not guaranteed at the bottom of the packet is not split and recombined. This problem can only be solved by the application of the upper layer protocol stack design, mainstream solutions:
- Message length, for example, the size of each packet is a fixed length of 200 bytes, if enough space is blank.
- Increase in trailer carriage returns divided, e.g. TFP protocol
- Message into the message header and a message body, the message header includes information indicating the total length of the message (or message body length) field.
netty For the first two have their own realization, and dubbo used are three kinds of stick package to solve the problem and unpacking.
2, dubbo custom protocol
Netty For developers, in fact, operating ChannelHandler this component. Before we analyze the network to send and receive dubbo request is to achieve NettyServerHandler ChannelHandler of. For to achieve ChannelHandler codec is also carried out.
NettyServer#doOpen
protected void doOpen() throws Throwable {
bootstrap = new ServerBootstrap();
bossGroup = new NioEventLoopGroup(1, new DefaultThreadFactory("NettyServerBoss", true));
workerGroup = new NioEventLoopGroup(getUrl().getPositiveParameter(Constants.IO_THREADS_KEY, Constants.DEFAULT_IO_THREADS),
new DefaultThreadFactory("NettyServerWorker", true));
final NettyServerHandler nettyServerHandler = new NettyServerHandler(getUrl(), this);
channels = nettyServerHandler.getChannels();
bootstrap.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childOption(ChannelOption.TCP_NODELAY, Boolean.TRUE)
.childOption(ChannelOption.SO_REUSEADDR, Boolean.TRUE)
.childOption(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT)
.childHandler(new ChannelInitializer<NioSocketChannel>() {
@Override
protected void initChannel(NioSocketChannel ch) throws Exception {
NettyCodecAdapter adapter = new NettyCodecAdapter(getCodec(), getUrl(), NettyServer.this);
ch.pipeline()//.addLast("logging",new LoggingHandler(LogLevel.INFO))//for debug
.addLast("decoder", adapter.getDecoder())
.addLast("encoder", adapter.getEncoder())
.addLast("handler", nettyServerHandler);
}
});
// bind
ChannelFuture channelFuture = bootstrap.bind(getBindAddress());
channelFuture.syncUninterruptibly();
channel = channelFuture.channel();
}Performing the service time of exposure, in particular when added initialized ChannelHandler encoding, decoding and processing the request in response to implementation class ChannelPipeline. dubbo particular codec is handled by NettyCodecAdapter.
Here we look at dubbo protocol headers conventions:
dubbo having a length of 16 byte array as the protocol header. 8 correspond to one byte. Therefore dubbo protocol header 128 bits (i.e. the figure from 0 to 127). We look at this 128 protocol headers represent what it means.
- 0 ~ 7: dubbo magic number (
(short) 0xdabb) high, i.e. (short) 0xda. - 8 ~ 15: dubbo magic number (
(short) 0xdabb) low, i.e. (short) 0xbb. - 16 ~ 20：序列化 id(Serialization id)，也就是 dubbo 支持的序列化中的
contentTypeId，比如 Hessian2Serialization#ID 为 2 - 21 ：是否事件(event )
- 22 ： 是否 Two way 模式(Two way)。默认是 Two-way 模式，
<dubbo:method>标签的 return 属性配置为false，则是oneway模式 - 23 ：标记是请求对象还是响应对象(Req/res)
- 24 ~ 31：response 的结果响应码 ，例如 OK=20
- 32 ~ 95：id(long)，异步变同步的全局唯一ID，用来做consumer和provider的来回通信标记。
- 96 ~ 127： data length，请求或响应数据体的数据长度也就是消息头+请求数据的长度。用于处理 dubbo 通信的粘包与拆包问题。
我们就根据源码来分析一下 dubbo 是如何进行编解码的。
3、协议源码分析
dubbo 的编解码可以分为以下 4 个部分来分析：
- consumer 请求编码
- consumer响应结果解码
- provider 请求解码
- provider 响应结果编码
在 dubbo 进行服务暴露的时候是通过 NettyCodecAdapter 来获取到需要添加的编码器与解码器。在 NettyCodecAdapter 里面定义内部类 InternalEncoder (继承 netty 中的 MessageToByteEncoder)实现 dubbo 的自定义编码器，定义内部类 ByteToMessageDecoder (继承 netty 中的 ByteToMessageDecoder) 实现 dubbo 自定义解码器。不管是自定义的编码器还是解码器最终都会调用到 dubbo 的 SPI 接口 Codec2 默认使用 DubboCodec。下面就具体的分析一下 dubbo 这 4 个编解码过程。
3.1 consumer request encoding
When the consumer requests provider Request object needs to be converted into a byte array, so it is a process that requires coding.
3.2 consumer response to the decoded result
On receiving the response to consumer provider needs to be converted into a byte array Response object, it is a process of decoding needs.
3.3 provider request decoder
provider when receiving consumer requests need to be converted into a byte array Request object, it is a process of decoding needs.
3.4 encoded response results
provider response is required when the consumer requests processed results need to be converted into byte array Response object, so it is a process that requires coding.
