关于 LengthFieldBasedFrameDecoder 不得不说的事

LengthFieldBasedFrameDecoder 是什么?

LengthFieldBasedFrameDecoder对于不经常接触netty 的人来说可能有些陌生,但是对于经常使用netty的人来说,对它一定不陌生。

A decoder that splits the received ByteBufs dynamically by the value of the length field in the message. It is particularly useful when you decode a binary message which has an integer header field that represents the length of the message body or the whole message.

一句话,LengthFieldBasedFrameDecoder是一个由netty提供的方便用户切割消息的组件!

LengthFieldBasedFrameDecoder 有什么用?

前面的描述很清楚,LengthFieldBasedFrameDecoder是用来split消息的,那为什么需要切割消息呢?
讲到这就不得不提一下 TCP/IP 协议的 粘包拆包 问题

粘包:指TCP协议中,发送方发送的若干包数据到接收方接收时粘成一包,从接收缓冲区看,后一包数据的头紧接着前一包数据的尾的现象。
拆包:与粘包对应,就是把发生粘包的数据拆分成独立数据包的过程。

粘包拆包发生的根本原因:
TCP是面向流,没有边界,而操作系统在发送TCP数据时,会通过缓冲区来进行优化,例如缓冲区为1024个字节大小。如果一次请求发送的数据量比较小,没达到缓冲区大小,TCP则会将多个请求合并为同一个请求进行发送,这就形成了粘包问题。如果一次请求发送的数据量比较大,超过了缓冲区大小,TCP就会将其拆分为多次发送,这就是拆包。

LengthFieldBasedFrameDecoder就是为了解决我们发送数据过程中的粘包拆包问题。

LengthFieldBasedFrameDecoder 参数详解

我们发现,LengthFieldBasedFrameDecoder构造方法的参数相对还是比较多的,下面我们看看它几个主要参数的意义

maxFrameLength ---- the maximum length of the frame. If the length of the frame is greater than this value, TooLongFrameException will be thrown.
数据包的最大长度,这个没什么好说的,一般我们设置为一个较大的值,比如Integer.MAX_VALUE即可。

扫描二维码关注公众号,回复: 13576957 查看本文章

lengthFieldOffset ---- the offset of the length field.
我们知道,在使用netty的过程中,我们经常自定义协议,而自定义协议中有一个表不可少的元素就是这条消息的长度。说到这儿lengthFieldOffset的意义也就一目了然了,即协议中存储消息长度字段在消息中的偏移量。
在这里插入图片描述

消息长度 消息内容
消息头 消息体
0x000C HELLO, WORLD

如上是一条完整的消息,消息头包含了消息长度一个字段,因此lengthFieldOffset的值为 0

消息头 消息体
序列化方式(1 btye) 消息长度 (4 byte) 消息内容
0x1 0x000C HELLO, WORLD

如上面的例子,消息头可能不止包含消息的长度,还有这条消息对应的其他信息,这时消息的长度字段可能就不是从第一位开始了,因此lengthFieldOffset的值需要根据实际情况调整。上例的lengthFieldOffset值为1,因为消息头的第一个字段为序列化方式,长度为 1 byte

lengthFieldLength ---- the length of the length field.
长度字段的长度,咋一听有点拗口,实际上这个字段很好理解,他和lengthFieldOffset相关联的一个属性,用来告诉LengthFieldBasedFrameDecoderlengthFieldOffset开始读取几个(byte)字节的值作为消息的长度。长度作为一个整形,用来存储整形对应的数据类型(java)分别是ByteShortIntegerLong,占用的字节长度分别为1248,因此lengthFieldLength的值为1248中的一个,这取决于你的协议中是使用了几个字节来存储消息的长度。

lengthAdjustment ---- the compensation value to add to the value of the length field.
要添加到长度字段值的补偿值,听起来就很难理解。
该参数简单来说,就是指Head里除了长度域还剩下几位,比如我的Head由Length和others组成,Length占用4个字节,others占用10个字节,那么这个参数就是10,这个参数的作用就是告诉Netty你的Head的长度是多少。
在这里插入图片描述
拿我 手写RPC里面定义的协议来分析,我定义的协议如下:

消息头 消息体
魔数(2 btye) 协议版本 (1 byte) 序列化方式(1 byte) 消息长度(4 byte) 消息类型(1 byte) 消息id(8 byte) 消息内容(不定长)

我定义的协议消息长度所在字段是第5(魔数(2 btye) + 协议版本(1 btye) + 序列化方式(1 btye))个字节开始,因此lengthFieldOffset = 4,消息长度为 4 byte,因此,lengthFieldLength = 4。又因为我的消息头中表示消息长度字段之后还有消息类型(1 byte)和消息id(8 byte)一共 9 byte,因此,lengthAdjustment = 5
如果把协议调整为

消息头 消息体
魔数(2 btye) 协议版本 (1 byte) 序列化方式(1 byte) 消息类型(1 byte) 消息id(8 byte) 消息长度(4 byte) 消息内容(不定长)

那么将有
lengthFieldOffset = 13
lengthFieldLength = 4
lengthAdjustment = 0

最后一个参数
initialBytesToStrip ---- the number of first bytes to strip out from the decoded frame.
这个也很好理解,解码器解码后,你需要多少数据,就是指舍弃的字节的数量,这里是从前到后舍弃的。比如上面的协议,如果我们希望解码后的消息不要header的内容,只需要消息体的内容,那么 initialBytesToStrip = 17,也即整个消息头占用的字节数。

至此,LengthFieldBasedFrameDecoder的介绍基本结束了,真正学会了你就会发现它的强大。
最后,来几个栗子巩固一下吧

LengthFieldBasedFrameDecoder has many configuration parameters so that it can decode any message with a length field, which is often seen in proprietary client-server protocols. Here are some example that will give you the basic idea on which option does what.
2 bytes length field at offset 0, do not strip header
The value of the length field in this example is 12 (0x0C) which represents the length of "HELLO, WORLD". By default, the decoder assumes that the length field represents the number of the bytes that follows the length field. Therefore, it can be decoded with the simplistic parameter combination.
   lengthFieldOffset   = 0
   lengthFieldLength   = 2
   lengthAdjustment    = 0
   initialBytesToStrip = 0 (= do not strip header)
  
   BEFORE DECODE (14 bytes)         AFTER DECODE (14 bytes)
   +--------+----------------+      +--------+----------------+
   | Length | Actual Content |----->| Length | Actual Content |
   | 0x000C | "HELLO, WORLD" |      | 0x000C | "HELLO, WORLD" |
   +--------+----------------+      +--------+----------------+
   
2 bytes length field at offset 0, strip header
Because we can get the length of the content by calling ByteBuf.readableBytes(), you might want to strip the length field by specifying initialBytesToStrip. In this example, we specified 2, that is same with the length of the length field, to strip the first two bytes.
   lengthFieldOffset   = 0
   lengthFieldLength   = 2
   lengthAdjustment    = 0
   initialBytesToStrip = 2 (= the length of the Length field)
  
   BEFORE DECODE (14 bytes)         AFTER DECODE (12 bytes)
   +--------+----------------+      +----------------+
   | Length | Actual Content |----->| Actual Content |
   | 0x000C | "HELLO, WORLD" |      | "HELLO, WORLD" |
   +--------+----------------+      +----------------+
   
2 bytes length field at offset 0, do not strip header, the length field represents the length of the whole message
In most cases, the length field represents the length of the message body only, as shown in the previous examples. However, in some protocols, the length field represents the length of the whole message, including the message header. In such a case, we specify a non-zero lengthAdjustment. Because the length value in this example message is always greater than the body length by 2, we specify -2 as lengthAdjustment for compensation.
   lengthFieldOffset   =  0
   lengthFieldLength   =  2
   lengthAdjustment    = -2 (= the length of the Length field)
   initialBytesToStrip =  0
  
   BEFORE DECODE (14 bytes)         AFTER DECODE (14 bytes)
   +--------+----------------+      +--------+----------------+
   | Length | Actual Content |----->| Length | Actual Content |
   | 0x000E | "HELLO, WORLD" |      | 0x000E | "HELLO, WORLD" |
   +--------+----------------+      +--------+----------------+
   
3 bytes length field at the end of 5 bytes header, do not strip header
The following message is a simple variation of the first example. An extra header value is prepended to the message. lengthAdjustment is zero again because the decoder always takes the length of the prepended data into account during frame length calculation.
   lengthFieldOffset   = 2 (= the length of Header 1)
   lengthFieldLength   = 3
   lengthAdjustment    = 0
   initialBytesToStrip = 0
  
   BEFORE DECODE (17 bytes)                      AFTER DECODE (17 bytes)
   +----------+----------+----------------+      +----------+----------+----------------+
   | Header 1 |  Length  | Actual Content |----->| Header 1 |  Length  | Actual Content |
   |  0xCAFE  | 0x00000C | "HELLO, WORLD" |      |  0xCAFE  | 0x00000C | "HELLO, WORLD" |
   +----------+----------+----------------+      +----------+----------+----------------+
   
3 bytes length field at the beginning of 5 bytes header, do not strip header
This is an advanced example that shows the case where there is an extra header between the length field and the message body. You have to specify a positive lengthAdjustment so that the decoder counts the extra header into the frame length calculation.
   lengthFieldOffset   = 0
   lengthFieldLength   = 3
   lengthAdjustment    = 2 (= the length of Header 1)
   initialBytesToStrip = 0
  
   BEFORE DECODE (17 bytes)                      AFTER DECODE (17 bytes)
   +----------+----------+----------------+      +----------+----------+----------------+
   |  Length  | Header 1 | Actual Content |----->|  Length  | Header 1 | Actual Content |
   | 0x00000C |  0xCAFE  | "HELLO, WORLD" |      | 0x00000C |  0xCAFE  | "HELLO, WORLD" |
   +----------+----------+----------------+      +----------+----------+----------------+
   
2 bytes length field at offset 1 in the middle of 4 bytes header, strip the first header field and the length field
This is a combination of all the examples above. There are the prepended header before the length field and the extra header after the length field. The prepended header affects the lengthFieldOffset and the extra header affects the lengthAdjustment. We also specified a non-zero initialBytesToStrip to strip the length field and the prepended header from the frame. If you don't want to strip the prepended header, you could specify 0 for initialBytesToSkip.
   lengthFieldOffset   = 1 (= the length of HDR1)
   lengthFieldLength   = 2
   lengthAdjustment    = 1 (= the length of HDR2)
   initialBytesToStrip = 3 (= the length of HDR1 + LEN)
  
   BEFORE DECODE (16 bytes)                       AFTER DECODE (13 bytes)
   +------+--------+------+----------------+      +------+----------------+
   | HDR1 | Length | HDR2 | Actual Content |----->| HDR2 | Actual Content |
   | 0xCA | 0x000C | 0xFE | "HELLO, WORLD" |      | 0xFE | "HELLO, WORLD" |
   +------+--------+------+----------------+      +------+----------------+
   
2 bytes length field at offset 1 in the middle of 4 bytes header, strip the first header field and the length field, the length field represents the length of the whole message
Let's give another twist to the previous example. The only difference from the previous example is that the length field represents the length of the whole message instead of the message body, just like the third example. We have to count the length of HDR1 and Length into lengthAdjustment. Please note that we don't need to take the length of HDR2 into account because the length field already includes the whole header length.
   lengthFieldOffset   =  1
   lengthFieldLength   =  2
   lengthAdjustment    = -3 (= the length of HDR1 + LEN, negative)
   initialBytesToStrip =  3
  
   BEFORE DECODE (16 bytes)                       AFTER DECODE (13 bytes)
   +------+--------+------+----------------+      +------+----------------+
   | HDR1 | Length | HDR2 | Actual Content |----->| HDR2 | Actual Content |
   | 0xCA | 0x0010 | 0xFE | "HELLO, WORLD" |      | 0xFE | "HELLO, WORLD" |
   +------+--------+------+----------------+      +------+----------------+

上面的栗子来源于官方对LengthFieldBasedFrameDecoder使用的说明,感兴趣的可以仔细阅读一下(粘贴起来很辛苦的)。

关于LengthFieldBasedFrameDecoder的介绍就到这里了,由于笔者认知有限,如文中有误,欢迎大家交流指正,感激不尽。

系列文章传送门如下:
手写RPC(一) 絮絮叨叨
手写RPC(二) 碎碎念
手写RPC(三) 基础结构搭建
手写RPC(四) 核心模块网络协议模块编写 ---- netty服务端
手写RPC(五) 核心模块网络协议模块编写 ---- 自定义协议
手写RPC(六) 核心模块网络协议模块编写 ---- 实现编解码器
手写RPC(七) 核心模块网络协议模块编写 ---- 实现客户端
手写RPC(八) provider、consumer 实现
手写RPC(九) 测试
手写RPC(十) 优化

猜你喜欢

转载自blog.csdn.net/hxj413977035/article/details/121633308