[Heel]
# 1 stick pack phenomenon long each TCP socket connection has its own cache buffer, the default size is 8K, can support manual settings. Long stick package TCP connection is the most common phenomenon, as FIG.
socket cache contained five (5 or package) the heartbeat data, header namely F0 AA 55 0F (hexadecimal), we confirm that by the number of header data out of the cache, there are five heartbeat package, but five each other end to end data linked bond together, this common TCP buffer phenomenon we call stick package.
# 2 ## 2.1 stick package reason. Same client continuously transmits the same client transmits heartbeat data continuously, When the TCP server had a chance to resolve (if parsed the cache will be cleared away). Causing bond the same cached data packets.
## 2.2. Network congestion caused by stick package at a time when network congestion occurs, for a while, suddenly smooth network, TCP server receives more of the same client heartbeat packet, multiple packets are cached in the server-side TCP They were bonded.
## 2.3 server stuck when the server because the calculation of excessive or other reasons, the calculation is slow, too late to deal with TCP Socket data cache, a plurality of heartbeat packets (or other messages) will be in the socket buffer end to end, stick package.
In summary, a plurality of data packets is a phenomenon in the same end to end TCP socket buffer, i.e. stick package phenomenon.
# 3 harm stick package due to the presence of the stick pack phenomenon of objectivity, we must artificially in the program logic in its distinction, if they do not distinguish, let each packet adhesion, has the following hazards: ## 3.1 unable to resolve the server correctly packets will continue to recognize as invalid package, tell the client, the client will be reported again, thus increasing the operating pressure of the client service side, if a large amount of computation itself, it will be some abnormal phenomena Ben collapse .
## 3.2 Error error packets are parsed Coincidentally, if the wrong stick package, happened to be successfully parsed by the server, it will be wrong Handler process. This error handling will endanger more than 3.1.
## 3.3. Enter an infinite loop if the frequency is too fast, this phenomenon occurs, the server continues to identify stick package as invalid packet, the client continue to be reported, in order to consume CPU usage.
In summary, we have to be glued TCP packet processing, which is the basic software system with robust exception handling mechanism.
# 4 logic handling the stick package ## 4.1. Distinguish a predetermined number of bytes per frame trailer wherein TCP packet (for example, four bytes) The end of packet characteristic parameters, retrieving the entire byte cache socket, whenever a packet is detected when the end byte feature, it is divided into packets, in order to correctly split stick package. Wherein: each byte needs to be detected, low efficiency for the short message, if the packet is not suitable for very long.
## 4.2 4.1 distinguish similar, more feature parameters header detection section header trailer. Wherein: each byte detecting only the first frame, a second frame header portion detection only for short and long packets
## 4.3. Distinguish stick package according to the length of the packet according to the packet length of the offset value, the packet of the first frame is read, divided from the adhesive package (socket buffer) correct the first frame message, to find the second frame the packet length, dividing the second frame, in order to divide in the end. Example: 5 as biased length (counted from 0), i.e., the sixth, seventh byte packet length byte.
Wherein: only detection packet length portion, a viscosity suitable length packets into packets.
# 5 The packet length codes to distinguish stick package floor - based NewLife.Net pipeline processing pipeline architecture ## 5.1 NewLife.Net treatment Newlife.Net pipeline architecture, reference is made to the java Netty open source framework, so most Netty codec can be used here. Specific performance in the code
_pemsServer.Add(new StickPackageSplit { Size = 2 });
Coming LengthCodec This codec was added to the pipeline to all message will go through LengthCodec here mainly decoding, no coding (viscosity packets into a plurality of valid packets in accordance with the length of) the decoded successfully pushed OnReceive method to. Size = 2 indicating that the message length is 2 bytes. ## 5.2. Add analogy with pipes and ducts http Net Core of WEBAPI projects, whether found familiar?
app.UseAuthentication();
app.UseRequestLog();
app.UseCors(_defaultCorsPolicyName);
app.UseMvc();
I.e., the sequence of adding pipe sequential data flow through the pipeline. Just did not have to pursue is the first pipeline processing mechanism socket, or pipe handling mechanism http context. But the truth is the same.
## 5.3. Adherend split packet decoder (decoding according to the length) ### 5.3.1. Offset offset length attribute of the position where the length of the offset address. The default is 5, detailed explanation 4.3.
//
// 摘要:
// 长度所在位置
public int Offset
{
get;
set;
} = 5;
### 5.3.2. Of length bytes Size properties discussed herein, the number of length 2 bytes, see 4.3
//
// 摘要:
// 长度占据字节数，1/2/4个字节，0表示压缩编码整数，默认2
public int Size
{
get;
set;
} = 2;
### 5.3.3. Encode encoding method
//
// 摘要:
// 编码,此应用不需要编码，只需解码，
// 按长度将粘包划分成多个数据包
//
// 参数:
// context:
//
// msg:
protected override object Encode(IHandlerContext context, Packet msg)
{
return msg;
}
Here no coding, and therefore directly returned msg.
### 5.3.4. Decode decoding method
//
// 摘要:
// 解码
//
// 参数:
// context:
//
// pk:
protected override IList<Packet> Decode(IHandlerContext context, Packet pk)
{
IExtend extend = context.Owner as IExtend;
LengthCodec packetCodec = extend["Codec"] as LengthCodec;
if (packetCodec == null)
{
IExtend extend2 = extend;
LengthCodec obj = new LengthCodec
{
Expire = Expire,
GetLength = ((Packet p) => MessageCodec<Packet>.GetLength(p, Offset, Size))
};
packetCodec = obj;
extend2["Codec"] = obj;
}
Console.WriteLine("报文解码前：{0}", BitConverter.ToString(pk.ToArray()));
IList<Packet> list = packetCodec.Parse(pk);
Console.WriteLine("报文解码");
foreach (var item in list)
{
Console.WriteLine("粘包处理结果:{0}", BitConverter.ToString(item.ToArray()));
}
return list;
}
#### 5.3.4.1. Step 1 - decoding length decoder instantiates an object instance after completion of the length decoder, and to add it to the dictionary.
IExtend extend2 = extend;
LengthCodec obj = new LengthCodec
{
Expire = Expire,
GetLength = ((Packet p) => MessageCodec<Packet>.GetLength(p, Offset, Size))
};
packetCodec = obj;
extend2["Codec"] = obj;
#### 5.3.4.2. Step 2 - decoding the packet before decoding step Print this non-essential, in order to allow readers to see an increase in the final results.
Console.WriteLine("报文解码前：{0}", BitConverteToString(pk.ToArray()));
#### 5.3.4.3. Step 3 - The decoder decodes packets
IList<Packet> list = packetCodec.Parse(pk);
Decoded code as follows:
//
// 摘要:
// 分析数据流，得到一帧数据
//
// 参数:
// pk:
// 待分析数据包
public virtual IList<Packet> Parse(Packet pk)
{
MemoryStream stream = Stream;
bool num = stream == null || stream.Position < 0 || stream.Position >= stream.Length;
List<Packet> list = new List<Packet>();
if (num)
{
if (pk == null)
{
return list.ToArray();
}
int i;
int num2;
for (i = 0; i < pk.Total; i += num2)
{
Packet packet = pk.Slice(i);
num2 = GetLength(packet);
Console.WriteLine(" pk. GetLength(packet):{0}", num2);
if (num2 <= 0 || num2 > packet.Total)
{
break;
}
packet.Set(packet.Data, packet.Offset, num2);
list.Add(packet);
}
if (i == pk.Total)
{
return list.ToArray();
}
pk = pk.Slice(i);
}
lock (this)
{
CheckCache();
stream = Stream;
if (pk != null && pk.Total > 0)
{
long position = stream.Position;
stream.Position = stream.Length;
pk.CopyTo(stream);
stream.Position = position;
}
while (stream.Position < stream.Length)
{
Packet packet2 = new Packet(stream);
int num3 = GetLength(packet2);
if (num3 <= 0 || num3 > packet2.Total)
{
break;
}
packet2.Set(packet2.Data, packet2.Offset, num3);
list.Add(packet2);
stream.Seek(num3, SeekOrigin.Current);
}
if (stream.Position >= stream.Length)
{
stream.SetLength(0L);
stream.Position = 0L;
}
return list;
}
}
Decoding the core code is as follows: i.e., per frame to obtain the packet length, GetLength (packet) by delegating method, and then loop through all packets stick package, to save list is divided according to the length of the packet of each frame, and finally returned list. Each element of the list will trigger message reception event.
Please use delegates stay at one concern, commissioned see the code 6.
for (i = 0; i < pk.Total; i += num2)
{
Packet packet = pk.Slice(i);
num2 = GetLength(packet);
Console.WriteLine(" pk. GetLength(packet):{0}", num2);
if (num2 <= 0 || num2 > packet.Total)
{
break;
}
packet.Set(packet.Data, packet.Offset, num2);
list.Add(packet);
}
#### 5.3.4.4. The print processing result stick package
foreach (var item in list)
{
Console.WriteLine("粘包处理结果:{0}"BitConverter.ToString(item.ToArray()));
}
### 5.3.5. Empty sticky packet encoder which made NewLife.Net network library calls, we do not care.
//
// 摘要:
// 连接关闭时，清空粘包编码器
//
// 参数:
// context:
//
// reason:
public override bool Close(IHandlerContext contextstring reason)
{
IExtend extend = context.Owner as IExtend;
if (extend != null)
{
extend["Codec"] = null;
}
return base.Close(context, reason);
}
### 5.3.6. Full resolution stick package code decoder
// 摘要:
// 长度字段作为头部
//
public class StickPackageSplit : MessageCodec<Packet>
{
//
// 摘要:
// 长度所在位置
public int Offset
{
get;
set;
} = 5;
//
// 摘要:
// 长度占据字节数，1/2/4个字节，0表示压缩编码整数，默认2
public int Size
{
get;
set;
} = 2;
//
// 摘要:
// 过期时间，超过该时间后按废弃数据处理，默认500ms
public int Expire
{
get;
set;
} = 500;
//
// 摘要:
// 编码,此应用不需要编码，只需解码，
// 按长度将粘包划分成多个数据包
//
// 参数:
// context:
//
// msg:
protected override object Encode(IHandlerContext context, Packet msg)
{
return msg;
}
//
// 摘要:
// 解码
//
// 参数:
// context:
//
// pk:
protected override IList<Packet> Decode(IHandlerContext context, Packet pk)
{
IExtend extend = context.Owner as IExtend;
LengthCodec packetCodec = extend["Codec"] as LengthCodec;
if (packetCodec == null)
{
IExtend extend2 = extend;
LengthCodec obj = new LengthCodec
{
Expire = Expire,
GetLength = ((Packet p) => MessageCodec<Packet>.GetLength(p, Offset, Size))
};
packetCodec = obj;
extend2["Codec"] = obj;
}
Console.WriteLine("报文解码前：{0}", BitConverter.ToString(pk.ToArray()));
IList<Packet> list = packetCodec.Parse(pk);
Console.WriteLine("报文解码");
foreach (var item in list)
{
Console.WriteLine("粘包处理结果:{0}", BitConverter.ToString(item.ToArray()));
}
return list;
}
//
// 摘要:
// 连接关闭时，清空粘包编码器
//
// 参数:
// context:
//
// reason:
public override bool Close(IHandlerContext context, string reason)
{
IExtend extend = context.Owner as IExtend;
if (extend != null)
{
extend["Codec"] = null;
}
return base.Close(context, reason);
}
}
# 6. GetLength 5.3.6 length calculating commission will call length of each packet is calculated as follows delegate. About delegate will use the next one to explain, it will not start.
//
// 摘要:
// 从数据流中获取整帧数据长度
//
// 参数:
// pk:
//
// offset:
//
// size:
//
// 返回结果:
// 数据帧长度（包含头部长度位）
protected static int GetLength(Packet pk, int offsetint size)
{
if (offset < 0)
{
return pk.Total - pk.Offset;
}
int offset2 = pk.Offset;
if (offset >= pk.Total)
{
return 0;
}
int num = 0;
switch (size)
{
case 0:
{
MemoryStream stream = pk.GetStream();
if (offset > 0)
{
stream.Seek(offset, SeekOrigiCurrent);
}
num = stream.ReadEncodedInt();
num += (int)(stream.Position - offset);
break;
}
case 1:
num = pk[offset];
break;
case 2:
num = pk.ReadBytes(offset, 2).ToUInt16();
break;
case 4:
num = (int)pk.ReadBytes(offset, 4).ToUInt32;
break;
case -2:
num = pk.ReadBytes(offset, 2).ToUInt16(0isLittleEndian: false);
break;
case -4:
num = (int)pk.ReadBytes(offset, 4).ToUInt(0, isLittleEndian: false);
break;
default:
throw new NotSupportedException();
}
if (num > pk.Total)
{
return 0;
}
return num;
}
# 7. The final stick package split renderings
Disclaimer: This article is a blogger original article, follow the CC 4.0 BY-SA copyright agreement, reproduced, please attach the original source link and this statement.
This link: https: //www.cnblogs.com/JerryMouseLi/p/12659903.html