In the usual client socket development, if the client continuously send data packets to the server, the server receives the data packet will be the case of two stick together, which is stuck in the TCP protocol commonly encountered problems package and unpacking.
We all know that TCP is the transport layer protocol, in addition to the transport layer TCP protocol but also have the UDP protocol. So whether UDP stick package or unpacking phenomenon will happen then? The answer is no. UDP packets are sent on, it can be seen from the frame structure of the UDP, the UDP header to indicate the use of the 16bit length of the UDP data packets , and therefore can well separate different areas of the data packet at the application layer, thereby stick package to avoid problems and unpacking. TCP is a byte-oriented and, although the data exchange between the application layer and the TCP transport layer is a data block sizes, but it is only TCP these data blocks as a stream of bytes unstructured series, no border ; Further can be seen from the frame structure of the TCP, the TCP header length field is not data indicating , based on the above points, when data transfer using TCP, it may have a stick package or unpacking phenomenon occurs.
In order to improve the performance of TCP, transmission data to be transmitted to the terminal will buffer, wait for the buffer is full, then the data sent to the receiver's buffer. Similarly, the receiver also has such a buffer mechanism, to receive data.
Stick package / unpacking manifestations
Now assume that the client to the server continuously transmits two data packets , and with packet1 represented packet2, the server receives the data may be divided into three, we are listed below:
The first case, the receiving terminal normally receives two packets, i.e., stick package and unpacking phenomenon does not occur , such conditions are not within the scope of this discussion.
The second case, the receiving terminal receives only one packet, since TCP packet loss will not occur, so that a packet contains two packets of information sent by a transmitter, a phenomenon that is stick package . This case because the receiver does not know the boundaries of these two packets , it is difficult to handle for the receiving terminal.
The third case, this case has two forms, as shown below. The receiver receives two packets, but both packets is either incomplete, or is out of a plurality, i.e. this situation occurs and stick package unpacking . If both cases without special treatment, is also bad for the receiving side processing.
Cause stick package / unpacking occur
TCP stick package or unpacking occur for many reasons, now lists common points, may be incomplete, please add
- Data to be transmitted is greater than the size of the TCP send buffer space remaining, unpacking will occur.
- Data to be transmitted is larger than the MSS (maximum packet size), TCP will be unpacked before transmission.
- Data to be transmitted is smaller than the size of the TCP send buffer, the TCP data buffer multiple writes once sent out will occur stick package.
- Data receiving side application layer did not read the received data buffer, stick package will occur.
The transmitting end only needs to send out the buffer is full and the like, resulting in stick package (data transmission time interval is short, the data is very small, will be sent out as a packet, to produce stick package)
########################
#服务端
from socket import *
phone = socket(AF_INET,SOCK_STREAM)
phone.setsockopt(SOL_SOCKET,SOCK_STREAM,1)
phone.bind(('127.0.0.1',8080))
phone.listen(5)
print('start running...')
coon,addr = phone.accept() #等待连接
data1 = coon.recv(10)
data2 = coon.recv(10)
print('------------>',data1.decode('utf-8'))
print('------------>',data2.decode('utf-8'))
coon.close()
phone.close()
###############################
#客户端
from socket import *
import time
phone = socket(AF_INET,SOCK_STREAM)
phone.connect(('127.0.0.1',8080))
phone.send('hello'.encode('utf-8'))
phone.send('helloworld'.encode('utf-8'))
phone.close()The recipient is not timely received packet buffers, resulting in multiple packet receive (a piece of data sent by the client, the server received only a small portion of the server the next time or take the time to close the last remaining data from the buffer generating stick package)
########################
#服务端
from socket import *
phone = socket(AF_INET,SOCK_STREAM)
phone.setsockopt(SOL_SOCKET,SOCK_STREAM,1)
phone.bind(('127.0.0.1',8080))
phone.listen(5)
print('start running...')
coon,addr = phone.accept() #等待连接
data1 = coon.recv(2) #一次没有接收完整
data2 = coon.recv(10) #下一次接收的时候会先取旧的数据,然后取新的
# data3 = coon.recv(1024) #接收等5秒后的信息
print('------------>',data1.decode('utf-8'))
print('------------>',data2.decode('utf-8'))
# print('------------>',data3.decode('utf-8'))
coon.close()
phone.close()
###############################
#客户端
from socket import *
import time
phone = socket(AF_INET,SOCK_STREAM)
phone.connect(('127.0.0.1',8080))
phone.send('hello'.encode('utf-8'))
time.sleep(5)
phone.send('haiyan'.encode('utf-8'))
phone.close()
Stick package / unpacking solutions
Through the above analysis, we know the cause of stick package or unpacking happen, then how to solve this problem? The key problem is how to add boundary information for each data packet , commonly used method has the following:
- Transmitting end for each data packet added packet header, header should contain at least a length of the packet , so that the receiving terminal after receiving the data by reading the packet length field of the header, they know the actual length of each data packet a.
- Each sending end is a fixed length packet is encapsulated (not 0 may be prepared by filling up) , so that the receiving end of each fixed length data read from the receive buffer to each data packet naturally split open.
- You may be disposed in a boundary between the data packet, such as adding special symbols , so that the receiving end this boundary can be a different data packet split open.
Root of the problem is that the receiver does not know the length of the byte stream of the sender to be transferred, so the solution stick package is around, how to get the sender before sending the data, byte stream, the total size of their own that will be sent to enable the receiver its end, and then fetched reception cycle of death has received all data
########################
#服务端
import socket
import subprocess
import struct
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM) #买手机
phone.bind(('127.0.0.1',8080)) #绑定手机卡
phone.listen(5) #阻塞的最大数
print('start runing.....')
while True: #链接循环
coon,addr = phone.accept()# 等待接电话
print(coon,addr)
while True: #通信循环
# 收发消息
cmd = coon.recv(1024) #接收的最大数
print('接收的是：%s'%cmd.decode('utf-8'))
#处理过程
res = subprocess.Popen(cmd.decode('utf-8'),shell = True,
stdout=subprocess.PIPE, #标准输出
stderr=subprocess.PIPE #标准错误
)
stdout = res.stdout.read()
stderr = res.stderr.read()
#先发报头(转成固定长度的bytes类型，那么怎么转呢？就用到了struct模块)
#len(stdout) + len(stderr)#统计数据的长度
header = struct.pack('i',len(stdout)+len(stderr))#制作报头
coon.send(header)
#再发命令的结果
coon.send(stdout)
coon.send(stderr)
coon.close()
phone.close()
###############################
#客户端
import socket
import struct
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
phone.connect(('127.0.0.1',8080)) #连接服
while True:
# 发收消息
cmd = input('请你输入命令>>：').strip()
if not cmd:continue
phone.send(cmd.encode('utf-8')) #发送
#先收报头
header_struct = phone.recv(4) #收四个
unpack_res = struct.unpack('i',header_struct)
total_size = unpack_res[0] #总长度
#后收数据
recv_size = 0
total_data=b''
while recv_size<total_size: #循环的收
recv_data = phone.recv(1024) #1024只是一个最大的限制
recv_size+=len(recv_data) #
total_data+=recv_data #
print('返回的消息：%s'%total_data.decode('gbk'))
phone.close()Stick package to solve the problem upgrade version
########################
#服务端
import socket
import subprocess
import struct
import json
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM) #买手机
phone.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
phone.bind(('127.0.0.1',8080)) #绑定手机卡
phone.listen(5) #阻塞的最大数
print('start runing.....')
while True: #链接循环
coon,addr = phone.accept()# 等待接电话
print(coon,addr)
while True: #通信循环
# 收发消息
cmd = coon.recv(1024) #接收的最大数
print('接收的是：%s'%cmd.decode('utf-8'))
#处理过程
res = subprocess.Popen(cmd.decode('utf-8'),shell = True,
stdout=subprocess.PIPE, #标准输出
stderr=subprocess.PIPE #标准错误
)
stdout = res.stdout.read()
stderr = res.stderr.read()
# 制作报头
header_dic = {
'total_size': len(stdout)+len(stderr), # 总共的大小
'filename': None,
'md5': None
}
header_json = json.dumps(header_dic) #字符串类型
header_bytes = header_json.encode('utf-8') #转成bytes类型(但是长度是可变的)
#先发报头的长度
coon.send(struct.pack('i',len(header_bytes))) #发送固定长度的报头
#再发报头
coon.send(header_bytes)
#最后发命令的结果
coon.send(stdout)
coon.send(stderr)
coon.close()
phone.close()
###############################
#客户端
import socket
import struct
import json
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
phone.connect(('127.0.0.1',8080)) #连接服务器
while True:
# 发收消息
cmd = input('请你输入命令>>：').strip()
if not cmd:continue
phone.send(cmd.encode('utf-8')) #发送
#先收报头的长度
header_len = struct.unpack('i',phone.recv(4))[0] #吧bytes类型的反解
#在收报头
header_bytes = phone.recv(header_len) #收过来的也是bytes类型
header_json = header_bytes.decode('utf-8') #拿到json格式的字典
header_dic = json.loads(header_json) #反序列化拿到字典了
total_size = header_dic['total_size'] #就拿到数据的总长度了
#最后收数据
recv_size = 0
total_data=b''
while recv_size<total_size: #循环的收
recv_data = phone.recv(1024) #1024只是一个最大的限制
recv_size+=len(recv_data) #有可能接收的不是1024个字节，或许比1024多呢，
# 那么接收的时候就接收不全，所以还要加上接收的那个长度
total_data+=recv_data #最终的结果
print('返回的消息：%s'%total_data.decode('gbk'))
phone.close()struct module
#该模块可以把一个类型，如数字，转成固定长度的bytes类型
import struct
res = struct.pack('i',12345)
print(res,len(res),type(res)) #长度是4
res2 = struct.pack('i',12345111)
print(res,len(res),type(res2)) #长度也是4
unpack_res =struct.unpack('i',res2)
print(unpack_res) #(12345111,)
print(unpack_res[0]) #12345111
