First, stick package
1 stick pack phenomenon
Based socket tcp protocol, client acceptance finish time, continued to receive the next (if the interval is relatively long, and the subsequent data will stick together before the remaining data), when the send data, continuous transmission of small amounts of data ( very short time interval), these data will be sent together backlog.
2. stick pack phenomenon
- The recipient does not receive timely packet buffer, resulting in multiple packet receive (a piece of data sent by the client, the server received only a small portion of the server the next time or take the time to close the last remaining data from the buffer generating stick package)
- The transmitting end only needs to send out the buffer is full and the like, resulting in stick package (data transmission time interval is short, the data is small, joined together to produce stick package)
3. The resulting stick package Cause
- TCP protocol is based on the data stream, no matter what the underlying segment slice, according to the TCP protocol does not limit the transmission of the data message, but the sort of data segments constituting the entire message is presented to the kernel buffer is completed, until the transmission buffer parties have enough data before sending a TCP segment
- When the data transmission is greater than the data received by the socket, the extra data will be placed in the buffer, the data placed in the buffer before receiving side next receives data recv once, it will continue to receive, instead of the latest sent from the transmitting side The data
4. Buffer
After each socket is created, two buffers are assigned, the input buffer and output buffer.
write () / send () in the network does not transmit data immediately, but writes data to the buffer, and then transmits the data from the buffer by the TCP protocol to the target machine. Once the data is written to the buffer, the function can return success, regardless of whether they have not reached the target machine, no matter when they are sent to the network, these are the TCP protocol is responsible for what.
TCP protocol independent write () / send () function, data may be sent to the buffer just written to the network, it may continue to backlog in the buffer, data is written many times sent to the network only once, which depending on prevailing network conditions, the current thread is idle and many other factors, not by the programmer to control.
read () / recv () function is true, read the data from the input buffer, instead of reading directly from the network.
These I / O buffers characteristics may be summarized as follows:
1.I / O buffer present in each individual TCP socket;
2.I / O buffer is automatically generated when creating the socket;
3. Close the socket even when left in the buffer will continue to transfer output data;
4. Close the socket will be lost in the input data buffer.
Second, stick package solution
1. low version
Server
import socket
import subprocess
import struct
server_side = socket.socket()
server_side.bind(("127.0.0.1", 8848))
server_side.listen(5)
while 1:
conn, addr = server_side.accept()
while 1:
try:
cmd = conn.recv(1024)
if cmd.decode("utf-8") == 'q':
break
obj = subprocess.Popen(cmd.decode("utf-8"),
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
result = obj.stdout.read() + obj.stderr.read()
# 制作报头
total_size = len(result)
# 将不固定长度的int数据类型的报头,转化成固定长度的4bytes
total_size_bytes = struct.pack("i", total_size)
# 发送报头
conn.send(total_size_bytes)
# 发送数据
conn.send(result)
except ConnectionResetError:
break
conn.close()
server_side.close()Client
import socket
import struct
client_side = socket.socket()
client_side.connect(("127.0.0.1", 8848))
while 1:
to_send = input("请输入你要执行的命令:")
if to_send.upper() == "Q":
client_side.send("q".encode("utf-8"))
break
client_side.send(to_send.encode("utf-8"))
# 接收报头
head_bytes = client_side.recv(4)
# 反解报头
total_size = struct.unpack("i", head_bytes)[0]
# 循环接收数据
data = b""
while len(data) < total_size:
data += client_side.recv(1024)
print(data.decode("gbk"))
client_side.close()
2. Ultimate
Server
import socket
import subprocess
import json
import struct
phone = socket.socket()
phone.bind(("127.0.0.1", 8848))
phone.listen(5)
conn, addr = phone.accept()
while 1:
try:
cmd = conn.recv(1024)
obj = subprocess.Popen(cmd.decode("utf-8"),
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
result = obj.stdout.read() + obj.stderr.read()
result = result.decode("gbk").encode("utf-8")
# 1.制作报头
head_dict = {"md5": "df",
"file_name": "新建文件夹",
"file_size": len(result)}
# 2. 将报头字典转化成json字符串
head_dict_json = json.dumps(head_dict)
# 3. 将json字符串转化成bytes
head_dict_json_bytes = head_dict_json.encode("utf-8")
# 4. 获取报头的长度
head_len = len(head_dict_json_bytes)
# 5. 将报头长度转化成固定的4个bytes
head_len_bytes = struct.pack("i", head_len)
# 6. 发送固定的4个字节
conn.send(head_len_bytes)
# 7. 发送报头
conn.send(head_dict_json_bytes)
# 8. 发送原数据
conn.send(result)
except ConnectionResetError:
break
conn.close()
phone.close()Client
import socket
import struct
import json
phone = socket.socket()
phone.connect(("127.0.0.1", 8848))
while 1:
cmd = input("请输入指令")
phone.send(cmd.encode("utf-8"))
# 1. 接收报头长度
head_len_bytes = phone.recv(4)
# 2. 将报头数字转化成int类型
head_len = struct.unpack("i", head_len_bytes)[0]
# 3. 接收bytes类型的报头字典
head_dict_json_bytes = phone.recv(head_len)
# 4. 将bytes类型的字典转化成json字符串
head_dict_json = head_dict_json_bytes.decode("utf-8")
# 5. 将json字符串转化成字典
head_dict = json.loads(head_dict_json)
# 6. 循环接收原数据
total_data = b''
while len(total_data) < head_dict["file_size"]:
total_data += phone.recv(1024)
print(total_data.decode("utf-8"))
phone.close