Chapter VIII of the socket network programming (8): stick pack phenomenon and solutions (Code Completion)

Stick pack phenomenon

.recv (1024) pit: When data is transferred more than 1024bytes time, because recv only once to accept 1024byte, the pipeline will transport data → backlog will continue to receive the next recv → backlog of data processing which leads back to this send the the results are part of the result may return the contents of the last,

Phenomenon stick package: TCP protocol is a protocol stream data (transport stream is a whole data), i.e., how many bytes corresponding to a message is not visible. This results when the recipient accepts the data do not know how to punctuate, making the data chaos. This is called the stick pack phenomenon

Note: Only TCP packet has a sticky phenomenon, UDP never stick package

reference

Stick package Detailed Explanation send and recv

First, we emphasize:
the application software can not directly operate the hardware, the hardware must be called by the OS. Therefore, the application send and recv the os are for data transmission (the application layer).
Achieve the underlying network protocols other is completely taken over by the os.

• Stick package Summary:

  • Whether send or recv are exchanged for the local os memory for data rather than direct data exchange with the target server

    Note: you can only send correspondence is not a once recv, may correspond to any number of times any number of times

  • recv, send processing:
    • recv：
      • wait data: os wait for time-consuming data (need to have to wait)
      • copy data: the data to be copied to the memory os application
    • send：
      • send data: copying data from the application software to memory os
    • Stick package:
      • Upon receiving the data fetch, a data packet is too large, so that the data in the remaining part of the memory.
      • Upon receiving the data fetch, the data packets as a plurality of packets (due to the Nagle) received.

  • This is a reason to stick package: TCP streaming the Nagle algorithm to optimize the packet transmission merged

    Nagle algorithm: a small amount of data and the relatively short time interval of a packet transmission packet composition

• Solution:
  • Know exactly the number of bytes packet, then it is possible to avoid the stick package → tell in advance the receiving end can be a packet of information

Stick package to solve the problem

We use ssh simulation program Description:

What can I recv although the parameters set by big, but this is a fixed value, will not exceed the maximum memory space, it is not a good plan

• So how do we analyze the case:
  • These messages are sent each other, so when sending data
    • The transmission data length to each other
    • Then send data to each other

server.py

import socket
import subprocess
import struct
import locale

ssh_server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssh_server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
ssh_server.bind(('127.0.0.1', 8080))
ssh_server.listen(3)

while True:
    conn, addr = ssh_server.accept()
    while True:
        try:
            cmd = conn.recv(1024).decode('utf-8')
            if not cmd: break
            print("excute cmd:", cmd)
            obj = subprocess.Popen(cmd, shell=True,
                                   stdout=subprocess.PIPE,
                                   stderr=subprocess.PIPE)
            stdout = obj.stdout.read()
            stderr = obj.stderr.read()

            os_locale = locale.getdefaultlocale()
            os_encode = os_locale[1]
            print(stdout.decode(os_encode))

            # 第一步：生成数据的报头(报头：固定长度)：需要用到struct模块
            # 我们首先需要把发送数据的描述信息发送给客户端，但是客户端并不知道这个包是干什么的，而且会和数据的包粘包
            # 所以我们把这段数据作为数据的报头发送
            total_size = len(stdout) + len(stderr)
            header = struct.pack('i', total_size)  # 打包报头，打包返回的数据是一个固定长度(4bytes)的bytes类型
            # struct.pack(模式，报头数据)，注意：'i'模式int范围：正负2*10^9之内(因为只有4bytes)
            # 另外有'l'模式：长整型。但是这个模式报头为8bytes。也有范围限制

            # 第二部：发送报头
            conn.send(header)  # 因为是bytes类型所以可以直接发送

            # 把命令结果返回给客户端
            # conn.send(stdout+stderr)  # 这里之前用+连接一起发送，但其实粘包也可以一起发送
            conn.send(stdout)  # 连续发送可以粘包
            conn.send(stderr)  # 与上一行粘包成一个数据包了
        except ConnectionResetError as e:
            print(e)
            break
    conn.close()

ssh_server.close()

client.py

import socket
import struct

ssh_client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

ssh_client.connect(('127.0.0.1', 8080))

while True:
    # 发命令
    cmd = input(">>>>:").strip()
    if not cmd:continue
    ssh_client.send(cmd.encode('utf-8'))

    # 第一步：先收报头
    header = ssh_client.recv(4)  # 因为报头固定4byte 所以我们收4

    # 第二部：从报头解析出数据的描述(数据的长度)
    total_size = struct.unpack('i', header)[0]  # 解包报头，返回一个元组类型，这里我们第一个元素是长度
    print(total_size)  # 打印一下报头内容

    # 第三部：接受真实收据
    recv_size = 0
    recv_data = b''
    while recv_size < total_size:
        res = ssh_client.recv(1024)
        recv_data += res
        recv_size += len(res)

    print(recv_data.decode('cp932'))  # 因为粘包问题解决了，这里我们直接打印

ssh_client.close()