table of Contents
Stick package problem
Note: Only TCP packet has a sticky phenomenon, UDP never stick package called stick pack problem mainly because the recipient does not know the boundaries between news, do not know how many bytes of data caused by one-time extraction.
- TCP (transport control protocol, the Transmission Control Protocol) is connection-oriented and stream-oriented, high reliability service. The receiver and transmitter (client and server) must have eleven pairs of socket, therefore, in order to transmit end a plurality of packets addressed to the receiving end, other more efficient to send, using the optimization method (the Nagle Algorithm), the times, at intervals smaller and smaller amount of data the data combined into a large block of data, then the packet. In this way, the receiving end, it is hard to tell the difference, and must provide scientific unpacking mechanism. I.e., non-oriented communication message stream is protected boundaries.
- UDP (user datagram protocol, User Datagram Protocol) is a connectionless, message-oriented, providing efficient service. The combined use of the optimization algorithm does not support block ,, since UDP is a many mode, the receiving end of the skbuff (socket buffer) using the chain structure to record the arrival of each UDP packet in each UDP We have a package (information source address and port) message header, so that, for the receiving side, it is easy to distinguish the process. That message is a message-oriented communication protected boundaries.
- TCP is a stream-based, so messages sent and received can not be empty, which requires all add empty message handling mechanism in the client and server to prevent jamming program, which is based on udp datagram, even if you enter empty content (direct carriage return), it was not an empty message, udp protocols will help you package the message headers, a little experiment
Four cases TCP send data
Suppose the client send two packets D1 and D2 to the server, the server due to a number of bytes read is uncertain, so that there may be the following four cases.
- Twice server to read two separate data packets, respectively D1 and D2, and not stick package unpacking;
- The server receives a packet two, D1 and D2 are bonded together, it referred to as TCP stick package;
- Twice server to read the two data packets, the first time a complete read packet D1 and D2 parts of the package, the second reading to the remaining package contents D2, which is referred to as TCP unpacking;
- Twice server to read two data packets, the first read part to the package D1_1 D1, the second reading to the remaining package content D1_2 D1 and D2 of the whole package bag.
Exception: If at this time the server receives the TCP sliding window is very small, and the data packet D1 and D2 is relatively large, is very likely to occur fifth, that service will end points several times to complete packet reception D1 and D2, occurred during repeatedly unpacking.
In both cases the stick package
1. The transmitting side need to wait until the buffer is full sent out, resulting in stick package (data transmission time interval is short, the data is small, to join together to produce stick package)
The recipient is not timely received packet buffers, resulting in multiple packet receive (a piece of data sent by the client, the server received only a small portion of the server the next time or take the time to close the last remaining data from the buffer generating stick package)
Why TCP is a reliable transport, udp is unreliable transmission
- Another article based on TCP data transmission, refer to nickl teacher https://www.cnblogs.com/nickchen121/p/11027575.html , in TCP data transmission, the sender first sends data to its own cache, then the data cache control protocol sent to the peer, the peer returns an ack = 1, a transmitting side data buffer is clean, the peer returns ack = 0, then resend the data, TCP is a reliable
- udp transmission data, the peer returns an acknowledgment message is not therefore unreliable
send (byte stream) and the recv (1024) and sendall
- recv means 1024 specified in a 1024-byte data out from the cache
- The byte stream is to send into the hexyl side cache, then the cache is controlled by a protocol to send the content to the end, if the size of the byte stream to be transmitted is greater than the remaining buffer space, then the data is lost, it will cycle with sendall send calls, data is not lost
Stick package to solve the problem
Root of the problem is that the receiver does not know the length of the byte stream of the sender to be transferred, so the solution stick package is around, how to get the sender before sending the data, byte stream, the total size of their own that will be sent to enable the receiver its end, and then fetched reception cycle of death has received all data.
#服务端
import socket, subprocess
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('127.0.0.1', 8000))
server.listen(5)
while True:
conn, addr = server.accept()
print('start...')
while True:
cmd = conn.recv(1024)
print('cmd:', cmd)
obj = subprocess.Popen(cmd.decode('utf8'),
shell=True,
stderr=subprocess.PIPE,
stdout=subprocess.PIPE)
stdout = obj.stdout.read()
if stdout:
ret = stdout
else:
stderr = obj.stderr.read()
ret = stderr
ret_len = len(ret)
conn.send(str(ret_len).encode('utf8'))
data = conn.recv(1024).decode('utf8')
if data == 'recv_ready':
conn.sendall(ret)
conn.close()
server.close()
#客户端
import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(('127.0.0.1', 8000))
while True:
msg = input('please enter your cmd you want>>>').strip()
if len(msg) == 0: continue
client.send(msg.encode('utf8'))
length = int(client.recv(1024))
client.send('recv_ready'.encode('utf8'))
send_size = 0
recv_size = 0
data = b''
while recv_size < length:
data = client.recv(1024)
recv_size += len(data)
print(data.decode('utf8'))
#此版比较low,程序的运行速度远快于网络传输速度，所以在发送一段字节前，先用send去发送该字节流长度，这种方式会放大网络延迟带来的性能损耗
解决粘包问题的核心就是：为字节流加上自定义固定长度报头，报头中包含字节流长度，然后一次send到对端，对端在接收时，先从缓存中取出定长的报头，然后再取真实数据
具体版本详情请看 nick老师的博客 https://www.cnblogs.com/nickchen121/p/11032005.htmlBased on UDP socket socket programming
UPD protocol is typically not used for transmission of large data.
Although there is no stick package UDP socket problem, but not a substitute for a TCP socket, because UPD protocol has a flaw: If the data is sent on his way, the data is lost, the data is lost, and there will be no such defects TCP protocol , it is generally irrelevant UPD socket user data transmission, chat qq e.g.
#服务器 #_*_coding:utf-8_*_ __author__ = 'nick' import socket ip_port = ('127.0.0.1', 8081) UDP_server_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) #买手机 UDP_server_sock.bind(ip_port) while True: qq_msg, addr = UDP_server_sock.recvfrom(1024) print('来自[%s:%s]的一条消息:\033[1;44m%s\033[0m' % (addr[0], addr[1], qq_msg.decode('utf-8'))) back_msg = input('回复消息: ').strip() UDP_server_sock.sendto(back_msg.encode('utf-8'), addr) # 客户端1 #_*_coding:utf-8_*_ __author__ = 'nick' import socket BUFSIZE = 1024 UDP_client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) qq_name_dic = { '狗哥alex': ('127.0.0.1', 8081), '瞎驴': ('127.0.0.1', 8081), '一棵树': ('127.0.0.1', 8081), '武大郎': ('127.0.0.1', 8081), } while True: qq_name = input('请选择聊天对象: ').strip() while True: msg = input('请输入消息,回车发送: ').strip() if msg == 'quit': break if not msg or not qq_name or qq_name not in qq_name_dic: continue UDP_client_socket.sendto(msg.encode('utf-8'), qq_name_dic[qq_name]) back_msg, addr = UDP_client_socket.recvfrom(BUFSIZE) print('来自[%s:%s]的一条消息:\033[1;44m%s\033[0m' % (addr[0], addr[1], back_msg.decode('utf-8'))) UDP_client_socket.close() #客户端2 #_*_coding:utf-8_*_ __author__ = 'nick' import socket BUFSIZE = 1024 UDP_client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) qq_name_dic = { '狗哥alex': ('127.0.0.1', 8081), '瞎驴': ('127.0.0.1', 8081), '一棵树': ('127.0.0.1', 8081), '武大郎': ('127.0.0.1', 8081), } while True: qq_name = input('请选择聊天对象: ').strip() while True: msg = input('请输入消息,回车发送: ').strip() if msg == 'quit': break if not msg or not qq_name or qq_name not in qq_name_dic: continue UDP_client_socket.sendto(msg.encode('utf-8'), qq_name_dic[qq_name]) back_msg, addr = UDP_client_socket.recvfrom(BUFSIZE) print('来自[%s:%s]的一条消息:\033[1;44m%s\033[0m' % (addr[0], addr[1], back_msg.decode('utf-8'))) UDP_client_socket.close()
Based on socket socket programming socketserver achieve concurrency
- For details, see https://www.cnblogs.com/nickchen121/p/11032290.html