Small knowledge
s1 = 'q'
s2 = b'q'
print(type(s1),type(s2))
print(s1.encode('utf-8'))
#<class 'str'> <class 'bytes'>
# b'q'bytes Type:
ASCII characters in front of the character b ""
Non-ASCII type, such as Chinese, first into a string, and then type into bytes
s1 = "太白jx"
print(len(s1)) # 4
b1 = s1.encode('utf=8') # bytes 类型
print(b1) # b'\xe5\xa4\xaa\xe7\x99\xbdjx'
print(len(b1)) # 8A: socket
Before looking socket, first look at five communication process:
But in fact, from the beginning of the transport layer and below are the operating systems to help our complete, the following various processes Baotou package, let's go one by one to do with it? NO!
Socket also known as a socket, which is the intermediate layer and the application software TCP / IP protocol suite to communicate abstraction layer, which is a set of interfaces. In design mode, Socket is actually a facade pattern, it is the complexity of TCP / IP protocol suite is hidden behind the Socket interface for users, a simple interface is all set, let Socket to organize data in order to comply with the specified protocol. When we communicate using different protocols have different interfaces, have to deal with the details of different protocols, which increases the difficulty of development, the software is not easy to expand (as we develop a management system as , reimbursement, schedule a meeting, leave and so do not need to write a separate system, but on a system more functional interface, do not need to know how each function to achieve).
Thus UNIX BSD socket invented such things, details of each socket shielding communication protocol, so that a programmer need not concern itself protocol, the interface directly to provide a socket for communication between the processes of different hosts interconnected. It's like operating system provides us with the use of the underlying hardware functions of the system calls, system calls, we can easily use the disk (file operations), the use of memory, without having to own disk read and write, memory management. socket is actually the same thing, is to provide an abstract tcp / ip protocols, provides a set of external interfaces, this interface can be had with a unified, easy to use function tcp / ip protocol of the.
In fact, standing on your point of view, socket is a module. We establish a connection and communication between two processes by calling a method module has been implemented. It was also said to the socket ip + port, because ip is used to identify the location of a host in the Internet, and the port is used to identify an application on this machine. So long as we establish the ip and port will be able to find an application, and use the socket module to communicate with it.
The first edition, a single client and server communication (low version) simply based communication socket TCP protocol of
server: server
# 网络通信与打电话（诺基亚）是一样的。
import socket
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM) # 买电话
phone.bind(('127.0.0.1',8080)) # 0 ~ 65535 1024之前系统分配好的端口 绑定电话卡
phone.listen(5) # 同一时刻有5个请求，但是可以有N多个链接。 开机。
conn, client_addr = phone.accept() # 接电话
print(conn, client_addr, sep='\n')
from_client_data = conn.recv(1024) # 一次接收的最大限制 bytes
print(from_client_data.decode('utf-8'))
conn.send(from_client_data.upper())
conn.close() # 挂电话
phone.close() # 关机client: The client
import socket
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM) # 买电话
phone.connect(('127.0.0.1',8080)) # 与客户端建立连接， 拨号
phone.send('hello'.encode('utf-8'))
from_server_data = phone.recv(1024)
print(from_server_data)
phone.close() # 挂电话Second Edition, based on the communication cycles Cycles socket TCP communication protocol
server
import socket
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
phone.bind(('127.0.0.1',8080))
phone.listen(5)
conn, client_addr = phone.accept()
print(conn, client_addr, sep='\n')
while 1: # 循环收发消息
try:
from_client_data = conn.recv(1024)
print(from_client_data.decode('utf-8'))
conn.send(from_client_data + b'SB')
except ConnectionResetError:
break
conn.close()
phone.close()client
import socket
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM) # 买电话
phone.connect(('127.0.0.1',8080)) # 与客户端建立连接， 拨号
while 1: # 循环收发消息
client_data = input('>>>')
phone.send(client_data.encode('utf-8'))
from_server_data = phone.recv(1024)
print(from_server_data.decode('utf-8'))
phone.close() # 挂电话Third Edition, we communicate, connect loop connector loop plus socket TCP-based communication protocol
server
import socket
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
phone.bind(('127.0.0.1',8080))
phone.listen(5)
while 1 : # 循环连接客户端
conn, client_addr = phone.accept()
print(client_addr)
while 1:
try:
from_client_data = conn.recv(1024)
print(from_client_data.decode('utf-8'))
conn.send(from_client_data + b'SB')
except ConnectionResetError:
break
conn.close()
phone.close()client
import socket
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM) # 买电话
phone.connect(('127.0.0.1',8080)) # 与客户端建立连接， 拨号
while 1:
client_data = input('>>>')
phone.send(client_data.encode('utf-8'))
from_server_data = phone.recv(1024)
print(from_server_data.decode('utf-8'))
phone.close() # 挂电话Examples of remote command execution: Example socket TCP-based application protocol, remote operation
import socket
import subprocess
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
phone.bind(('127.0.0.1',8080))
phone.listen(5)
while 1 : # 循环连接客户端
conn, client_addr = phone.accept()
print(client_addr)
while 1:
try:
cmd = conn.recv(1024)
ret = subprocess.Popen(cmd.decode('utf-8'),shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
correct_msg = ret.stdout.read()
error_msg = ret.stderr.read()
conn.send(correct_msg + error_msg)
except ConnectionResetError:
break
conn.close()
phone.close()import socket
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM) # 买电话
phone.connect(('127.0.0.1',8080)) # 与客户端建立连接， 拨号
while 1:
cmd = input('>>>')
phone.send(cmd.encode('utf-8'))
from_server_data = phone.recv(1024)
print(from_server_data.decode('gbk'))
phone.close() # 挂电话Stick package
What is the stick package ??
Refers to the TCP protocol, a number of data sent by the sender to the recipient receives stick into a packet, the receive buffer from the point of view, after a first packet of data, immediately before the end of a data packet.
What happens stick package ???
- The first: continuous short send times (the amount of data is small), recv unified cache to cache, your data will be sent unity
- The second: send the data is too large, the other is greater than the upper limit of recv, while the other second recv, receives no recv finished the last of the remaining data
Take a look at the issue socket buffer before speaking stick package:
Why set the buffer ???
- Temporarily cache some data
- Cache area if there are fluctuations in your network, send and receive data to ensure the stability of uniform
Disadvantages:
It caused a stick pack phenomenon
Why is there an error ???
Sometimes the Chinese split in half, and is received bytes, an error
Stick package will occur in both cases.
1, the receiver does not timely packet receive buffer, resulting in multiple packet receive (a piece of data sent by the client, the server received only a small portion of the server the next time or the time received from the buffer to get the last legacy data generating stick package)
Classroom teachers say means: continuous short send multiple (small amount of data, your data will be unified sent)
import socket
import subprocess
phone = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
phone.bind(('127.0.0.1', 8080))
phone.listen(5)
while 1: # 循环连接客户端
conn, client_addr = phone.accept()
print(client_addr)
while 1:
try:
cmd = conn.recv(1024)
ret = subprocess.Popen(cmd.decode('utf-8'), shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
correct_msg = ret.stdout.read()
error_msg = ret.stderr.read()
conn.send(correct_msg + error_msg)
except ConnectionResetError:
break
conn.close()
phone.close()import socket
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM) # 买电话
phone.connect(('127.0.0.1',8080)) # 与客户端建立连接， 拨号
while 1:
cmd = input('>>>')
phone.send(cmd.encode('utf-8'))
from_server_data = phone.recv(1024)
print(from_server_data.decode('gbk'))
phone.close()
# 由于客户端发的命令获取的结果大小已经超过1024，那么下次在输入命令，会继续取上次残留到缓存区的数据。2, the transmitting side need to wait until the buffer is full sent out, resulting in stick package (data transmission time interval is short, the data is small, joined together to produce stick package) Send the data is too large, other than the upper limit of the recv, the second time recv, receives the other one did not finish the rest of the recv
import socket
phone = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
phone.bind(('127.0.0.1', 8080))
phone.listen(5)
conn, client_addr = phone.accept()
frist_data = conn.recv(1024)
print('1:',frist_data.decode('utf-8')) # 1: helloworld
second_data = conn.recv(1024)
print('2:',second_data.decode('utf-8'))
conn.close()
phone.close()import socket
phone = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
phone.connect(('127.0.0.1', 8080))
phone.send(b'hello')
phone.send(b'world')
phone.close()
# 两次返送信息时间间隔太短，数据小，造成服务端一次收取Stick package solution:
First introduce struct module:
The module can be a type, such as numbers, conversion to a fixed length of bytes
import struct
# 将一个数字转化成等长度的bytes类型。
ret = struct.pack('i', 183346)
print(ret, type(ret), len(ret))
# 通过unpack反解回来
ret1 = struct.unpack('i',ret)[0]
print(ret1, type(ret1), len(ret1))
# 但是通过struct 处理不能处理太大
ret = struct.pack('l', 4323241232132324)
print(ret, type(ret), len(ret)) # 报错How to solve the stick package?
A server sends data, 10000 bytes, the client receives the data, each cycle of receiving up to 1024 bytes received, until all of the bytes on the receiving end, the received data is stitched together, the final decoding.
Encounter problems
Recv received number can not be determined, the total specific data before you send, give me send a total data length of 5000 bytes, and then send the total data
Client: receiving a first length of 5000 bytes, I then recycled,
Then I recycle conditions recv control cycle is as long as you accept the Data <5000 has been receiving.
Problems encountered: the number is not converted to the fixed length of the total data byte
服务端：
conn.send(total_size)
conn.send(result)
total_size int类型
客户端：
total_size_bytes = phone.recv(4)
total_size
data = b''
while len(data) < total_size:
data = data + phone.recv(1024)You want to total_size int type is converted into bytes types can send
387 ----> str (387) '387' ----> bytes b'387 'length 3bytes
4185 ----> str (4185) '4185' ----> bytes b'4185 'length 4bytes
18,000 ------------------------------------------------- -----> length 5bytes
We have to solve:
The fixed length not converted to type int bytes of fixed length and may also be flipped back.
stuct module number is not fixed length, into fixed length of bytes, and then flipped back
end server
# 1. 粘包第一种： send的数据过大，大于对方recv的上限时，对方第二次recv时，会接收上一次没有recv完的剩余的数据。
import socket
import subprocess
import struct
phone = socket.socket()
phone.bind(('127.0.0.1',8848))
phone.listen(2)
# listen: 2 允许有两个客户端加到半链接池，超过两个则会报错
while 1:
conn,addr = phone.accept() # 等待客户端链接我,阻塞状态中
# print(f'链接来了: {conn,addr}')
while 1:
try:
from_client_data = conn.recv(1024) # 接收命令
if from_client_data.upper() == b'Q':
print('客户端正常退出聊天了')
break
obj = subprocess.Popen(from_client_data.decode('utf-8'),
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
result = obj.stdout.read() + obj.stderr.read()
total_size = len(result)
print(f'总字节数：{total_size}')
# 1. 制作固定长度的报头
head_bytes = struct.pack('i',total_size)
# 2. 发送固定长度的报头
conn.send(head_bytes)
# 3. 发送总数据
conn.send(result)
except ConnectionResetError:
print('客户端链接中断了')
break
conn.close()
phone.close()
# import struct
# # 将一个数字转化成等长度的bytes类型。
# ret = struct.pack('i', 180000000)
# # print(ret, type(ret), len(ret))
#
# # 通过unpack反解回来
# ret1 = struct.unpack('i',ret)[0]
# # print(ret1)
# print(ret1, type(ret1))
# 总数据：总数据长度
# s1 = 'fjdslf太白金星jsfk疯狂夺金分离式的疯狂的数量方式登记拉开lagfdkjglkhjklh'
# b1 = s1.encode('utf-8')
# print(b1)
# print(len(b1))client
import socket
import struct
phone = socket.socket()
phone.connect(('127.0.0.1',8848))
while 1:
to_server_data = input('>>>输入q或者Q退出').strip().encode('utf-8')
if not to_server_data:
# 服务端如果接受到了空的内容，服务端就会一直阻塞中，所以无论哪一端发送内容时，都不能为空发送
print('发送内容不能为空')
continue
phone.send(to_server_data)
if to_server_data.upper() == b'Q':
break
# 1. 接收报头
head_bytes = phone.recv(4)
# 2. 反解报头
total_size = struct.unpack('i',head_bytes)[0]
total_data = b''
while len(total_data) < total_size:
total_data += phone.recv(1024)
print(len(total_data))
print(total_data.decode('gbk'))
phone.close()