Thirteen, TCP network programming

Socket socket

Interaction between the network socket connection is established through a process that both sides must have IP + port.

python provides standard socket library, it is very bottom of the library, Socket role in the application layer and the intermediate TCP / IP communications protocol suite, which is a set of interfaces. In design mode, Socket is actually a facade pattern, it is the complexity of TCP / IP protocol suite is hidden behind the Socket interface for users, a simple interface is all set; the user interface to make calls, to organize a Socket data in order to comply with the specified protocol.

In fact, standing on the perspective of a programmer, socket is a module. We establish a connection and communication between two processes by calling a method module has been implemented.

Connection establishment

TCP programming

Server-side programming step

1. Create a Socket object
2. Bind IP address and port for the Socket object. Use bind () method, passing as a parameter tuple, which is a format ( 'IP address of the string', Port)
3. Start listening, listen () method.
4. Socket object is used to obtain data exchange, a process with multiple socket objects
   using the accept (), accept method blocks waiting for the client to establish a connection, returning a tuple new Socket object and client addresses. The new Socket object includes a recv (bufsize [, flags]) using the buffer to receive data; send (bytes) of data transmission

代码演示上诉步骤

import socket

sk=socket.socket()   ##通过socket模块中的socket的class 实例化一个sokcet，命名为sk
sk.bind(("127.0.0.1",8080))    ##sk绑定在在ip：127.0.0.1 端口8080。注意bind()方法接收的是ip和端口组成的元组
sk.listen()          ##套接字sk开始在指定端口上监听

conn,addr=sk.accept()       ##获得套接字的一个连接对象，和地址对象，此次发送等待阻塞，地址对象也是个元组('127.0.0.1', 55616)
ret=conn.recv(1024)         ##通过连接对象.recv方法 接收内容，1024表示一次接收最多1024字节
print(ret)
conn.send(b"12345")         ##通过连接对象.send方法 传送内容， 传送的内容必须是bytes类型

conn.close()                ##关闭连接对象
sk.close()                  ##销毁套接字sk

Case presentation

Multiplayer chat -server

import logging
import socket
import time
import threading

FORMAT = '%(asctime)s %(threadName)s %(message)s'
logging.basicConfig(format=FORMAT, level=logging.INFO)

class ChatSever():
    def __init__(self,ip,port):
        self.sk=socket.socket()
        self.sk.bind((ip,port))
        self.clients=[]

    def start(self):
        self.sk.listen()
        #conn,addr=self.sk.accept() #会发生阻塞等待，阻碍主线程，故线程化需要accept的函数
        threading.Thread(target=self.accept,name='accept').start()

    def stop(self):
        for client in self.clients:
            client.close()
        self.sk.close()

    def accept(self):
        while True:
            conn,addr=self.sk.accept()
            self.clients.append(conn)
            #conn.recv() #会发生阻塞等待，accept的线程卡住，故线程化需要recv的函数
            threading.Thread(target=self.recv,args=(conn,),name='recv').start()

    def recv(self,conn):
        while True:
            data=conn.recv(1024).decode()
            logging.info(data)
            if not data:
                conn.close()
                self.clients.remove(conn)
                break
            for client in self.clients:
                client.send(data.encode())

def showthreadinfo():
    while True:
        time.sleep(1)
        logging.info(threading.enumerate())

cs=ChatSever('127.0.0.1',9999)
cs.start()

while True:
    threading.Thread(target=showthreadinfo,name='jiankong',daemon=True).start()
    cmd=input()
    if cmd=='quit':
        time.sleep(3)
        cs.stop()
        break

Multiplayer chat -client

import logging
import socket
import threading

FORMAT = '%(asctime)s %(threadName)s  %(message)s'
logging.basicConfig(format=FORMAT, level=logging.INFO)

ck=socket.socket()
ck.connect(('127.0.0.1',9999))


def send(ck):
    while True:
        data=input('want to say:')
        ck.send(data.encode())
        logging.info('send {}'.format(data))

def recv(ck):
    while True:
        ret=ck.recv(1024).decode()
        logging.info('recv {}'.format(ret))

threading.Thread(target=send,args=(ck,)).start()
threading.Thread(target=recv,args=(ck,)).start()

Package sticky problem

In the network transmission, because the receiver is used to receive data through the buffer, it is easy to package sticky phenomenon, such as other continuously transmitting a plurality of times, and the recipient receives a resulting mixed data received.

Solution viscosity package: custom messages, to inform the client about the transmission properties of the content, such as size, MD5 codes.

import socket
import struct
import json
import subprocess
sk=socket.socket()
sk.bind(('127.0.0.1',8080))
sk.listen()

conn,addr=sk.accept()

while True:
    cmd=conn.recv(1024).decode()
    print(cmd)
    if cmd =='q':break
    ret_cmd=subprocess.Popen(cmd,shell=True,
                             stdout=subprocess.PIPE,stderr=subprocess.PIPE)

    err=ret_cmd.stderr.read()       ##注意此处必须先将ret_cmd的stderr属性read出，然后赋值给某个变量。
                                    ##因为stderr属性类似一个队列，read取走一次后将变空
    if err:
        send_msg=err
    else:
        send_msg=ret_cmd.stdout.read()
    print(send_msg)
    ##判断命令执行结果，然后决定发送给客户端的内容
    data_size=len(send_msg)
    header={"data_size":data_size}      ##定义header报文为一个字典，里面的k-v对用于表示传输内容的大小，也可以扩展说明其他属性
    header_json=json.dumps(header)      ##将header报文字符串化，才能在下面使用struct进行转换。
    header_len_struct=struct.pack('i',len(header_json))     ##将表示header报文的大小的数值，转换成用4个字节大小的bytes数据来表示
    conn.send(header_len_struct)        ##先发送表示header报文大小的bytes类型数据
    conn.send(header_json.encode())     ##再发送header报文
    conn.send(send_msg)                 ##最后发送执行结果

conn.close()
sk.close()

客户端
import socket
import struct
import json

ck=socket.socket()
ck.connect(('127.0.0.1',8080))

while True:
    cmd=input('输入命令：')
    ck.send(cmd.encode())
    if cmd=='q':break
    header_len_struct=ck.recv(4)        ##接收表示报文大小的bytes类型数据
    header_len=struct.unpack('i',header_len_struct)[0]  ##将bytes类型数据反转换，从中取出报文大小。
    header=json.loads(ck.recv(header_len).decode())   ##按header报文的大小接受数据，取得header完整的报文
                                                      ##将报文从字符串反序列化为字典
    print(header)
    cmd_ret=ck.recv(header['data_size']).decode()     ##根据字典中的执行结果大小，获取完整的执行结果
    print(cmd_ret)

ck.close()

socketserver

underlying socket programming too difficult to build robust code, python has been expanded and improved based on the socket module, launched socketserver module is a network service programming framework for building enterprise-level applications, has entered the python standard library.

socketserver module offers five service classes

• BaseServer：服务器的基类，定义了API。
• TCPServer：使用TCP/IP套接字。
• UDPServer：使用数据报套接字
• UnixStreamServer：使用UNIX套接字，只适用UNIX平台
• UnixDatagramServer：使用UNIX套接字，只适用UNIX平台

这五种服务基类都是同步类型，他们的直接存储继承关系

在socketserver模块中还提供类Mixin类，与服务类组合，实现server异步。

#创建多进程UDP服务，fork需要操作系统支持，Windows不支持。
class ForkingUDPServer(ForkingMixIn, UDPServer): pass  
#创建多进程TCP服务，fork需要操作系统支持，Windows不支持。
class ForkingTCPServer(ForkingMixIn, TCPServer): pass
#创建多线程UDP服务
class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
#创建多线程TCP服务
class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass

通常编程server端服务时调用上面四种异步类ThreadingUDPServer，ThreadingTCPServer，ForkingUDPServer，ForkingTCPServer

浅分析sockerserver模块中服务类的工作过程，以ThreadingTCPServer类为例

1、通过ThreadingTCPServer的继承，查看BaseServer的__init__方法知道创建服务实例时需要传入server_address, RequestHandlerClass

##sockerserver.py文件中
def __init__(self, server_address, RequestHandlerClass):
    """Constructor.  May be extended, do not override."""
    self.server_address = server_address #服务实例监听的socket，以二元组('ip',port)
    self.RequestHandlerClass = RequestHandlerClass  # 将RequestHandlerClass当作类属性付给了实例本身
    self.__is_shut_down = threading.Event()
    self.__shutdown_request = False

2、服务实例最终调用RequestHandlerClass类来处理一个客户端的连接

def finish_request(self, request, client_address): # 处理请求的方法
"""Finish one request by instantiating RequestHandlerClass.""" 
self.RequestHandlerClass(request, client_address, self) 

3、如何创建RequestHandlerClass呢？在sockerserver模块中提供基类BaseRequestHandler供RequestHandlerClass继承。BaseRequestHandler源码如下：

class BaseRequestHandler:

    # 在初始化时就会调用这些方法
    def __init__(self, request, client_address, server):
        self.request = request
        self.client_address = client_address
        self.server = server
        self.setup()
        try:
            self.handle()
        finally:
            self.finish()

    def setup(self):    # 每一个连接初始化
        pass

    def handle(self):   # 每一次请求处理
        pass 

    def finish(self):   # 每一个连接清理
        pass

由以上源码可以知道在初始化时，setup、handler、finish就会被执行
setup()：执行处理请求之前的初始化相关的各种工作。默认不会做任何事。
handler()：执行那些所有与处理请求相关的工作。默认也不会做任何事。
finish()：执行当处理完请求后的清理工作，默认不会做任何事。

setup, handler, finish the default method does nothing, it is to allow the user to write a subclass RequestHandlerClass, write your own business code in a subclass, then incoming service instance, to handle the client's request.

In summary, use the following steps to create a server requires socketserver:

1. BaseRequestHandler from class to subclass, and covering its handler () method to create a request handler class, this method will process incoming requests.
2. Examples of a server-based, parameter passing, and the address of the server request handler class
3. Call server instance handler_request () or serve_forever () method (to start the service)
4. Call server_close () method (shut down service)

For example

import socketserver
class MyRequestHandler(socketserver.BaseRequestHandler):
    def handle(self):
        print(self.server)  # <socketserver.TCPServer object at 0x00000153270DA320>
        print(self.client_address)  # ('127.0.0.1', 62124)
        print(self.request)  # <socket.socket fd=296, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 999), raddr=('127.0.0.1', 62124)>

s = socketserver.TCPServer(('127.0.0.1', 999), MyRequestHandler)
s.serve_forever()