SocketServer module in Python

SocketServer module in Python

underlying socket programming too, although there are routine programming, but want to write robust code is quite difficult, so many languages ​​of the socket encapsulates the underlying API, Python package is --socketserver module. It is a network service programming framework to facilitate the rapid development of enterprise.

1. The class hierarchy

   +------------+
   | BaseServer |
   +------------+
       |
       v
   +-----------+        +------------------+
   | TCPServer |------->| UnixStreamServer |
   +-----------+        +------------------+
       |
       v
   +-----------+        +--------------------+
   | UDPServer |------->| UnixDatagramServer |
   +-----------+        +--------------------+
   

2. SocketServer simplify writing network servers

   • It has four synchronization classes:
     1. TCPServer
     2. UDPServer
     3. UnixStreamServer
     4. UnixDatagramServer
   • Two Mixin class to support asynchronous.
     • ForkingMixIn
     • ThreadingMixIn
   • Obtained by combining
     • class ForkingUDPServer(ForkingMixIn,UDPServer):pass
     • class ForkingTCPServer(ForkingMixIn,TCPServer):pass
     • class ThreadingUDPServer(ThreadingMixIn,UDPServer):pass
     • class ThreadingTCPServer(ThreadingMixIn,TCPServer):pass

• fork is to create a multi-process, thread is to create multiple threads.
• fork requires operating system support, Windows does not support.
• ThreadingUDPServer and ThreadingTCPServer class specific properties:
  1. daemon_threads = False # The default value is False representation threads created are not daemon threads to True indicates that all threads are created daemon thread
  2. block_on_close = False # The default is Fasle, If True, you can set a daemon thread can use version 3.7

Programming Interface

1. socketserver.BaseServer (server_address, RequestHandlerClass) # instantiates a server

   • Address information server_address # server binding is a tuple (ip, prost)
   • RequestHandlerClass # must be a subclass of BaseRequestHandler.
   • In Central BaseServer code following code:

   def __init__(self, server_address, RequestHandlerClass):
       """Constructor.  May be extended, do not override."""
       self.server_address = server_address
       self.RequestHandlerClass = RequestHandlerClass
       self.__is_shut_down = threading.Event()
       self.__shutdown_request = False
   
   # 处理请求的方法，会实例化一个RequestHandlerClass对象
   def finish_request(self, request, client_address):
       """Finish one request by instantiating RequestHandlerClass."""
       self.RequestHandlerClass(request, client_address, self)
   

2. Interface common attributes and methods defined in BaseServer

method	meaning
server_address	Server address and port is listening, it is not the same in different protocol format. It is a tuple on the Internet Protocol ( "127.0.0.1", 80)
socket	Server is listening socket object. socket
request_queue_size	Request queue size. If a single request takes a long time, then any request arrives at the server is busy will be queued until request_queue_size request. Once the queue is full, further requests from the client will get a "connection refused" error. The default value is usually 5, but can be overridden by a subclass.
address_family	Server socket protocol family belongs. A common example is a socket. AF_INET socket.AF_UNIX.
socket_type	Socket type used by the server; sockets. SOCK_STREAM sockets. SOCK_DGRAM are two common values.
timeout	Timeout duration, in seconds metric, if no timeout, compared None. If handle_request () does not receive incoming requests during the timeout, the call handle_timeout () method.
handle_request()	A single request, the synchronization executing this function call the following method sequence: get_request (), verify_request () and process_request (). If the user handler class provided handle () method throws an exception, the server calls handle_error () method. If you do not receive any request within the timeout seconds, the call handle_timeout () and return handle_request ().
server_forever(poll_interval=0.5)	Performed asynchronously processing requests . Poll_interval poll every second. Ignore timeout attribute, also calls service_actions (), defined in the subclass ForkingMixIn, the process can be used to clean up zombie.
shutdown()	Tell serve_forever cycle stops. And wait for him to end.
server_close()	Shut down the server
finish_request(request,client_address)	RequestHandlerClass by instantiating and call its handle () method to process the request
server_bind()	Called by the server's constructor to bind the socket to the desired address. It may be overwritten.
verify_request(request,client_address)	It must return a Boolean value; if the value is True, the request will be processed, if the value is False, the request is rejected. You can override this function to implement server access control. The default implementation always returns True.

BaseRequestHandler class

1. And the user of the user is connected to the base class processing request class,
2. BaseRequestHandler(request,client_address,server) #Constructor
   • request # is the object and the client socket connections
   • client_address # client address
   • server # is the instance itself TCPServer
3. After the server receives a user request Server instance, eventually instantiate this class. It is initialized, configured into three parameters: request, client_address, after the server itself can be used in the following instance attributes BaseRequestHandler class:
   • self.request socket object and the client is connected
   • self.server instance itself is TCPServer
   • self.client_address client address
4. This class during initialization, it will call three methods in turn. Subclasses can override these methods.

class BaseRequestHandler:

    def __init__(self, request, client_address, server):
        self.request = request
        self.client_address = client_address
        self.server = server
        self.setup()
        try:
            self.handle()
        finally:
            self.finish()

    def setup(self): #每一个连接初始化
        pass

    def handle(self): #每一次请求处理
        pass

    def finish(self): #每一个连接清理
        pass

• Test code

import socketserver
import socket

class MyBaseRequestHandle(socketserver.BaseRequestHandler):

    def setup(self):
        super().setup() #可以不调用父类的setup()方法，父类的setup方法什么都没做
        print("----setup方法被执行-----")

    def handle(self):
        super().handle() #可以不调用父类的handler(),方法，父类的handler方法什么都没做
        print("-------handler方法被执行----")
        print(self.server)
        print(self.request) #服务
        print(self.client_address) #客户端地址
        print(self.__dict__)
        print("- "*30)
        print(self.server.__dict__)
        print("- "*30)
        sk:socket.socket = self.request
        data = sk.recv(1024)
        print(data)
        sk.send("{}-{}".format(sk.getpeername(),data).encode())

        print("----------handler end ----------")

    def finish(self):
        super().finish() #可以不调用父类的finish(),方法，父类的finish方法什么都没做
        print("--------finish方法被执行---")

laddr = "127.0.0.1",3999
tcpserver = socketserver.TCPServer(laddr,MyBaseRequestHandle) #注意：参数是MyBaseRequestHandle
tcpserver.handle_request() #只接受一个客户端连接
# tcpserver.serve_forever() #永久循环执行,可以接受多个客户端连接

• The request came on each different connection, generating the connected socket object self.request i.e., client address self.client_address.
• The ThreadingTCPServer replaced TCPServer, when each client connection comes in will create a new thread.
• ThreadingTCPServer is asynchronous and can handle multiple connections.
• TCPServer are synchronized, a connection process is over, that is connected to a handle method of execution is over, in order to deal with another connection, and only the main thread.

to sum up

• Creating a server requires several steps:
  1. BaseRequestHandler classes derive from a subclass, and covering its handle () method to create a request handler class, this method will process incoming requests
  2. Examples of a server-based, parameter passing, and the address of the server request handler class
  3. Call server instance handle_request () or serve_forever () method
  4. Call server_close () closes the socket

Achieve EchoServer

• Echo : to declare the message, echoing what the message is, that is what the client sent a message, the message is what is returned

import logging
import sys
import socketserver
import socket
import threading

logging.basicConfig(format="%(asctime)s %(thread)d %(threadName)s %(message)s",stream=sys.stdout,level=logging.INFO)

class Handler(socketserver.BaseRequestHandler):
    def setup(self):
        super().setup()
        self.event = threading.Event()
        logging.info("新加入了一个连接{}".format(self.client_address))

    def handle(self):
        super().handle()
        sk:socket.socket = self.request
        while not self.event.is_set():
            try:
                data = sk.recv(1024).decode()
            except Exception as e:
                logging.info(e)
                break

            logging.info(data)
            msg = "{}-{}".format(self.client_address,data).encode()
            sk.send(msg)

    def finish(self):
        super().finish()
        self.event.set()
        self.request.close()

if __name__ == "__main__":
    server = socketserver.ThreadingTCPServer(("127.0.0.1",3999),Handler)
    threading.Thread(target=server.serve_forever,name="server").start()
    while True:
        cmd = input(">>>")
        if cmd.strip() == "quit":
            server.server_close()
            break
        logging.info(threading.enumerate())

Combat, using socketserver achieve group chat server

• Use ThreadingTCPServer rewrite ChatServer
• Use BaseRequestHandler defined Handler

import logging
import sys
import socketserver
import socket
import threading

logging.basicConfig(format="%(asctime)s %(thread)d %(threadName)s %(message)s",stream=sys.stdout,level=logging.INFO)
log = logging.getLogger()

class Handler(socketserver.BaseRequestHandler):
    lock = threading.Lock()
    clients = {}

    def setup(self):
        super().setup()
        self.event = threading.Event()
        with self.lock:
            self.clients[self.client_address] = self.request
        log.info("新加入了一个连接{}".format(self.client_address))

    def handle(self):
        super().handle()
        sock:socket.socket = self.request
        while not self.event.is_set():
            try:
                data = sock.recv(1024)
            except Exception as e:
                log.error(e)
                data = b""
            log.info(data)
            if data == b"by" or data == b"":
                break
            msg = "service:{}-->{}".format(self.client_address, data).encode()
            expc = []  # 记录sock出错时对应的clients
            with self.lock:
                for c, sk in self.clients.items():
                    try:
                        sk.send(msg)  # 可能在发送消息是就出错
                    except:
                        expc.append(c)
                for c in expc:
                    self.clients.pop(c)

    def finish(self):
        super().finish()
        self.event.set()
        with self.lock:
            if self.client_address in self.clients:
                self.clients.pop(self.client_address)
        self.request.close()
        log.info("{}退出了".format(self.client_address))

if __name__ == "__main__":
    server = socketserver.ThreadingTCPServer(("127.0.0.1",3999),Handler)
    server.daemon_threads = True  #设置所有创建的线程都为Daemo线程
    threading.Thread(target=server.serve_forever,name="server",daemon=True).start()
    while True:
        cmd = input(">>>")
        if cmd.strip() == "quit":
            server.shutdown() #告诉serve_forever循环停止。
            server.server_close()
            break
        logging.info(threading.enumerate())

• Use StreamRequestHandler defined handler

import logging
import sys
import socketserver
import socket
import threading

logging.basicConfig(format="%(asctime)s %(thread)d %(threadName)s %(message)s",stream=sys.stdout,level=logging.INFO)
log = logging.getLogger()

class Handler(socketserver.StreamRequestHandler):
    lock = threading.Lock()
    clients = {}

    def setup(self):
        super().setup()
        self.event = threading.Event()
        with self.lock:
            self.clients[self.client_address] = self.request
        log.info("新加入了一个连接{}".format(self.client_address))

    def handle(self):
        super().handle()
        import io
        rfile:io.TextIOWrapper= self.rfile
        while not self.event.is_set():
            try:
                data = rfile.read1(1024) #类似于sock.recv(1024)
                # data = rfile.readline() #行读取
            except Exception as e:
                log.error(e)
                data = b""
            log.info(data)
            if data == b"by" or data == b"":
                break
            msg = "service:{}-->{}".format(self.client_address, data).encode()
            expc = []  # 记录sock出错时对应的clients
            with self.lock:
                for c, sk in self.clients.items():
                    try:
                        sk.send(msg)  # 可能在发送消息是就出错
                    except:
                        expc.append(c)
                for c in expc:
                    self.clients.pop(c)

    def finish(self):
        super().finish()
        self.event.set()
        with self.lock:
            if self.client_address in self.clients:
                self.clients.pop(self.client_address)
        self.request.close()
        log.info("{}退出了".format(self.client_address))

if __name__ == "__main__":
    server = socketserver.ThreadingTCPServer(("127.0.0.1",3999),Handler)
    server.daemon_threads = True  #设置所有创建的线程都为Daemo线程
    threading.Thread(target=server.serve_forever,name="server",daemon=True).start()
    while True:
        cmd = input(">>>")
        if cmd.strip() == "quit":
            server.shutdown() #告诉serve_forever循环停止。
            server.server_close()
            break
        logging.info(threading.enumerate())

• to sum up
  1. Each connection provides a class RequestHandlerClass examples, in turn call setup, handle, fi nish methods, and uses try ... fi nally fi nish method must ensure that the structure can be called. These methods are sequentially executed, and if you want to maintain this client communications connection, it is necessary to use in the loop handle function.
  2. Different classes socketserver module, but the programming interface is the same, even if the multi-process, multi-threaded class is the same, greatly reducing the difficulty of programming.
  3. The simplified socket programming, the programmer only needs to focus on data processing itself, to achieve Handler class on the line. This style is very common in Python.