Python network programming basics: network server

In some ways, server programs and client programs are similar. Many of the commands you are familiar with used in the network client program can also be used in the server program, because the server uses the same socket interface as the client.
There are still some important details that are different, the most obvious is the establishment of sockets.

One, ready to connect

For the client, the process of establishing a TCP connection is divided into two steps, including establishment socket对象and invocation connect()to establish a connection with the server.

For the server, this process requires the following 4 steps:

1. Create a socket object
2. Set socket options (optional)
3. Bind to a port (similarly, it can also be a designated network card)
4. Listen for connections

The simplest server-client implementation is as follows:

服务器端：
import socket

host = ''        # Bind to all interfaces
port = 51423

#1
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#2
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
#3
s.bind((host, port))
print("Waiting for connections...")
#4
s.listen(1)

while True:
    clientsock, clientaddr = s.accept()
    print("Got connection from", clientsock.getpeername())
    clientsock.close()

客户端：
import socket

print("Creating socket...")
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print("done.")

print("Looking up port number...")
port = 51423    # socket.getservbyname('http', 'tcp')
print("done.")

print("Connecting to remote host on port %d..." % port)
s.connect(("127.0.0.1", port))
print("done.")

print("Connected from", s.getsockname())
print("Connected to", s.getpeername())

Back to the server-side program:

1. Create a socket object

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

2. Set and get socket options

For a socket, many different options can be set.

For example, if SO_REUSEADDRthe flag is set to true, the operating system will release the server port immediately after the server socket is closed or the server process is terminated. Doing so can make the debugging program easier, and the main thing is to prevent other processes (even another instance of the server itself) from using this port before the timeout.

s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

Method: setsockopt(level, optname, value)andgetsockopt(level, optname[, buflen])

• The level defines which option will be used. UsuallySOL_SOCKET
• The optname parameter provides special options for use, which will vary slightly depending on the operating system. If you want the return value to be an integer, then buflen should not be specified. If you want the return value to be a string, you must specify buflen and give the maximum string length you can accept.

3. Bind the socket

Each server program has its own port, and this port number is well known.
In order to bind a port, you need to use bind():

s.bind((host, port))

• The first parameter is the IP address to be bound. It is usually empty, meaning that it can be bound to all interfaces and addresses.
• One parameter is the port number to be bound.

In fact, the client socket can also be bound to a specific IP address and port number by calling the bindO function. However, this capability of the client is rarely used because the operating system automatically provides the appropriate value.

4. Listen for connections

listen() This call of the function informs the operating system that it is ready to receive connections:

s.listen(1)

• The parameter specifies how many pending (waiting) connections are allowed to wait in the queue when the server actually processes the connection. For modern multi-threaded or multi-tasking servers, this parameter is not very meaningful, but it is also necessary.

Two, accept the connection

Most servers are designed to run for an indefinite period of time (months or even years) and serve multiple connections simultaneously.
In contrast, the client generally has only a few connections and will run until the task is completed or the user terminates them.

The usual way to keep the server running continuously is to carefully design an infinite loop:

while 1:
    clientsock, clientaddr = s.accept()
    print("Got connection from", clientsock.getpeername())
    clientsock.close()

In general, infinite loops are not good because they will exhaust the system's CPU resources. However, this cycle is different: when you call accept()time, there will only return a client connection before.

Three, handling errors

Any exception that is not caught will terminate your program. For the client, this is usually acceptable. In many cases, it is understandable that the client program exits after an error occurs. For servers, this situation is very bad.

In a Python-based network program, an error handling is a simple, standard Python exception handling, which can handle network-related problems.

Therefore, the server program needs to catch all possible network errors and handle these errors in a way that guarantees that the service will not be terminated:

import socket, traceback

host = ''                               # Bind to all interfaces
port = 51423

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((host, port))
s.listen(1)

while 1:
    try:
        clientsock, clientaddr = s.accept()
    except KeyboardInterrupt:
        raise
    except:
        traceback.print_exc()
        continue

    # Process the connection
    try:
        print("Got connection from", clientsock.getpeername())
        # Process the request here
    except (KeyboardInterrupt, SystemExit):
        raise
    except:
        traceback.print_exc()

    # Close the connection
    try:
        clientsock.close()
    except KeyboardInterrupt:
        raise
    except:
        traceback.print_exc()

• The first block contains a call to accept(), where an exception may be generated. The program will regenerate Keyboardinterrupt, so the person running the server presses Ctri-C, and the program will also be terminated as usual. All other exceptions are printed, but the program will not terminate. Instead, it runs a continue statement, which can jump back to the beginning of the loop.
• The second block contains the code that actually handles the connection. It delivers two exceptions: Keyboardinterrupt and the same as before SystemExit. SystemExitIt is to sys.exit ()call generated, if not successfully delivered it will make the program should not be terminated at the time of termination.
• The third block contains a close()call. This call is not part of the second try block, because if it is, an early exception will generate a right close()call and be ignored. In this way, you can ensure that it can close()always be called when needed .

In large programs, it makes sense to use Python's try... and finally block to ensure that the socket is closed. After the accept() function is successfully called, the try statement can be inserted immediately. Before finally calling the close() function, use a finally statement to close the socket.

Fourth, use UDP

In order to use UDP on the server side, you can create a socket, set options, and call bind()functions like TCP . However, there is no need to use listen() or accept() functions, just use recvfrom()functions. This function actually returns two pieces of information: the data received, and the address and port number of the program that sent the data. Because UDP is a connectionless protocol, only one reply needs to be sent. There is no need to have a dedicated socket connected to the remote machine like TCP.

UDP server:

import socket, traceback

host = ''                               # Bind to all interfaces
port = 51423

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((host, port))

while 1:
    try:
        message, address = s.recvfrom(8192)
        print("Got data from", address)
        # Echo it back
        s.sendto(message, address)
    except (KeyboardInterrupt, SystemExit):
        raise
    except:
        traceback.print_exc()

UDP client:

import socket, sys, time

host = '127.0.0.1'
textport = 51423

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
    port = int(textport)
except ValueError:
    # That didn't work.  Look it up instread.
    port = socket.getservbyname(str(textport), 'udp')

s.connect((host, port))
print("Enter data to transmit: ")
data = sys.stdin.readline().strip().encode('utf-8')
s.sendall(bytes(data))
s.shutdown(1)
print("Looking for replies; press Ctrl-C or Ctrl-Break to stop.")
while 1:
    buf = s.recv(2048)
    if not len(buf):
        break
    print("Received: %s" % buf)

A UDP network time client:

import socket, sys, struct, time

hostname = 'time.nist.gov'
port = 37

host = socket.gethostbyname(hostname)

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.sendto(b'', (host, port))

print("Looking for replies; press Ctrl-C to stop.")
buf = s.recvfrom(2048)[0]
if len(buf) != 4:
    print("Wrong-sized reply %d: %s" % (len(buf), buf))
    sys.exit(1)

secs = struct.unpack("!I", buf)[0]
secs -= 2208988800
print(time.ctime(int(secs)))

Five, use inetd or xinetd

So far, all server program examples have some things in common: they all start a process on the server to wait for a connection (or packet), and process them when there is a connection. If you run many different server programs on your machine at the same time, and they are not used frequently, your machine will be consumed by most of the idle processes.

And the TCP examples also have one thing in common: they can only serve a single client at the same time. In actual production servers, this is inappropriate. There are many ways to solve this problem. One way is to use internal methods to handle multiple clients. Another way is to start a copy of the server every time a new client connects.

UNIX and UNIX-like operating systems provide a program called inetdor xinetdto solve these problems. The inetdor xinetdprogram opens, bind, listener and accept every request from the server port. When a client connects, it inetdknows which server program it is requesting (according to the port number that the client information arrives at). Then it inetdwill call the server program and pass the socket to it.

Six, avoid deadlock

A deadlock occurs when a server and a client try to write to and read from a connection at the same time. In these cases, no process can get any data (if they are all reading). If they are writing, the outward buffer will be filled.

There are many ways to solve this problem. The most straightforward is to ensure that the client performs recv() every time after the client finishes executing send(). Another method is to simply make the client send less data (if you change 10485760 to 1024), you will find that there is no problem at all. The third method is to use multiple threads or some other methods, so that the client can send and receive at the same time.