Python socket programming

1. Socket

1. What is a socket?

　　Socket is an intermediate software abstraction layer for communication between the application layer and the TCP / IP protocol family. It is a set of interfaces. In the design mode, Socket is actually a facade mode. It hides the complex TCP / IP protocol family behind the Socket interface. For users, a simple set of interfaces is all. Let Socket organize the data to meet the specified protocol.

　　Therefore, we do not need to deeply understand the tcp / udp protocol. The socket has been encapsulated for us. We only need to follow the socket's regulations to program, and the written program naturally follows the tcp / udp standard.

　　The application layer is actually TCP or UDP protocol, because socket can be regarded as non-existent, because it is only a encapsulation process, from the perspective of the protocol does not belong to the transport layer, so usually based on network communication protocols are these two .

2. Socket development history

　　Sockets originated in the 1970s at the University of California, Berkeley version of Unix, or BSD Unix. Therefore, sometimes people also refer to sockets as "Berkeley sockets" or "BSD sockets". In the beginning, sockets were designed to communicate between multiple applications on the same host. This is also known as inter-process communication, or IPC. There are two kinds of sockets (or called two races), which are file-based and network-based.

* Socket family based on file type *

Socket family name: AF_UNIX

Everything in Unix is ​​a file. The file-based socket calls the underlying file system to fetch data. Two socket processes run on the same machine and can communicate indirectly by accessing the same file system.

* Socket family based on network type *

Socket family name: AF_INET

(Also AF_INET6 is used for ipv6, and there are some other address families, but they are either only used for a certain platform, or are either abandoned, or rarely used, or not implemented at all, all addresses In the family, AF_INET is the most widely used one. Python supports many address families, but since we only care about network programming, most of the time we only use AF_INET)

Second, the socket workflow

　A scene in life. You want to call a friend, dial the number first, the friend raises the phone after hearing the ringtone, and then you establish a connection with your friend and you can talk. When the communication is over, hang up the phone to end the conversation. The scenes in life explain how this works.

　　Let's start with the server. The server first initializes the Socket, then binds to the port (bind), listens to the port (listen), calls accept to block, and waits for the client to connect. At this time, if a client initializes a Socket and then connects to the server (connect), if the connection is successful, then the connection between the client and the server is established. The client sends a data request, the server receives the request and processes the request, then sends the response data to the client, the client reads the data, and finally closes the connection, one interaction ends

socket () module function usage

import socket
socket.socket(socket_family,socket_type,protocal=0)
"""
socket_family 可以是 AF_UNIX 或 AF_INET。
socket_type 可以是 SOCK_STREAM  流式协议-->tcp协议
			     SOCK_DGRAM.protocol   报式协议-->udp协议
			     一般不填,默认值为 0,也就是流式协议-->tcp协议
"""

# 获取tcp/ip套接字(流式协议)
tcpSock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

#获取udp/ip套接字(报式协议)
udpSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
 
"""
由于 socket 模块中有太多的属性。我们在这里破例使用了'from module import *'语句。使用 'from socket import *',我们就把 socket 模块里的所有属性都带到我们的命名空间里了,这样能 大幅减短我们的代码。
"""
tcpSock = socket(AF_INET, SOCK_STREAM

The server socket function
s.bind () binds (host, port number) to the socket
s.listen () starts TCP listening
s.accept () passively accepts TCP client connections, (blocking) waiting for connection arrival

The client socket function
s.connect () actively initializes the TCP server connection
s.connect_ex () The extended version of the connect () function returns an error code when an error occurs instead of throwing an exception

The public socket function
s.recv () receives TCP data
s.send () sends TCP data (send data is lost when the amount of data to be sent is greater than the remaining space in the own buffer area, and will not be sent out)
s.sendall () Send complete TCP data (essentially, send is cyclically called, sendall will not lose data when the amount of data to be sent is greater than the remaining space in the own buffer area, cyclically call send until it is finished)
s.recvfrom () Receive UDP data
s. sendto () sends UDP data
s.getpeername () connects to the address of the remote end of the
current socket
s.getsockname () returnsthe address of the current sockets.getsockopt () returns the parameters of the specified socket
s.setsockopt () Set the parameter
s.close () of thespecified socket toclose the socket

The lock-oriented socket method
s.setblocking () sets the blocking and non-blocking modes of the socket
s.settimeout () sets the timeout time for blocking socket operations
s.gettimeout () gets the timeout time for blocking socket operations

File-oriented socket function
s.fileno () socket file descriptor
s.makefile () creates a file related to the socket

Three, TCP-based socket

* tcp is based on the link, you must start the server first, and then start the client to link the server *

* tcp server *

1 ss = socket() #创建服务器套接字
2 ss.bind()      #把地址绑定到套接字
3 ss.listen()      #监听链接
4 inf_loop:      #服务器无限循环
5     cs = ss.accept() #接受客户端链接
6     comm_loop:         #通讯循环
7         cs.recv()/cs.send() #对话(接收与发送)
8     cs.close()    #关闭客户端套接字
9 ss.close()        #关闭服务器套接字(可选)

* tcp client *

1 cs = socket()    # 创建客户套接字
2 cs.connect()    # 尝试连接服务器
3 comm_loop:        # 通讯循环
4     cs.send()/cs.recv()    # 对话(发送/接收)
5 cs.close()            # 关闭客户套接字

The socket communication process is similar to the call process. We will use a call as an example to implement a low version of socket communication:

Process: Run the server first, and find that the server is blocked in place. The discovery is caused by accept, because you have to go to the semi-connection pool to get things, but no, you can only wait. After the server is connected, the server will print the ip and port. When the client does not send data, although the server has passed the accept, but there is no recv to the data. After the client sends the data, it immediately prints out the received Content, then the server capitalizes the received content, and then sends it back to the client, then the client prints out the data processed by the server, and finally the client closes and recycles the operating system resources

Note: accept, recv, send will cause blocking

Server:

import socket

# 买手机
phone = socket.socket(socket.AF_INET,socket.SOCK_STREAM)

# 绑定手机卡
phone.bind(('127.0.0.1',8080)) # 0-65535,1024以前的都被系统保留使用

# 开机
phone.listen(backlog=5) # 5指的是半连接池的大小
print('服务端启动完成')

# 等待电话连接请求
conn,client_addr = phone.accept() # 执行一次接收一个连接请求
print(conn)
print('客户端的Ip和端口:',client_addr)

# 收消息
data = conn.recv(1024) # 最大接收的数据量为1024Bytes，收到的是bytes类型,无穷增大没有意义，而且再大不能大过内存的大小，要像水流一样一点一点接收
print('客户端的Ip和端口:',client_addr)
conn.send(data.upper())

"""
关闭电话连接coon
因为accept建立的连接请求占的是python应用程序的内存空间所占
但这个连接请求是操作系统帮我们维护者tcp的双向连接，所以要考虑回收操作系统资源
"""
conn.close()

# 关机(可选操作)
phone.close()

Client:

import socket

#1、买手机
phone=socket.socket(socket.AF_INET,socket.SOCK_STREAM) # 流式协议=》tcp协议

#2、拨通服务端电话
phone.connect(('127.0.0.1',8080))

#3、通信
import time
time.sleep(10)
phone.send('hello 噗噗'.encode('utf-8'))
data=phone.recv(1024)
print(data.decode('utf-8'))

#4、关闭连接(必选的回收资源的操作)
phone.close()

Plus communication loop

Bug exists:

　　1. Send empty is no problem, but recv empty will cause blocking

　　Both sending and receiving are not operating for the other party. They are all sending system calls to call their own operations from their own cache. In fact, they are all in their own cache and are managed by their own operating system, but they will eventually reach the other party. If it is sent, it is called from the cache. If there is nothing, it will not be done, but if it is received, it will be found in the cache. If it is not, it will wait until it knows that there is data in the cache.

　　2. If the client is forcibly terminated, the server will also collapse, and the situation of the server under different systems is different, there are two types:

　　　　Windows will directly report an error: the remote host forced to close an existing link

　　　　unix: will enter an endless loop

　　Because the link is a two-way link, when the client performs the input operation, there is a link maintained between the server and the client. The client waits to lose, and the server waits to receive under normal circumstances, but now The client forcibly terminates, and the server thinks that the connection is normal, so an error occurs.

Improved server version:

import socket

# 1、买手机
phone=socket.socket(socket.AF_INET,socket.SOCK_STREAM) # 流式协议=》tcp协议

# 2、绑定手机卡
phone.bind(('127.0.0.1',8080)) # 0-65535, 1024以前的都被系统保留使用

# 3、开机
phone.listen(5)
print('服务端启动完成，监听地址为:%s:%s' %('127.0.0.1',8080))

# 4、等待电话连接请求：拿到电话连接conn
conn,client_addr=phone.accept()

# 5、通信：收\发消息
while True:
    try:
        data=conn.recv(1024) # 最大接收的数据量为1024Bytes，收到的是bytes类型
        if len(data) == 0:
            # 在unix系统洗，一旦data收到的是空
            # 意味着是一种异常的行为：客户度非法断开了链接
            break
        print("客户端发来的消息：",data.decode('utf-8'))
        conn.send(data.upper())
    except Exception:
        # 针对windows系统
        break

# 6、关闭电话连接conn（必选的回收资源的操作)
conn.close()

# 7、关机（可选操作）
phone.close()

Improved version of the client

import socket

#1、买手机
phone=socket.socket(socket.AF_INET,socket.SOCK_STREAM) # 流式协议=》tcp协议

#2、拨通服务端电话
phone.connect(('127.0.0.1',8080))

#3、通信
while True:
    msg=input("输入要发送的消息>>>: ").strip() #msg=''
    if len(msg) == 0:continue
    phone.send(msg.encode('utf-8'))
    print('======')
    data=phone.recv(1024)
    print(data.decode('utf-8'))

#4、关闭连接(必选的回收资源的操作)
phone.close()

Plus link loop (solve existing problems)

The characteristics that the server should meet:

　　1. Always provide services

　　2. Provide services concurrently

Improved server version:

import socket

# 1、买手机
phone=socket.socket(socket.AF_INET,socket.SOCK_STREAM) # 流式协议=》tcp协议

# 2、绑定手机卡
phone.bind(('127.0.0.1',8080)) # 0-65535, 1024以前的都被系统保留使用

# 3、开机
phone.listen(5) # 5指的是半连接池的大小
print('服务端启动完成，监听地址为:%s:%s' %('127.0.0.1',8080))

# 4、等待电话连接请求：拿到电话连接conn
# 加上链接循环
while True:
    conn,client_addr=phone.accept()

    # 5、通信：收\发消息
    while True:
        try:
            data=conn.recv(1024) # 最大接收的数据量为1024Bytes，收到的是bytes类型
            if len(data) == 0:
                # 在unix系统洗，一旦data收到的是空
                # 意味着是一种异常的行为：客户度非法断开了链接
                break
            print("客户端发来的消息：",data.decode('utf-8'))
            conn.send(data.upper())
        except Exception:
            # 针对windows系统
            break

    # 6、关闭电话连接conn（必选的回收资源的操作)
    conn.close()

# 7、关机（可选操作）这样就没有用了
phone.close()

problem:

Some students may encounter when restarting the server

This is because your server still has four waved time_wait states occupying the address (if you do n’t understand, please study 1.tcp three handshake, four waved 2.syn flood attack 3. There will be a lot of high server concurrency The optimization method of time_wait state)

Solution:

method one

#加入一条socket配置，重用ip和端口

phone=socket(AF_INET,SOCK_STREAM)
phone.setsockopt(SOL_SOCKET,SO_REUSEADDR,1) #就是它，在bind前加
phone.bind(('127.0.0.1',8080))

Method Two

发现系统存在大量TIME_WAIT状态的连接，通过调整linux内核参数解决，
vi /etc/sysctl.conf

编辑文件，加入以下内容：
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 30

然后执行 /sbin/sysctl -p 让参数生效。

net.ipv4.tcp_syncookies = 1 表示开启SYN Cookies。当出现SYN等待队列溢出时，启用cookies来处理，可防范少量SYN攻击，默认为0，表示关闭；

net.ipv4.tcp_tw_reuse = 1 表示开启重用。允许将TIME-WAIT sockets重新用于新的TCP连接，默认为0，表示关闭；

net.ipv4.tcp_tw_recycle = 1 表示开启TCP连接中TIME-WAIT sockets的快速回收，默认为0，表示关闭。

net.ipv4.tcp_fin_timeout 修改系統默认的 TIMEOUT 时间

Method three:

　　Change the port number, remember to change the client and server!

Three, UDP-based socket

• udp client

  import socket
  
  client=socket.socket(socket.AF_INET,socket.SOCK_DGRAM) # 流式协议=》tcp协议
  
  while True:
      msg=input('>>>: ').strip()
      client.sendto(msg.encode('utf-8'),('127.0.0.1',8080))
      res=client.recvfrom(1024)
      print(res)
  
  client.close()
  

• udp server

  import socket
  
  server=socket.socket(socket.AF_INET,socket.SOCK_DGRAM) # 数据报协议=》udp协议
  
  server.bind(('127.0.0.1',8080))
  
  while True:
      data,client_addr=server.recvfrom(1024)
      server.sendto(data.upper(),client_addr)
  
  server.close()