[Notes] python advanced static web server (including epoll realize)

table of Contents

1.http protocol (understand)

1.1. HTTP request

1.2. HTTP format

2. Web server Static

2.1 py simple http server

2.2. Py implementation returns a browser page http server needed

3. Web static server (multi-tasking)

3.1. Implement multi-process http server

3.2. Multithread http server

3.3. Gevent achieve http server

3.4. Non-blocking mode

    3.4.1 non-blocking single task

    3.4.2 non-blocking single task (long link)

    3.4.3 epoll realize http server


 

1.http protocol (understand)

HTTP (Hyper Text Transfer Protocol) <Hypertext Transfer Protocol>.

1.1. HTTP request

1) First, the browser sends an HTTP request to the server, the request comprising:

Method: GET or POST, GET only request resources, POST will be included with the user data;

Path: / full / url / path;

Domain Name: Specify the Host header: Host: www.sina.com

And other related Header;

If is POST, then the request further comprises a Body, including user data

2) server returns the HTTP response to the browser, the response comprising:

Response code: 200 indicates success, 3xx redirection represents, 4xx representing a request sent by a client error, 5xx an error occurred while the server process;

Response Type: specified by the Content-Type;

And other related Header;

Usually HTTP server's response may carry the content, that is, there is a Body, a response of the content, the page's HTML source code in the Body.

3) If the browser is required to continue to request additional resources to the server, such as pictures, it makes an HTTP request again, repeat steps 1 and 2.

Web HTTP protocol used with a very simple request - response pattern, which greatly simplifies the development. When we write a page, we just need to put HTML in the HTTP request is sent out, no need to consider how that came with pictures, video and other browsers if needed request pictures and videos, it will send another HTTP request, therefore, an HTTP processing a resource request only (at this time can be understood as a short connection in the TCP protocol, only one link for each resource, such as the need to establish multiple links requires a plurality)

 

1.2. HTTP format

Each HTTP requests and responses follow the same format, an HTTP Header and Body comprising two portions, wherein Body is optional.

HTTP protocol is a text protocol, so its format is very simple.

. 1) HTTP GET request format:

    GET /path HTTP/1.1
    Header1: Value1
    Header2: Value2
    Header3: Value3

. 2) HTTP POST request format:

    POST /path HTTP/1.1
    Header1: Value1
    Header2: Value2
    Header3: Value3

    body data goes here...

NOTE: When faced with two consecutive \ r \ When n (a blank line), the end of the Header portion, all of the latter data Body.

. 3) HTTP response format:

    200 OK
    Header1: Value1
    Header2: Value2
    Header3: Value3

    body data goes here...

NOTE: HTTP response if containing body, but also by \ r \ n \ r \ n separated. Body type of data is determined by the Content-Type head, if a Web page, is the Body text, if a picture, the picture is the Body of binary data.

 

2. Web server Static

 

2.1 py simple http server

import socket


def service_client(new_socket):
    """为这个客户端返回数据"""

    # 1. 接收浏览器发送过来的请求 ,即http请求  
    # GET / HTTP/1.1
    # .....
    request = new_socket.recv(1024)
    print(request)

    # 2. 返回http格式的数据,给浏览器
    # 2.1 准备发送给浏览器的数据---header
    response = "HTTP/1.1 200 OK\r\n"
    response += "\r\n"
    # 2.2 准备发送给浏览器的数据---boy
    response += "hello world"
    new_socket.send(response.encode("utf-8"))  # python3发送的必须是字节类型

    # 关闭套接字
    new_socket.close()
    

def main():
    """用来完成整体的控制"""
    # 1. 创建套接字
    tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    # 设置当服务器先close 即服务器端4次挥手之后资源能够立即释放,这样就保证了,下次运行程序时 可以立即绑定7788端口
    tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

    # 2. 绑定
    tcp_server_socket.bind(("", 7890))

    # 3. 变为监听套接字
    tcp_server_socket.listen(128)

    while True:
        # 4. 等待新客户端的链接
        new_socket, client_addr = tcp_server_socket.accept()

        # 5. 为这个客户端服务
        service_client(new_socket)

    # 关闭监听套接字
    tcp_server_socket.close()


if __name__ == "__main__":
    main()

After running in the browser input  http://127.0.0.1:7890/  , the server will then return a response, the browser displays hello world.

 

Note: it is best to understand the process of the TCP three-way handshake, waving four times (detailed reference area Network knowledge).

Simply refer to the following:

General client to emphasize close.

 

 2.2. Py implementation returns a browser page http server needed

In 2.1, http server returns a fixed string, this section required to achieve return to the browser page http server.

 

Example: first implementation returns index.html

First prepare a folder index.html file in the html file before running for return to the client.

import socket


def service_client(new_socket):
    """为这个客户端返回数据"""

    # 1. 接收浏览器发送过来的请求 ,即http请求  
    # GET / HTTP/1.1
    # .....
    request = new_socket.recv(1024)
    print(">>>"*50)
    print(request)

    # 2. 返回http格式的数据,给浏览器
    # 2.1 准备发送给浏览器的数据---header
    response = "HTTP/1.1 200 OK\r\n"
    response += "\r\n"
    # 2.2 准备发送给浏览器的数据---boy
    # response += "hahahhah"
    
    f = open("./html/index.html", "rb")
    html_content = f.read()
    f.close()

    # 因为是以二进制打开的文件,不能直接response+=html_content

    # 将response header发送给浏览器
    new_socket.send(response.encode("utf-8"))
    # 将response body发送给浏览器
    new_socket.send(html_content)

    # 关闭套接
    new_socket.close()
    

def main():
    """用来完成整体的控制"""
    # 1. 创建套接字
    tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

    # 2. 绑定
    tcp_server_socket.bind(("", 7890))

    # 3. 变为监听套接字
    tcp_server_socket.listen(128)

    while True:
        # 4. 等待新客户端的链接
        new_socket, client_addr = tcp_server_socket.accept()

        # 5. 为这个客户端服务
        service_client(new_socket)

    # 关闭监听套接字
    tcp_server_socket.close()


if __name__ == "__main__":
    main()

After running in the browser input  http://127.0.0.1:7890/  , i.e., the server returns a response, the browser displays the contents of index.html corresponding to the front end.

 

Examples : to improve , to achieve the required return to the browser page . At this time first sent, to the client decodes the request. After decoding the extracted file name of the page requested by the user.

import socket
import re


def service_client(new_socket):
    """为这个客户端返回数据"""

    # 1. 接收浏览器发送过来的请求 ,即http请求  
    # GET / HTTP/1.1
    # .....
    request = new_socket.recv(1024).decode("utf-8")  # decode进行解码
    # print(">>>"*50)
    # print(request)  # 会打印解码后的请求

    request_lines = request.splitlines()  # 对请求按行(请求的每行内容都有一个含义)分割,返回一个列表
    print("")
    print(">"*20)
    print(request_lines)

    # GET /index.html HTTP/1.1   (这是请求的第一行)
    # get post put del
    file_name = ""
    ret = re.match(r"[^/]+(/[^ ]*)", request_lines[0])  # 依据请求第一行解析文件名([^/]+匹配多个非/,[^ ]*匹配0或多个非空格)
    if ret:
        file_name = ret.group(1)
        # print("*"*50, file_name)
        if file_name == "/":
            file_name = "/index.html"  # 设置请求没指定文件名时,默认的请求页面

    # 2. 返回http格式的数据,给浏览器
    try:
        f = open("./html" + file_name, "rb")  # 上面匹配不到时,没有file_name会异常
    except:
        response = "HTTP/1.1 404 NOT FOUND\r\n"
        response += "\r\n"
        response += "------file not found-----"
        new_socket.send(response.encode("utf-8"))
    else:
        html_content = f.read()
        f.close()
        # 2.1 准备发送给浏览器的数据---header
        response = "HTTP/1.1 200 OK\r\n"
        response += "\r\n"
        # 2.2 准备发送给浏览器的数据---boy
        # response += "hahahhah"

        # 将response header发送给浏览器
        new_socket.send(response.encode("utf-8"))
        # 将response body发送给浏览器
        new_socket.send(html_content)


    # 关闭套接
    new_socket.close()
    

def main():
    """用来完成整体的控制"""
    # 1. 创建套接字
    tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

    # 2. 绑定
    tcp_server_socket.bind(("", 7890))

    # 3. 变为监听套接字
    tcp_server_socket.listen(128)

    while True:
        # 4. 等待新客户端的链接
        new_socket, client_addr = tcp_server_socket.accept()

        # 5. 为这个客户端服务
        service_client(new_socket)

    # 关闭监听套接字
    tcp_server_socket.close()


if __name__ == "__main__":
    main()

After running in the browser input http://127.0.0.1:7890/  specified file name (eg: http://127.0.0.1:7890/ index.html), the server will then return a response, the browser displays the file exists the corresponding content. 

 

3. Web static server (multi-tasking)

 

3.1. Implement multi-process http server

accept a process that is still to achieve, and service of process with a new process to achieve.

(Note: the first ready resources before running)

import socket
import re
import multiprocessing


def service_client(new_socket):
    """为这个客户端返回数据"""

    # 1. 接收浏览器发送过来的请求 ,即http请求  
    # GET / HTTP/1.1
    # .....
    request = new_socket.recv(1024).decode("utf-8")
    # print(">>>"*50)
    # print(request)

    request_lines = request.splitlines()
    print("")
    print(">"*20)
    print(request_lines)

    # GET /index.html HTTP/1.1
    # get post put del
    file_name = ""
    ret = re.match(r"[^/]+(/[^ ]*)", request_lines[0])
    if ret:
        file_name = ret.group(1)
        # print("*"*50, file_name)
        if file_name == "/":
            file_name = "/index.html"




    # 2. 返回http格式的数据,给浏览器
    
    try:
        f = open("./html" + file_name, "rb")
    except:
        response = "HTTP/1.1 404 NOT FOUND\r\n"
        response += "\r\n"
        response += "------file not found-----"
        new_socket.send(response.encode("utf-8"))
    else:
        html_content = f.read()
        f.close()
        # 2.1 准备发送给浏览器的数据---header
        response = "HTTP/1.1 200 OK\r\n"
        response += "\r\n"
        # 2.2 准备发送给浏览器的数据---boy
        # response += "hahahhah"

        # 将response header发送给浏览器
        new_socket.send(response.encode("utf-8"))
        # 将response body发送给浏览器
        new_socket.send(html_content)


    # 关闭套接
    new_socket.close()
    

def main():
    """用来完成整体的控制"""
    # 1. 创建套接字
    tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

    # 2. 绑定
    tcp_server_socket.bind(("", 7890))

    # 3. 变为监听套接字
    tcp_server_socket.listen(128)

    while True:
        # 4. 等待新客户端的链接
        new_socket, client_addr = tcp_server_socket.accept()

        # 5. 为这个客户端服务(服务过程用新的进程实现)
        p = multiprocessing.Process(target=service_client, args=(new_socket,))  # 进程执行service_client,参数new_socket
        p.start()

        new_socket.close()  # 通俗理解:由于创建了新的进程(复制了父进程的socket)


    # 关闭监听套接字
    tcp_server_socket.close()


if __name__ == "__main__":
    main()

3.2. Multithread http server

The main difference between the code and the multi-process, the multiprocessing into threading. Note that child thread socket does not copy the main thread, so only once new_socket.close () can be.

import socket
import re
import threading


def service_client(new_socket):
    """为这个客户端返回数据"""

    # 1. 接收浏览器发送过来的请求 ,即http请求  
    # GET / HTTP/1.1
    # .....
    request = new_socket.recv(1024).decode("utf-8")
    # print(">>>"*50)
    # print(request)

    request_lines = request.splitlines()
    print("")
    print(">"*20)
    print(request_lines)

    # GET /index.html HTTP/1.1
    # get post put del
    file_name = ""
    ret = re.match(r"[^/]+(/[^ ]*)", request_lines[0])
    if ret:
        file_name = ret.group(1)
        # print("*"*50, file_name)
        if file_name == "/":
            file_name = "/index.html"




    # 2. 返回http格式的数据,给浏览器
    
    try:
        f = open("./html" + file_name, "rb")
    except:
        response = "HTTP/1.1 404 NOT FOUND\r\n"
        response += "\r\n"
        response += "------file not found-----"
        new_socket.send(response.encode("utf-8"))
    else:
        html_content = f.read()
        f.close()
        # 2.1 准备发送给浏览器的数据---header
        response = "HTTP/1.1 200 OK\r\n"
        response += "\r\n"
        # 2.2 准备发送给浏览器的数据---boy
        # response += "hahahhah"

        # 将response header发送给浏览器
        new_socket.send(response.encode("utf-8"))
        # 将response body发送给浏览器
        new_socket.send(html_content)


    # 关闭套接
    new_socket.close()
    

def main():
    """用来完成整体的控制"""
    # 1. 创建套接字
    tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

    # 2. 绑定
    tcp_server_socket.bind(("", 7890))

    # 3. 变为监听套接字
    tcp_server_socket.listen(128)

    while True:
        # 4. 等待新客户端的链接
        new_socket, client_addr = tcp_server_socket.accept()

        # 5. 为这个客户端服务
        p = threading.Thread(target=service_client, args=(new_socket,))
        p.start()

        # new_socket.close()  # 线程不会复制主线程的socket,所以只用一次new_socket.close()即可。


    # 关闭监听套接字
    tcp_server_socket.close()


if __name__ == "__main__":
    main()

 

Update : The  multi-process web server object oriented

import socket
import re
import multiprocessing


class WSGIServer(object):
    def __init__(self):
        # 1. 创建套接字
        self.tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

        # 2. 绑定
        self.tcp_server_socket.bind(("", 7890))

        # 3. 变为监听套接字
        self.tcp_server_socket.listen(128)

    def service_client(self, new_socket):
        """为这个客户端返回数据"""

        # 1. 接收浏览器发送过来的请求 ,即http请求  
        # GET / HTTP/1.1
        # .....
        request = new_socket.recv(1024).decode("utf-8")
        # print(">>>"*50)
        # print(request)

        request_lines = request.splitlines()
        print("")
        print(">"*20)
        print(request_lines)

        # GET /index.html HTTP/1.1
        # get post put del
        file_name = ""
        ret = re.match(r"[^/]+(/[^ ]*)", request_lines[0])
        if ret:
            file_name = ret.group(1)
            # print("*"*50, file_name)
            if file_name == "/":
                file_name = "/index.html"


        # 2. 返回http格式的数据,给浏览器
        
        try:
            f = open("./html" + file_name, "rb")
        except:
            response = "HTTP/1.1 404 NOT FOUND\r\n"
            response += "\r\n"
            response += "------file not found-----"
            new_socket.send(response.encode("utf-8"))
        else:
            html_content = f.read()
            f.close()
            # 2.1 准备发送给浏览器的数据---header
            response = "HTTP/1.1 200 OK\r\n"
            response += "\r\n"
            # 2.2 准备发送给浏览器的数据---boy
            # response += "hahahhah"

            # 将response header发送给浏览器
            new_socket.send(response.encode("utf-8"))
            # 将response body发送给浏览器
            new_socket.send(html_content)


        # 关闭套接
        new_socket.close()
        

    def run_forever(self):
        """用来完成整体的控制"""

        while True:
            # 4. 等待新客户端的链接
            new_socket, client_addr = self.tcp_server_socket.accept()

            # 5. 为这个客户端服务
            p = multiprocessing.Process(target=self.service_client, args=(new_socket,))
            p.start()

            new_socket.close()


        # 关闭监听套接字
        self.tcp_server_socket.close()


def main():
    """控制整体,创建一个web 服务器对象,然后调用这个对象的run_forever方法运行"""
    wsgi_server = WSGIServer()
    wsgi_server.run_forever()


if __name__ == "__main__":
    main()

 

3.3. Gevent achieve http server

Question: Why after this example gevent.spawn create an object, the object does not need to call the .join () or gevent.joinall () the thread execution ? ? ?

import socket
import re
import gevent
from gevent import monkey

monkey.patch_all()


def service_client(new_socket):
    """为这个客户端返回数据"""

    # 1. 接收浏览器发送过来的请求 ,即http请求  
    # GET / HTTP/1.1
    # .....
    request = new_socket.recv(1024).decode("utf-8")
    # print(">>>"*50)
    # print(request)

    request_lines = request.splitlines()
    print("")
    print(">"*20)
    print(request_lines)

    # GET /index.html HTTP/1.1
    # get post put del
    file_name = ""
    ret = re.match(r"[^/]+(/[^ ]*)", request_lines[0])
    if ret:
        file_name = ret.group(1)
        # print("*"*50, file_name)
        if file_name == "/":
            file_name = "/index.html"

    # 2. 返回http格式的数据,给浏览器
    try:
        f = open("./html" + file_name, "rb")
    except:
        response = "HTTP/1.1 404 NOT FOUND\r\n"
        response += "\r\n"
        response += "------file not found-----"
        new_socket.send(response.encode("utf-8"))
    else:
        html_content = f.read()
        f.close()
        # 2.1 准备发送给浏览器的数据---header
        response = "HTTP/1.1 200 OK\r\n"
        response += "\r\n"
        # 2.2 准备发送给浏览器的数据---boy
        # response += "hahahhah"

        # 将response header发送给浏览器
        new_socket.send(response.encode("utf-8"))
        # 将response body发送给浏览器
        new_socket.send(html_content)

    # 关闭套接
    new_socket.close()
    

def main():
    """用来完成整体的控制"""
    # 1. 创建套接字
    tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

    # 2. 绑定
    tcp_server_socket.bind(("", 7890))

    # 3. 变为监听套接字
    tcp_server_socket.listen(128)

    while True:
        # 4. 等待新客户端的链接
        new_socket, client_addr = tcp_server_socket.accept()

        # 5. 为这个客户端服务
        gevent.spawn(service_client, new_socket)  # 协程在service_client执行,参数new_socket

        # new_socket.close()

    # 关闭监听套接字
    tcp_server_socket.close()


if __name__ == "__main__":
    main()

 

3.4. Non-blocking mode

In python, sleeve byte may be set to blocking mode or non-blocking mode . In non-blocking mode, after calling API, such as send () or recv () method will throw an exception if you encounter problems. In blocking mode, popular to say that about every poll monitor encountered an error and does not prevent the operation.

Object .setblocking (False)   , the socket is plugged in the manner provided.

3.4.1 non-blocking single task

The key code within the while, socket waiting for data within the socket waiting for the arrival of the client list and affect each other.

Note: The practical application of this small ways, but the process is to be understood in this way is concurrent rather than in parallel.

import socket
import time

tcp_server_tcp = socket.socket(socket.AF_INET, socket.SOCK_STREAM)  # 创建套接字
tcp_server_tcp.bind(("", 7899))  # 绑定
tcp_server_tcp.listen(128)
tcp_server_tcp.setblocking(False)  # 设置套接字为非堵塞的方式

client_socket_list = list()  # 列表,有新的客户端链接时,将其客户端添加到列表中

while True:

    # time.sleep(0.5)  # 延时方便看效果

    try:  # 收不到值时产生异常
        new_socket, new_addr = tcp_server_tcp.accept()
    except Exception as ret:
        print("---没有新的客户端到来---")
    else:
        print("---只要没有产生异常,那么也就意味着 来了一个新的客户端----")  # 此时还不能调用recv,因为调用后又会堵塞
        new_socket.setblocking(False)  # 设置套接字为非堵塞的方式
        client_socket_list.append(new_socket)  # 有新的客户端连接,将其socket添加到列表
        
    for client_socket in client_socket_list:
        try:
            recv_data = client_socket.recv(1024)  # 检测列表的socket是否有数据
        except Exception as ret:
            print(ret)
            print("----这个客户端没有发送过来数据----")  # client没发送数据,产生异常
        else:
            print("-----没有异常-----")
            print(recv_data)
            if recv_data:
                # 对方发送过来数据
                print("----客户端发送过来了数据-----")
            else:
                # 对方调用close 导致了有数据,但recv返回空,recv_data为空
                client_socket.close()
                client_socket_list.remove(client_socket)
                print("---客户端已关闭----")

 

3.4.2 non-blocking single task (long link)

Popular, short each acquire a data connection to establish a link (three-way handshake connection waved off four times). Procedure: establishing a connection - Data Transfer - Close ... connected to establish a connection - Data Transfer - close the connection;

In a long link link, the same data through a socket to obtain several needs, so you can save server resources. Procedure: establishing a connection - Data Transfer ... (stay connected) ... Data Transfer - close the connection.

Note: http1.1 version uses a long link. Although the previous code with the 1.1 version of the http protocol, but the connection manually after each call to the socket object .close (), or lead to short links.

Non-blocking single task (long link) code: add in the header Content-the Length: , used to record the contents of a length, to tell the browser how long the package, the browser and so determine whether the transmission finished. So you do not like to judge by the same short link disconnected pass over.

import socket
import re


def service_client(new_socket, request):
    """为这个客户端返回数据"""

    # 1. 接收浏览器发送过来的请求 ,即http请求  
    # GET / HTTP/1.1
    # .....
    # request = new_socket.recv(1024).decode("utf-8")
    # print(">>>"*50)
    # print(request)

    request_lines = request.splitlines()
    print("")
    print(">"*20)
    print(request_lines)

    # GET /index.html HTTP/1.1
    # get post put del
    file_name = ""
    ret = re.match(r"[^/]+(/[^ ]*)", request_lines[0])
    if ret:
        file_name = ret.group(1)
        # print("*"*50, file_name)
        if file_name == "/":
            file_name = "/index.html"

    # 2. 返回http格式的数据,给浏览器
    
    try:
        f = open("./html" + file_name, "rb")
    except:
        response = "HTTP/1.1 404 NOT FOUND\r\n"
        response += "\r\n"
        response += "------file not found-----"
        new_socket.send(response.encode("utf-8"))
    else:
        html_content = f.read()
        f.close()

        response_body = html_content

        response_header = "HTTP/1.1 200 OK\r\n"
        response_header += "Content-Length:%d\r\n" % len(response_body)  # 记下内容的长度,告诉浏览器包有多长
        response_header += "\r\n"

        response = response_header.encode("utf-8") + response_body  # 读到的response_body是二进制,response_header是字符串

        new_socket.send(response)  # 发送二进制的数据
   
    # # 关闭监听套接字
    # new_socket.close()


def main():
    """用来完成整体的控制"""
    # 1. 创建套接字
    tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

    # 2. 绑定
    tcp_server_socket.bind(("", 7890))

    # 3. 变为监听套接字
    tcp_server_socket.listen(128)
    tcp_server_socket.setblocking(False)  # 将套接字变为非堵塞

    client_socket_list = list()
    while True:
        # 4. 等待新客户端的链接
        try:
            new_socket, client_addr = tcp_server_socket.accept()
        except Exception as ret:
            pass
        else:  # 没有异常
            new_socket.setblocking(False)
            client_socket_list.append(new_socket)

        for client_socket in client_socket_list:
            try:
                recv_data = client_socket.recv(1024).decode("utf-8")
            except Exception as ret:
                pass
            else:
                if recv_data:  # 有数据
                    service_client(client_socket, recv_data)
                else:  # 空数据,即浏览器断开连接
                    client_socket.close()
                    client_socket_list.remove(client_socket)

    # 关闭监听套接字
    tcp_server_socket.close()


if __name__ == "__main__":
    main()

 

3.4.3 epoll realize http server

Linux servers are usually implemented using epoll (Note: epoll is a single-process, single-threaded), can achieve high concurrency.

Detailed epoll may refer to: [learning] Linux epoll explain  .

select / epoll use of IO multiplexing (sometimes called event driven IO), when a socket data arrives, the user is notified process. Benefits of a single IO can be processed simultaneously process a plurality of network connections.

Reference features are as follows: By a mechanism for simultaneously process a plurality of waiting file descriptors, and these file descriptors (socket descriptor) any of which enters a state ready for reading, the epoll () function may return. So, IO multiplexing, there will be no concurrent functions in nature, because any time or only one process or thread to work, it has been able to improve efficiency because select \ epoll to come into their socket 'monitoring 'list which, if any socket has a readable and writable data processing immediately, and that if the select \ epoll hands simultaneously detect a lot of socket, there is a movement to return immediately to the treatment process, are better than one over a socket, block waiting for processing high effectiveness.

 

code: epoll realize HTTP (understanding the implementation process):

  1. Import: import select
  2. Epoll create objects: epl = select.epoll ()
  3. The listening socket corresponding FD (File Discriptor file descriptor) registered to the epoll: epl.register (tcp_server_socket.fileno (), select.EPOLLIN)  ( Note : Parameter Two: select.EPOLLIN | select.EPOLLET, the former detecting whether there is an input, which is the ET mode )
  4. fd_event_list = epl.poll () # epl.poll ( ) default will block until the os monitoring data to come tell the program through an event notification method, this time will de-clog. Fd and returns a list consisting of event. (List format: [(fd, Event), (corresponding socket file descriptors, file descriptors in the end this is what you can call recv events such as reception, etc.) ...] )

  5. Data processing ......

  6. Logout: epl.unregister (fd).

Note: The above 3, epoll file descriptor has two operation modes: LT (Trigger Level) and the ET (Edge Trigger) . LT mode is the default mode, the difference between LT and ET mode mode are as follows:

  • LT mode: When epoll descriptor detected events and event notification application, the application may not process the immediate event. The next call to epoll, will respond to the application again and notice this event.
  • ET mode: When epoll descriptor detected events and event notification application, the application must handle the event immediately. If untreated, the next time you call epoll, will not respond to the application again and notice this event.

EPOLLIN (read)

EPOLLOUT (writable)

EPOLLET (ET mode)

import socket
import re
import select


def service_client(new_socket, request):
    """为这个客户端返回数据"""

    # 1. 接收浏览器发送过来的请求 ,即http请求  
    # GET / HTTP/1.1
    # .....
    # request = new_socket.recv(1024).decode("utf-8")
    # print(">>>"*50)
    # print(request)

    request_lines = request.splitlines()
    print("")
    print(">"*20)
    print(request_lines)

    # GET /index.html HTTP/1.1
    # get post put del
    file_name = ""
    ret = re.match(r"[^/]+(/[^ ]*)", request_lines[0])
    if ret:
        file_name = ret.group(1)
        # print("*"*50, file_name)
        if file_name == "/":
            file_name = "/index.html"

    # 2. 返回http格式的数据,给浏览器
    
    try:
        f = open("./html" + file_name, "rb")
    except:
        response = "HTTP/1.1 404 NOT FOUND\r\n"
        response += "\r\n"
        response += "------file not found-----"
        new_socket.send(response.encode("utf-8"))
    else:
        html_content = f.read()
        f.close()

        response_body = html_content

        response_header = "HTTP/1.1 200 OK\r\n"
        response_header += "Content-Length:%d\r\n" % len(response_body)
        response_header += "\r\n"

        response = response_header.encode("utf-8") + response_body

        new_socket.send(response)


def main():
    """用来完成整体的控制"""
    # 1. 创建套接字
    tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

    # 2. 绑定
    tcp_server_socket.bind(("", 7890))

    # 3. 变为监听套接字
    tcp_server_socket.listen(128)
    tcp_server_socket.setblocking(False)  # 将套接字变为非堵塞

    # 创建一个epoll对象!!!
    epl = select.epoll()

    # 将监听套接字对应的fd注册到epoll中
    epl.register(tcp_server_socket.fileno(), select.EPOLLIN)

    fd_event_dict = dict()  # 定义字典用来存放fd对应的套接字

    while True:

        fd_event_list = epl.poll()  # 默认会堵塞,直到 os监测到数据到来 通过事件通知方式 告诉这个程序,此时才会解堵塞。返回event和fd组成的列表

        # [(fd, event), (套接字对应的文件描述符, 这个文件描述符到底是什么事件 例如 可以调用recv接收等)]
        for fd, event in fd_event_list:
            # 等待新客户端的链接
            if fd == tcp_server_socket.fileno():  # 接收到新的客户端链接。产生新的套接字。将其注册到epoll中
                new_socket, client_addr = tcp_server_socket.accept()
                epl.register(new_socket.fileno(), select.EPOLLIN)
                fd_event_dict[new_socket.fileno()] = new_socket  # 存放fd对应的套接字
            elif event == select.EPOLLIN:  # 事件类型为EPOLLIN,有数据
                # 判断已经链接的客户端是否有数据发送过来
                recv_data = fd_event_dict[fd].recv(1024).decode("utf-8")  # fd_event_dict[fd]根据套接字取socket
                if recv_data:
                    service_client(fd_event_dict[fd], recv_data)
                else:  # 收到空数据
                    fd_event_dict[fd].close()
                    epl.unregister(fd)
                    del fd_event_dict[fd]  # 删除字典对应的ky


    # 关闭监听套接字
    tcp_server_socket.close()


if __name__ == "__main__":
    main()

 

 

Published 50 original articles · won praise 10 · views 6615

Guess you like

Origin blog.csdn.net/qq_23996069/article/details/104076227