An introduction to the IO model
To better understand the IO model, we need to review beforehand: synchronous, asynchronous, blocking, non-blocking
What are synchronous IO and asynchronous IO, blocking IO and non-blocking IO, and what is the difference? In fact, different people may give different answers to this question. For example, wikis think that asynchronous IO and non-blocking IO are the same thing. This is actually because different people have different knowledge backgrounds, and the context when discussing this issue is also different. Therefore, in order to better answer this question, I will first limit the context of this article.
The background discussed in this article is network IO in the Linux environment. The most important reference for this article is "UNIX® Network Programming Volume 1, Third Edition: The Sockets Networking" by Richard Stevens, Section 6.2 "I/O Models", where Stevens details the characteristics and differences of various IOs , if the English is good enough, it is recommended to read it directly. Stevens' style of writing is famous for explaining the simple language, so don't worry about not understanding it. The flow charts in this article are also taken from references.
Stevens compared a total of five IO models in the article:
* blocking IO
* nonblocking IO
* IO multiplexing
* signal driven IO
* asynchronous IO
is not commonly used in practice by signal driven IO (signal driven IO), so the other four IO Models are mainly introduced.
Let's talk about the objects and steps involved when IO occurs. For a network IO (here we take read as an example), it will involve two system objects, one is the process (or thread) that calls this IO, and the other is the system kernel (kernel). When a read operation occurs, the operation goes through two phases:
#waiting for the data to be ready #copying the data from the kernel to the process
It's important to keep these two points in mind, because the difference between these IO models is that there are different situations in each of these two phases.
Replenish:
1. Input operation: read, readv, recv, recvfrom, recvmsg, a total of five functions. If the state is blocked, it will go through two stages of wait data and copy data. If it is set to non-blocking, it will wait and throw when data is not available. exception 2. Operation output: write, writev, send, sendto, sendmsg, a total of five functions. When the send buffer is full, it will block in place. If it is set to non-blocking, an exception will be thrown. 3. Accept external connections: accept, and Input operation is similar 4. Send out an external link: connect, similar to the output operation
Two blocking IO (blocking IO)
In linux, all sockets are blocked by default, a typical read operation flow is probably like this
When the user process calls the recvfrom system call, the kernel begins the first stage of IO: preparing data. For netword io, many times the data has not arrived at the beginning (for example, a complete UDP packet has not been received), at this time the kernel has to wait for enough data to arrive.
On the user process side, the entire process will be blocked. When the kernel waits until the data is ready, it will copy the data from the kernel to the user memory
, and then the kernel returns the result, and the user process releases the block state and starts running again.
Therefore, the characteristic of blocking IO is that both stages of IO execution (waiting for data and copying data) are blocked.
Many programmers first come into contact with network programming from the excuses such as listen()\send()\recv()\, which can be used to easily build a server/client model. However, most socket interfaces are blocking.
PS: The so-called blocking excuse means that a system call (usually an IO interface) does not return the call result and keeps the current thread blocked, and only returns when the system call obtains the result or the timeout error occurs.
A simple solution:
#Use multithreading (or multiprocessing) on the server side. The purpose of multithreading (or multiprocessing) is to allow each connection to have an independent thread (or process), so that the blocking of any one connection will not affect other connections
The problem with this program is:
#Enable multi-threading or multi-process method. When you encounter hundreds or thousands of connection requests at the same time, no matter multi-threading or multi-process will seriously occupy system resources, reduce the system's response efficiency to the outside world, and the threads and processes themselves will also Easier to enter suspended animation
improve proposals:
#Many programmers may consider using "thread pool" or "connection pool". The "thread pool" is designed to reduce the frequency of thread creation and destruction, maintain a reasonable number of threads, and let idle threads resume new execution tasks. The "connection pool" maintains a buffer pool of connections, reuses existing connections as much as possible, and reduces the frequency of creating and closing connections. Both of these two technologies can reduce system overhead very well, and are widely used in many large systems, such as websphere, tomcat and various databases.
There are actually problems with the improved plan
# The "thread pool" and "connection pool" technologies only alleviate the resource occupation caused by frequent calls to the IO interface to a certain extent. Moreover, the so-called "pool" always has its upper limit. When the request greatly exceeds the upper limit, the response of the system formed by the "pool" to the outside world is not much better than when there is no pool. Therefore, the use of "pool" must consider the size of the response it faces, and adjust the size of the "pool" according to the size of the response.
Corresponding to all the thousands or even tens of thousands of client requests that may appear at the same time in the above list, 'thread pool' or 'connection pool' may relieve some of the pressure, but it cannot solve all the problems. In short, the multi-threaded model can easily and efficiently solve small-scale service requests, but in the face of large-scale service requests, the multi-threaded model will also encounter bottlenecks. Non-blocking interfaces can be used to try to solve this problem.
Three non-blocking IO
Under Linux, you can make it non-blocking by setting the socket. When performing a read operation on a non-blocking socket, the flow looks like this:
As can be seen from the figure, when the user process issues a read operation, if the data in the kernel is not ready, it does not block the user process, but returns an error immediately. From the perspective of the user process, after it initiates a read operation, it does not need to wait, but gets a result immediately. When the user process judges that the result is an error, it knows that the data is not ready, so the user can do other things within the time interval from this time to the next time when the read query is issued, or directly send the read operation again. Once the data in the kernel is ready and the system call of the user process is received again, it immediately copies the data to the user memory (it is still blocked at this stage) and returns.
That is to say, after the non-blocking recvform system call is called, the process is not blocked, and the kernel returns to the process immediately. If the data is not ready, an error will be returned at this time. After the process returns, it can do something else and then issue the recvform system call. Repeat the above process, repeating the recvform system call. This process is often referred to as polling. Polling checks the kernel data until the data is ready, and then copies the data to the process for data processing. It should be noted that during the entire process of copying data, the process is still in a blocked state.
Therefore, in non-blocking IO, the user process actually needs to continuously and actively ask whether the kernel data is ready.
server
import time s=socket() s.bind(('127.0.0.1',8080)) s.listen(5) s.setblocking(False) r_list=[] w_list=[] while True: try: conn,addr=s.accept() r_list.append(conn) except BlockingIOError: print ( ' can do other work ' ) print ( ' rlist: ' ,Len(r_list)) del_rlist=[] for item in w _list: try: data=conn.recv(1024) conn.send(data.upper()) except BlockingIOError: continue del_wlist=[] for item in w_list: try: conn=item[0] res=item[1] conn.send(res) del_wlist.append(item) except BlockingIOError: continue except ConnectionAbortedError: conn.close() del_wlist.append(item) #Recycle useless connections for conn in del_rlist: r_list.remove(conn) for item in del_wlist: w_list.remove(item)
#客户端
import socket
import os
client=socket。socket（）
client.connect(('127.0.0.1',8080))
while True:
res=('%s hello'%os.getpid()).encode('utf-8')
client.send(res)
data=client.recv(1024)
print(data.decode('utf-8'))
Four multiplexed IO
The term IO multiplexing may be a bit unfamiliar, but if I say select/epoll, I can probably understand. In some places, this IO method is also called event driven IO (event driven IO). We all know that the advantage of select/epoll is that a single process can handle the IO of multiple network connections at the same time. Its basic principle is that the select/epoll function will continuously poll all the sockets it is responsible for. When a certain socket has data arriving, it will notify the user process. Its process is as follows:
When the user process calls select, the entire process will be blocked, and at the same time, the kernel will "monitor" all sockets that select is responsible for. When the data in any socket is ready, select will return. At this time, the user process calls the read operation again to copy the data from the kernel to the user process.
This graph is not much different from the blocking IO graph, in fact it is worse. Because two system calls (select and recvfrom) need to be used here, and blocking IO only calls one system call (recvfrom). However, the advantage of using select is that it can handle multiple connections at the same time.
emphasize:
1. If the number of connections processed is not very high, the web server using select/epoll is not necessarily better than the web server using multi-threading + blocking IO, and the delay may be greater. The advantage of select/epoll is not that it can handle a single connection faster, but that it can handle more connections.
2. In the multiplexing model, for each socket, it is generally set to non-blocking, but, as shown in the figure above, the entire user process is actually blocked all the time. It's just that the process is blocked by the select function, not by socket IO.
in conclusion:
The advantage of select is that it can handle multiple connections, not a single connection
# select network IO model # server #server _ from socket import * import select server = socket (AF_INET, SOCK_STREAM) server.bind(('127.0.0.1',8093)) server.listen(5) server.setblocking(False) print('starting.....') rlist=[server,] wlist = [] wdata=[] while True: rl,wl,xl=select.select(rlist,wlist,[],0.5) print(wl) for sock in rl: if sock ==server: conn,addr=sock.accept() rlist.append(conn) else: try: data=sock.recv(1024) if not data: sock.close() rlist.remove(sock) continue wlist.append(sock) wdata[sock]=data.upper() except Exception: sock.close() rlist.remove(sock) for sock in wl: sock.send(wdata[sock]) wlist.remove(sock) wdata.pop(sock)
#client from socket import * _ client = socket (AF_INET, SOCK_STREAM) client.connect(('127.0.0.1',8093)) while True: msg=input('>>: ').strip() if not msg:continue client.send(msg.encode('utf-8')) data=client.recv(1024) print(data.decode('utf-8')) client.close()
Process analysis of select monitoring fd changes:
#The user process creates a socket object and copies the monitored fd to the kernel space. Each fd will correspond to a system file table. After the fd in the kernel space responds to the data, it will send a signal to the user process that the data has arrived; #User process again Send a system call, such as (accept) to copy the data in the kernel space to the user space, and at the same time clear the data in the kernel space as the accepting data end, so that the fd can respond again when there is new data when listening again (the sender is based on the TCP protocol. So it needs to be cleared after receiving a response).
Advantages of this model:
#Compared with other models, the event-driven model using select() only uses a single thread (process) for execution, occupies less resources, does not consume too much CPU, and can provide services for multiple clients at the same time. If the view builds a simple time-driven server program, then the model has a certain reference value.
Disadvantages of this model:
#First of all the select() interface is not the best choice for implementing "event-driven". Because when the handle value to be detected is large, the select() interface itself needs to consume a lot of time to poll each handle. Many operating systems provide more efficient interfaces, such as linux provides epoll, BSD provides kqueue, Solaris provides /dev/poll, …. If you need to implement a more efficient server program, an interface like epoll is more recommended. Unfortunately, the epoll interface provided by different operating systems is very different, so it is difficult to implement a server with better cross-platform capability using an interface similar to epoll. #Secondly , this model mixes event detection and event response. Once the executive body of event response is huge, it will be catastrophic to the entire model.
Five asynchronous IO
Asynchronous IO under linux is not very useful, and it was introduced from the kernel version 2.6. Let's take a look at its process
After the user process initiates the read operation, it immediately starts to do other things. On the other hand, from the perspective of the kernel, when it receives an asynchronous read, it will return immediately, so it will not generate any block for the user process. Then, the kernel will wait for the data preparation to complete, and then copy the data to user memory. When all this is done, the kernel will send a signal to the user process, telling it that the read operation is complete.