1. Understanding of key concepts
-
Synchronous: Initiate a call and return after getting the result.
-
Asynchronous: After the call is initiated, the call returns directly; the caller actively asks the callee to obtain the result, or the callee uses the callback function.
-
Blocking: Calling means that the current thread will be suspended before the call result is returned. The calling thread will return only after getting the result.
-
Non-blocking: The call means that the call will not block the current thread before the result can not be obtained immediately.
Synchronization can be divided into blocking and non-blocking; blocking and non-blocking are related to how to treat the results of things (blocking: I will not leave until the desired result)
2. Clear process status
Understand the state transition of the process
-
Ready state -> Running state: After the process in the ready state is scheduled, it obtains CPU resources (dispatching CPU time slices), so the process changes from the ready state to the running state.
-
Running state -> Ready state: After the time slice runs out, the process in the running state has to give up the CPU, so the process changes from the running state to the ready state. In addition, in a deprivable operating system, when a higher priority
process is ready and ready, the scheduling degree will convert the running process to a ready state, allowing the higher priority process to execute. -
Running state -> blocking state: When a process requests the use and allocation of a certain resource (such as peripherals) or waits for an event (such as the completion of an I/O operation), it transitions from the running state to the blocking state. The process requests the operating system to provide services in the form of system calls. This is a special form in which the operating system kernel process is called by the running user mode program.
-
Blocking state -> Ready state: When the event that the process is waiting for arrives, such as the end of the I/O operation or the end of the interrupt, the interrupt handler must change the state of the corresponding process from the blocked state to the ready state.
The editor recommends my own Linux and C/C++ technical exchange group: [960994558] I have compiled some learning books and video materials that I think are better for sharing (including C/C++, Linux, Nginx, ZeroMQ, MySQL, Redis, fastdfs) , MongoDB, ZK, streaming media, CDN, P2P, K8S, Docker, TCP/IP, coroutine, DPDK, etc.), you can add it yourself if you need it! ~
3. Understand the IO model by executing applications at the operating system level
Blocking IO model:
-
Introduction: The process will be blocked until the data copy is complete, the application calls an IO function, causing the application to block, waiting for the data to be ready.
If the data is not ready, wait forever... The data is ready, copy from the kernel to the user space, and the IO function returns a success indication. The network programming that we first came into contact with
started from interfaces such as listen(), send(), recv(). Using these interfaces can easily build a server/client model. -
Blocking I/O model diagram: When the recv()/recvfrom() function is called, the process of waiting for data and copying data occurs in the kernel.
When the recv() function is called, the system first checks whether there is prepared data. If the data is not ready, then the system is in a waiting state. When the data is ready, copy the data from the system buffer to the user space, and then the function returns. In a socket application, when the recv() function is called, data may not already exist in the user space, then the recv() function will be in a waiting state at this time. The blocking mode brings a big problem to network programming. For example, when calling send(), the thread will be blocked. During this period, the thread will not be able to perform any operations or respond to any network requests. This brings challenges to the network programming of multi-client and multi-service logic. At this time, we may choose a multi-threaded approach to solve this problem. To deal with multi-client network applications, the simplest solution is to use multiple threads (or multiple processes) on the server side. The purpose of multi-threading (or multi-process) is to allow each connection to have an independent thread (or process), so that the blocking of any one connection will not affect other connections. There is no specific model to use multi-process or multi-thread. In the traditional sense, the cost of a process is much greater than that of a thread. Therefore, if you need to provide services to more clients at the same time, it is not recommended to use multiple processes; if a single service execution body needs to consume more
CPU resources, such as Large-scale or long-term data operations or file access, the process is more secure.
Non-blocking IO model
- Introduction: Non-blocking IO repeatedly calls the IO function through the process (multiple system calls and returns immediately); in the process of data copying, the process is blocked;
we set a SOCKET interface to non-blocking to tell the kernel when the requested When the I/O operation cannot be completed, do not put the process to sleep, but return an error. In this way, our I/O operation function will continuously test whether the data is ready, and if it is not ready, continue testing until the data is ready. In this process of continuous testing, a large amount of CPU time will be consumed.
IO reuse model:
- Introduction: IO multiplexing is what we call select, poll, and epoll. In some places, this IO method is also called event driven
IO. The advantage of select/epoll is that a single process can handle multiple network-connected
IO at the same time . Its basic principle is that the function of select, poll, and epoll will continuously poll all the sockets it is responsible for. When a socket has data arrived, it will notify the user process.
When the user process calls select, the entire process will be blocked. At the same time, the kernel will "monitor" all the sockets responsible for select. When the data in any socket is ready, select will return. At this time, the user process calls the read operation again to copy the data from the kernel to the user process. Therefore, the characteristic of I/O multiplexing is that through a mechanism, a process can wait for multiple file descriptors at the same time, and any of these file descriptors (socket descriptors) enters the read-ready state, select( ) Function can return.
Asynchronous IO model
- Introduction: After the user process initiates the read operation, it can start to do other things immediately. On the other hand, from the perspective of the kernel, when it receives an asynchronous
read, it will first return immediately, so it will not generate any block to the user process. Then, the kernel will wait for the completion of the data preparation, and then copy the data to the user memory. When all this is completed, the kernel will send a signal to the user process to tell it that the read operation is complete.
4. Distinguish the select poll epoll in IO multiplexing
select
- int select (int n, fd_set *readfds, fd_set writefds, fd_set exceptfds, struct timeval *timeout); The file descriptors monitored by the select function are divided into 3 categories, namely writefds, readfds, and exceptfds. After the call, the select function will block until a descriptor is ready (there is data to read, write, or except), or timeout (timeout specifies the waiting time, if immediate return is set to null), the function returns. When the select function returns, you can find the ready descriptor by traversing the fdset
poll
- int poll (struct pollfd *fds, unsigned int nfds, int timeout); Unlike select, which uses three bitmaps to represent three fdsets, poll uses a pollfd pointer. There is no limit to the maximum number of pollfd (but performance will decrease if the number is too large). Like the select function, after poll returns, pollfd needs to be polled to obtain the ready descriptor.
epoll
-
Epoll is the ready notification method of the event, calling epoll_create to create an instance, calling epoll_ctl to add or delete the monitored file descriptor, calling epoll_wait to block until there is a ready file descriptor, and returning the file descriptor and event of the ready state through the epoll_event parameter .
-
The epoll operation process requires three interfaces, which are as follows: int epoll_create(int
size);//Create an epoll handle, size is used to tell the kernel how much the number of listeners is to generate an
epoll-specific file descriptor, which is actually an application A kernel space used to store whether and what events happened on the socket fd you want to pay attention to. -
int epoll_ctl(int epfd, int op, int fd, struct epoll_event
*event); Control events on an epoll file descriptor: registration, modification, and deletion. The parameter epfd is a file descriptor dedicated to epoll_create() to create epoll. -
int epoll_wait(int epfd, struct epoll_event * events, int maxevents,
int timeout); Wait for the occurrence of I/O events; return the number of occurrences.
Parameter description:
epfd: Epoll-specific file descriptor generated by epoll_create();
epoll_event: array used to pass back generation processing events;
maxevents: the number of events that can be processed each time;
timeout: timeout waiting for I/O events to occur value;
Difference summary
- The select and poll implementations need to continuously poll all fd sets by themselves until the device is ready, during which it may alternate between sleep and wake up multiple times. In fact, epoll also needs to call epoll_wait to continuously poll the ready list. During this period, it may sleep and wake up alternately. However, it calls the callback function when the device is ready, puts the ready fd into the ready list, and wakes up to sleep in epoll_wait Process. Although they have to sleep and alternate, select and poll need to traverse the entire fd set when they are "awake", and when epoll is "awake", it only needs to judge whether the ready list is empty, which saves a lot of CPU time. This is the performance improvement brought by the callback mechanism.
- Each time select and poll are called, the fd collection is copied from user mode to kernel mode. Epoll maps the kernel space and user space to the same memory through mmap, eliminating the need for copy operations.
Application examples
- Tornado:
Use a single-threaded approach to avoid the performance overhead of thread switching, and at the same time avoid thread insecurity when using some functional interfaces. Support the asynchronous non-blocking network IO model to avoid the main process from blocking waiting.
Tornado's IOLoop module is the core of the asynchronous mechanism. It contains a series of opened file descriptors and handlers for each descriptor. These handlers are packages of select, poll, epoll, etc. (So essentially it is IO multiplexing)
- Django
does not use asynchrony. It uses a multi-process WSGI server (such as uWSGI) to achieve concurrency, which is also a common practice in WSGI.