Transfer from: https://zhuanlan.zhihu.com/p/344581947
For personal backup only, please see the original text for browsing
table of Contents
IO multiplexing (including: select/poll/epoll)
The difference between select/poll/epoll
Golang asynchronous IO implementation ideas
Using Golang can easily create a coroutine for each TCP connection to serve without worrying about performance issues. This is because Go internally uses goroutine combined with IO multiplexing to implement an "asynchronous" IO model, which makes developers unnecessary Too much attention to the bottom layer, and only the upper-layer business logic needs to be written as required. How is this asynchronous IO achieved? Below I will analyze the Linux system.
Under Unix/Linux system, everything is a file. Each TCP connection corresponds to a socket handle. This handle can also be regarded as a file. Sending and receiving data on the socket is equivalent to reading and writing a file, so a socket handle , Usually also used to represent the file descriptor fd. You can enter /proc/PID/fd/ to view the fd occupied by the process.
The system kernel allocates a read (receive) buffer and a write (send) buffer for each socket handle. Sending data is to write data on the write buffer corresponding to this fd, and receiving data is to read data on the read buffer. When the program calls write or send, it does not mean that the data is sent out. It just copies the data to the write buffer. When the time is right (accumulated to a certain amount), the data will be sent to the destination.
Golang runtime still needs to frequently check whether fd is ready, strictly speaking, it is not really asynchronous, it is a kind of non-blocking IO reuse.
io model
Blocking IO
When the program wants to read data in the buffer, the buffer does not necessarily have data, which will cause a system call, and can only wait for the data to be read. When there is no data to read, it will block the process. This is blocking IO. When you need to provide services for multiple clients, you can use the threading method, each socket handle uses a thread to serve, so that a certain thread is blocked. Although this can solve process congestion, there is still a considerable amount of CPU resources wasted on waiting for data. At the same time, using threads to serve fd is a waste of resources, because if there are more fd to be processed, it is another resource overhead.
Non-blocking IO
Corresponding to it is non-blocking IO. When the program wants to read data, if the buffer does not exist, it will directly return to the user program, but the user program needs to check frequently until the data is ready. This will also cause empty CPU consumption.
IO multiplexing (including: select/poll/epoll)
Contains: select/poll/epoll
And IO multiplexing is different, he will use a thread to manage multiple fd, you can add multiple fd to the IO multiplexing function, each time the function is called, pass in the fd to be checked, if it is ready FD, directly return the ready fd, and then start the thread processing or sequentially process the ready fd. This achieves that one thread manages multiple fd tasks, which is relatively efficient. Common IO multiplexing functions include select, poll, and epoll. The biggest disadvantage of select and poll is that each call needs to pass in all the fd collections to be monitored, and the kernel traverses this incoming fd collection. When the amount of concurrency is large, the data copy between the user mode and the kernel mode and the kernel Polling fd will waste a wave of system resources (not to expand on select and poll here).
The difference between select/poll/epoll
I/O multiplexing is a mechanism through which a process can monitor multiple file descriptors. Once a descriptor is ready (ready to read or write), it can notify the program to perform the corresponding read and write operations. However, the nature of select, poll, and epoll is still in the category of synchronous I/O (I/O multiplexing itself is synchronous IO), because they all need the thread to read and write after the read and write event is ready, and the process of reading and writing is blocked. of. The realization of asynchronous I/O is that the system will be responsible for copying the data from the kernel space to the user space, without the thread itself blocking read and write, the kernel is ready to be completed.
(1) The implementation of select and poll needs to continuously poll all fd collections by itself until the device is ready, during which it may have to sleep and wake up multiple times. In fact, epoll also needs to call epoll_wait to continuously poll the ready list. During this period, it may sleep and wake up alternately. However, it calls the callback function when the device is ready, puts the ready fd into the ready list, and wakes up to sleep in epoll_wait Process. Although they have to sleep and alternate, select and poll need to traverse the entire fd set when they are "awake", and when epoll is "awake", it only needs to judge whether the ready list is empty, which saves a lot of CPU. time. This is the performance improvement brought by the callback mechanism.
(2) Select, poll each call to copy the fd collection from user mode to kernel mode, and hang current to the device waiting queue once, while epoll only needs to copy once, and hang current to the waiting queue. Hang only once (at the beginning of epoll_wait, note that the waiting queue here is not a device waiting queue, but a waiting queue defined internally by epoll). This can also save a lot of overhead.
epoll introduction
Next, introduce the epoll system call
Compared with select and poll, epoll is more flexible and efficient. It provides users with three system call functions. The bottom layer of Golang is the "asynchronous" IO completed by combining these three system calls with goroutine.
//用于创建并返回一个epfd句柄，后续关于fd的添加删除等操作都依据这个句柄。
int epoll_create(int size);
//用于向epfd添加，删除，修改要监听的fd。
int epoll_ctl(int epfd, int op, int fd, struct epoll_event* event);
//传入创建返回的epfd句柄，以及超时时间，返回就绪的fd句柄。
int epoll_wait(int epfd, struct epoll_event* events, int maxevents, int timeout);- Calling epoll_create will create an eventpoll object in the kernel. This object will maintain an epitem collection, which can be simply understood as an fd collection.
- Calling the epoll_ctl function is used to encapsulate fd into epitem to join the eventpoll object, and add a callback function to this epitem to register it with the kernel, which will be triggered when the fd state changes, making the epitem join the eventpoll ready list rdlist.
- When the corresponding data arrives, the interrupt response program is triggered, the data is copied to the socket buffer of fd, the state of the fd buffer changes, and the callback function adds the epitem corresponding to fd to the rdlist ready queue.
- When calling epoll_wait, there is no need to traverse, but the ready rdlist queue is returned. If the rdlist queue is empty, the wait is blocked or the timeout time arrives.
The general working principle is shown in the figure
Asynchronous IO
When the user program wants to read fd data, the system call directly informs the kernel and returns to handle other things. After the kernel prepares the data, it notifies the user program, and the user program processes the event on the fd.
Golang asynchronous IO implementation ideas
We all know that the resource occupancy of the coroutine is very small, and the coroutine also has a variety of states such as blocked, ready, running, etc. You can use a coroutine to serve one fd without worrying about resource issues. The event of monitoring fd is handed over to the runtime to manage, to realize the coroutine scheduling and events that depend on fd. When the coroutine is required to read fd data but there is no data, park the coroutine (change to Gwaiting) and schedule other coroutines to execute.
When executing the coroutine scheduling, check whether the fd is ready. If it is ready, the scheduler informs the coroutine fd that the park lives in can be processed (changed to Grunnable and added to the execution queue), and the coroutine processes the fd data, so that both Reduced the empty consumption of the CPU, but also realized the notification of the message, and realized an asynchronous IO model from the user level.
The general idea of Golang netpoll is this