Editor's recommendation:
Multi-channel I/O reuse is the cornerstone of high-concurrency services. Why redis and nginx perform so sturdily? It is worth your serious study.
The following article comes from the code farmer’s survival on the deserted island, the author of the code farmer’s survival on the deserted island
This article is the third article in the high-performance and high-concurrency series. It follows the previous " What did the program go through when reading files ". After explaining processes, threads and I/O, we came to another key technology in high-concurrency , That is, I/O multiplexing.
Before explaining the technology, we need to preview files and file descriptors.
What is a file
Programmers cannot escape the concept of files when they use I/O.
In the Linux world, a file is a very simple concept. As a programmer, we only need to understand it as a sequence of N bytes :
b1, b2, b3, b4, ....... bN
In fact, all I/O devices are abstracted to the concept of files. Everything is a file. Everything is File, disks, network data, terminals, and even inter-process communication tools, pipes, are all treated as files.
All I/O operations can also be achieved through file reading and writing. This very elegant abstraction allows programmers to use a set of interfaces to perform I/O operations on all peripherals .
Commonly used I/O operation interfaces generally have the following categories:
-
Open the file, open
-
Change the read and write position, seek
-
File reading and writing, read, write
-
Close the file, close
Programmers can achieve almost all I/O operations through these interfaces, which is the power of the concept of files.
File descriptor
In the previous article " What did the program go through when reading files ", we talked about that if we want to perform I/O read operations, like disk data, we need to specify a buff to load the data, which is generally written like this :
read(buff);But here we ignore a key issue, that is, although we specify where to write data, where should we read data from?
We know from the previous section that we can achieve almost all I/O operations through the concept of files, so the one less protagonist here is the file .
So how do we generally use files?
If you go to a hot restaurant on the weekend, you should have experience. Generally, the most popular restaurants will queue up on weekends, and then the waiter will give you a queue number. The waiter can find you through this number. The advantage here is that the waiter does not need to remember you. Who is it, what is your name, where you come from, what is your preference, whether it is to protect the environment and love small animals, etc. The key point here is that the waiter knows nothing about you, but you can still be found by a number .
Similarly, if we want to use files in the Linux world, we also need to use a number. According to the " do not understand the principle ", this number is called file descriptors, file descriptors , which are famous in the Linux world. The reason and the above The queue number is the same.
Therefore, the file description is just a number, but we can operate an open file through this number. Keep this in mind.
With file descriptors, the process can know nothing about the file , such as where the file is on the disk, how it is loaded into the memory, and how to manage it. This information is all taken care of by the operating system, and the process does not need to care about it. The system only needs to give the process one file descriptor.
So let's improve the above procedure:
int fd = open(file_name); // 获取文件描述符read(fd, buff);
How is it, is it very simple?
What to do if there are too many file descriptors
After so much foreshadowing, we finally come to the theme of high performance and high concurrency.
We know from the previous sections that all I/O operations can be carried out through file-like concepts, which of course include network communication.
If you have a web server, after the three-way handshake is successful, we will call accept to get a link. Calling this function will also get a file descriptor. Through this file descriptor, the request sent by the client can be processed and processed The result is sent back. In other words, we can communicate with the client through this descriptor.
// 通过accept获取客户端的文件描述符int conn_fd = accept(...);
The processing logic of the server is usually to read the client request data, and then execute some specific logic:
if(read(conn_fd, request_buff) > 0) {do_something(request_buff);}
Is it very simple, but the world is complicated after all, and of course it is not that simple.
The next step is more complicated.
Since our topic is high concurrency, the server cannot communicate with only one client , but may communicate with thousands of clients at the same time. At this time, you need to process is no longer as simple as a descriptor, but may have to process thousands of descriptors .
In order not to make the problem too complicated, let's simplify it first, assuming that only two client requests are processed at the same time.
Some students may say that this is not easy, just write it like this:
if(read(socket_fd1, buff) > 0) { // 处理第一个do_something();}if(read(socket_fd2, buff) > 0) { // 处理第二个do_something();
In the previous article " What did the program experience when reading files ", we discussed that this is a very typical blocking I/O. If there is no data to read at this time, the process will be blocked and suspended. At this time, we you can not handle the second request, the second request even if the data is already in place, which means that when processing certain client-side due to the lead the rest of the process is blocked for all other clients must wait , while addressing a few This is obviously intolerable on the server of 10,000 clients.
Smart, you will definitely think of using multithreading to open a thread for each client request, so that the blocking of a client will not affect the threads processing other clients. Note that since it is high concurrency, then we have to be successful Do thousands of requests open thousands of threads? A large number of threads created and destroyed will seriously affect system performance.
So how can this problem be solved?
The key point here is that we don’t know in advance whether the corresponding I/O device of a file description is readable and writable. I/O in the unreadable or unwritable state of the peripheral will only lead to The process is blocked and suspended.
Therefore, to solve this problem elegantly, we must think about it from other angles.
Don't call me, I will call you if needed
Everyone must have received sales calls in their lives, and more than one, you will be hollowed out if you receive ten or eight sales calls in a day.
The key point of this scenario is that the caller doesn’t know if you are going to buy something and can only ask you over and over again. Therefore, a better strategy is not to let them call you and write down their phone number. Call them if necessary so that the salesman will not bother you over and over again (although this is not possible in real life).
In this example, you are like the kernel, the promoter is like the application, the phone number is like the file descriptor, and the communication with you on the phone is like I/O.
You should understand by now that the better way to handle multiple file descriptors is actually in the sales call.
Therefore, compared to the previous section, we actively asked the kernel whether the peripherals corresponding to these file descriptors are ready through the I/O interface . A better method is to throw these interested file descriptors at once. The kernel tells the kernel domineeringly: " I have 10,000 file descriptors here. You monitor them for me. When there are file descriptors that can be read and written, please tell me and I can handle them ." Instead of weakly asking the kernel: "Is the first file description readable and writable? Is the second file descriptor readable and writable? Is the third file descriptor readable and writable?..."
In this way, the application has changed from being "busy" to being idle and passive . Anyway, the file description is readable and writable and the kernel will notify me . If I can be lazy, I shouldn't be so hardworking.
This is a more efficient I/O processing mechanism. Now we can handle multiple I/Os at once. Let’s give this mechanism a name, and once again offer " I don’t understand the principle ", it’s called I/O multiple Multiplexing, this is I/O multiplexing.
I/O multiplexing, I/O multiplexing
In fact, the term multiplexing is mostly used in the communication field. In order to make full use of communication lines, it is hoped to transmit multiple signals in one channel. If you want to transmit multiple signals in one channel, you need to combine the multiple signals into one channel and combine multiple signals. The device that is combined into a signal is called a multiplexer. Obviously, the receiver needs to restore the original multiple signal after receiving this combined signal. This device is called a demultiplexer, as shown in the figure:
Back to our topic.
The so-called I/O multiplexing refers to such a process:
1. We got a bunch of file descriptors (whether it's network-related, disk file-related, etc., any file descriptor is fine)
2. Tell the kernel by calling a certain function : " Do not return this function yet. You will monitor these descriptors for me, and return when there are I/O read and write operations in this pile of file descriptors. "
3. When the called function returns, we can know which file descriptors can perform I/O operations.
In other words, we can handle multiple I/O at the same time through I/O multiplexing . So what functions can be used for I/O multiplexing?
There are three mechanisms for I/O multiplexing in the Linux world:
-
select
-
poll
-
epoll
Next, we will introduce NiuBai's I/O multiplexing three swordsmen.
I/O Multiplexing Three Musketeers
Essentially select, poll, and epoll are all blocking I/O, which is what we often call synchronous I/O, because when calling these I/O multiplexing functions, any file descriptor that needs to be monitored is not available. Reading or unwritable then the process will be blocked and suspended execution, until there is a file descriptor readable or writable to continue running.
1, select: fledgling
Under the select I/O multiplexing mechanism, we need to tell select the file description set we want to monitor in the form of function parameters, and then select will copy these file descriptor sets to the kernel. We know that the data copy is There is a performance loss, so in order to reduce the performance loss caused by this data copy, the Linux kernel limits the size of the collection, and stipulates that the collection of file descriptions monitored by the user cannot exceed 1024, and we can only know when the select returns Some file descriptors can be read and written, but we don't know which one , so the programmer must traverse to find out which file descriptor can read and write.
Therefore, in summary, select has several characteristics:
-
The number of file descriptors I can look after is limited, not more than 1024
-
The file descriptor given to me by the user needs to be copied in the kernel
-
I can only tell you that there are file descriptors that meet the requirements, but I don't know which one, you can find them one by one (traversal)
Therefore, we can see that these characteristics of the select mechanism are undoubtedly inefficient in a scenario where a high-concurrency network server often has tens of thousands of concurrent links.
2. Poll: A small achievement
Poll and select are very similar. The optimization of poll relative to select only solves the limitation that file descriptors cannot exceed 1024. Both select and poll will decrease in performance as the number of monitored file descriptions increases, so they are not suitable for high concurrency scenarios. .
3, epoll: unique in the world
Among the three problems that select faces, the limitation on the number of file descriptions has been resolved in poll. What about the remaining two problems?
For the copy problem, the strategy used by epoll is to break and share memory .
In fact, the change frequency of the file descriptor set is relatively low. Select and poll frequently copy the entire set. The kernel is almost annoying. By introducing epoll_ctl, epoll is very considerate and only operates on those file descriptors that have changed. At the same time, epoll I have also become good friends with the kernel, sharing the same memory, which stores the set of file descriptors that are already readable or writable, which reduces the copy overhead of the kernel and programs.
Aiming at the problem of needing to traverse file descriptors to know which is readable and writable, the strategy used by epoll is "being a kid".
Under the select and poll mechanisms, the process has to go to each file descriptor and wait for it . Any file description can be readable or writable to wake up the process, but after the process is awakened, it is still confused and does not know which file description it is. The characters are readable or writable, and you need to check it again from beginning to end.
But epoll is more sensible, and he takes the initiative to find the process to be the younger brother and the older brother.
Under this mechanism, the process does not need to end up in person. The process only needs to wait on epoll, epoll replaces the process to wait on each file descriptor, and tells epoll when which file descriptor is readable or writable, and epoll uses small Book carefully and then wake up the big brother: "Brother process, wake up soon, I have written down all the file descriptors you want to process", so that after the process is awakened, there is no need to check it from beginning to end, because younger brother epoll has already recorded Down.
Therefore, we can see that under the mechanism of epoll, the strategy "Don't call me, I will call you if necessary" is actually used. The process does not need to bother to ask each file descriptor over and over again. Instead, he turned himself over and became the master, "Which of your file descriptors are readable or writable and actively report it". This mechanism is actually the famous event-driven, event-driven, which is also the subject of our next article.
In fact, on the Linux platform, epoll is basically synonymous with high concurrency .
to sum up
Based on the design philosophy that everything is a file, I/O can also be realized in the form of files. Under high concurrency scenarios, it is necessary to interact with multiple files. This is inseparable from efficient I/O multiplexing technology. This article will explain in detail. What is I/O multiplexing and how to use it? Among them, I/O multiplexing (based on event-driven) technology represented by epoll is very widely used. In fact, you will find that whenever it involves high concurrency and high Event-driven programming methods can basically be seen in performance scenarios. Of course, this is also the topic we will focus on in the next article, so stay tuned.