In fact, everyone should know about the IO model of nginx. In short, it is a master process and multiple worker processes (the number of processes is determined by the configuration). The master process is responsible for accepting requests and queuing them, and finally forwarding them to the worker process for processing. The entire process of processing requests and responses. However, this is at the process level, and each process is single-threaded. Nginx provides multi-threading in version 1.7.11, but this multi-threading is only used in the operation of local files in the aio model (IO model). The starting point is to use non-blocking mode to improve the efficiency of file IO and concurrency capabilities; so this multi-threading is not used by nginx to process proxy requests in a multi-threaded way (this part is through epoll mode), but is used to process some local static files.
There are several instructions involved here: sendfile, aio, directio, they are all related to the operation of local files, let's take a look at their meanings:
1、sendfile：
It has the same semantics as the system "sendfile()" parameter. The purpose of sendfile is to improve the efficiency of sending local files through sockets; disks, network drives, and memory are three different transmission media. If a file is read locally and sent through socket Sending out, usually goes through the following steps:
1) The disk drive reads the byte data of a certain length (chunk) from the disk according to the scheduling of the CPU 2) The byte data is copied to the kernel memory 3) The data in the kernel memory is copied to the process workspace memory 4) The process Copy the data to the network drive cache through the socket, and send it out through the corresponding transmission protocol.
We can simply see that the data sending process involves multiple copies, which is limited by the design of the computer system; then the main starting point of sendfile is to reduce the copy of data to improve the sending efficiency, sendfile is a linux system-level call, socket The file data can be directly accessed through DMA (direct memory access) and sent through the transfer protocol, reducing 2 data copies (disk to kernel, kernel to workspace).
The sendfile_max_chunk parameter is used to limit the maximum data size sent by each sendfile() call. If the size is not limited, the entire worker process will be monopolized. The default is "unlimited".
For nginx, it is very efficient to proxy static local static file resources (usually small files). It is recommended to enable this parameter for some static files such as html, pictures, etc.
location /video {
sendfile on;
sendfile_max_chunk 128k;
is going to be;
}
2、directio
Enable the use of the O_DIRECT flag (BSD, linux), corresponding to the directio() system call, this parameter is set for large files, sendfile is for small files, and the limited size can be specified through directio. size file, will use directio (instead of using sendfile). According to the original design of directio, it has the basic principle of sendfile, but it does not use the kernel cache, but directly uses DMA, and the memory cache (page-aligned part) will also be released after use, so directio is usually suitable for large file reading, And usually the read frequency is very low, and it does not improve the efficiency for high-frequency reads (because it does not reuse the cache, but DMAs it every time). Because of performance tradeoffs, this parameter defaults to off.
location /video {
sendfile on;
direction 8m
is going to be;
}
3、aio
We will think of the aio model. In fact, the semantics are basically the same, that is, asynchronous file IO. nginx turns off this feature by default. It needs to be supported on a higher version of the linux platform (2.6.22+). On the linux platform, directio can only read blocks aligned based on 512-byte boundaries. Those unaligned blocks at the end of the file will be read in blocking mode. Likewise, if the file is not aligned at the beginning, the entire file will be read in blocking mode. , the so-called alignment here is the cache situation of the file data in the memory page.
When both aio and sendfile are enabled, the aio mechanism will be used for files whose size is greater than the set value of directio, that is, sendfile will be used directly for files smaller than the set value of directio (aio does not participate).
In simple terms, aio is to use multi-threaded asynchronous mode to read larger files to improve IO efficiency, but in fact, there may not be any improvement, because the reading of large files cannot use cache, and it is also time-consuming. Even if it is multi-threaded, the waiting time for requests is unpredictable, especially when the concurrent requests are high, but the concurrency ability of aio to improve IO is certain.
By default, multi-threading mode is turned off. We need to enable it through the "--with-threads" configuration. This feature is compatible with platforms that support epoll and kqueue. For the setting of the thread pool, we can declare it through "thread_pool" and specify it in the aio directive.
thread_pool default_pool threads=16;##main上下文
...
location /video {
sendfile on;
sendfile_max_chunk 128k;
direction 8M
aio threads=default_pool;
}
When all threads in the thread pool are in the busy state, new task requests will be added to the waiting queue. We can use the "max_queue" parameter in thread_pool to specify the size of the queue. The default queue size is 65536. When the queue is full Subsequent requests will throw an error.
4. Other
Nginx officially claims to use multi-threading mode. In the scenario of aio reading files, the performance is improved by 9 times. See the document ; but I still have a certain skepticism about this test. Multi-threading + aio can indeed improve file IO to a certain extent. Read performance, but for large files, this does not seem to be as good as imagined, which is limited by the underlying characteristics of the Linux platform, unless nginx itself does additional operations on the file cache.
So far, we still recommend that you:
1) For the static proxy of small files, we should turn on sendfile, which has a significant performance improvement.
2) For large file reading (low frequency), we can try to open aio and directio, and pay attention to the actual response efficiency of the request on the premise of improving the concurrency capability; since the official recommendation is to use it, we can report it with a try attitude.
3) For high-frequency single file reading, the performance of aio and directio may not be significantly improved, but it should not reduce performance.