foreword
This article about Nginx performance optimization is the result of my research and research. It has not been used in the actual production environment. If you want to use it in practice, please use it after careful testing.
Nginx performance optimization, mainly to reduce disk io.
The request header, request body, and response body are all operated in the buffer.
The reading of file information
reduces network io.
gzip compression. Front-end resources can also be gzip-compressed in advance, so that there is no need to compress them when requesting, reducing the loss of cpu.
Strong caching. Reduce requests for backend static resources.
The http link is released as soon as possible to reduce the accumulation of requests.
Linux kernel optimization. This part is mainly to consult information and add your own understanding. The content comes from "In-depth understanding of Nginx module development and architecture design".
After adjusting the account parameters, you can use ab and jmeter for pressure testing, and tune according to the actual effect.
Nginx must be optimized for performance in a safe situation. Otherwise, tcp attacks can hang up your server, and being able to use it is the kingly way. Secure access first, performance second.
After tuning, you must remember to limit the current.
The next series is going to write about Mysql, and it is already in preparation.
linux kernel optimization
You can modify /etc/sysctl.conf to modify kernel parameters. Carefully optimize this part
View tcp related system parameters
sysctl -a | grep 'net.ipv4.tcp' | grep -v grep
fs.file-max = 999999
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_keepalive_time = 15
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_max_tw_buckets = 5000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_rmem = 4096 32768 262144
net.ipv4.tcp_wmem = 4096 32768 262144
net.ipv4.tcp_max_orphans = 262144
net.core.netdev_max_backlog = 262144
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
net.core.somaxconn = 262144
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog=262144
Execute the following command after modification to make the configuration take effect.
sysctl -p
The meaning of the above parameters is explained as follows:
fs.file-max：999999
This parameter indicates the maximum number of handles that a process (such as a worker process) can open at the same time. This parameter directly limits the maximum number of concurrent connections and needs to be configured according to the actual situation.
net.ipv4.tcp_tw_reuse：
This parameter is set to 1, which means that the socket in the TIME-WAIT state is allowed to be reused for new TCP connections, which is very meaningful for the server, because there will always be a large number of connections in the TIME-WAIT state on the server.
net.ipv4.tcp_keepalive_time：
This parameter indicates how often TCP sends keepalive messages when keepalive is enabled. The default is 2 hours, if you set it smaller, you can clean up invalid connections faster.
net.ipv4.tcp_fin_timeout：
This parameter indicates the maximum time for the socket to remain in the FIN-WAIT-2 state when the server actively closes the connection.
net.ipv4.tcp_max_tw_buckets：
This parameter indicates the maximum number of TIME_WAIT sockets allowed by the operating system. If this number is exceeded, the TIME_WAIT socket will be cleared immediately and a warning message will be printed. The parameter defaults to 180000, too many TIME_WAIT sockets will slow down the web server.
net.ipv4.ip_local_port_range：
This parameter defines the value range of the local (not including the remote end of the connection) port in UDP and TCP connections.
net.ipv4.tcp_rmem：
This parameter defines the minimum, default, and maximum values of the TCP receive buffer (for the TCP receive sliding window).
net.ipv4.tcp_wmem：
This parameter defines the minimum, default, and maximum values of the TCP send buffer (for the TCP send sliding window).
net.ipv4.tcp_max_orphans
option is used to record the maximum number of connection requests that have not received confirmation from the client. The default value of this parameter is 1024 for systems with 128MB of memory, and 128 for systems with small memory.
net.core.netdev_max_backlog：
When the network card receives data packets faster than the kernel can process them, there will be a queue to hold these data packets. This parameter represents the maximum value of the queue.
net.core.net.core.rmem_default：
This parameter indicates the default size of the kernel socket receive buffer.
net.core.wmem_default：
This parameter indicates the default size of the kernel socket send buffer.
net.core.rmem_max：
This parameter indicates the maximum size of the kernel socket receive buffer.
net.core.wmem_max：
This parameter indicates the maximum size of the kernel socket send buffer.
The size of the sliding window and the socket buffer will affect the number of concurrent connections to a certain extent.
Each TCP connection consumes memory to maintain the TCP sliding window, which shrinks or expands according to the processing speed of the server.
The setting of the parameter wmem_max needs to be determined by balancing the total size of physical memory and the maximum number of connections processed by Nginx concurrently (determined by the worker_processes and worker_connections parameters in nginx.conf).
Of course, it is not appropriate to reduce the size of the sliding window just to increase the concurrency so that the server does not have the Out Of Memory problem, because too small a sliding window will affect the transmission speed of large amounts of data.
The settings of the four parameters rmem_default, wmem_default, rmem_max, and wmem_max need to be considered comprehensively according to our business characteristics and actual hardware costs.
net.core.somaxconn
option indicates the maximum number of packets allowed to be sent to the queue when each network interface is receiving packets faster than the kernel can process them.
net.ipv4.tcp_syncookies：
Set to 1. This parameter has nothing to do with performance and is used to solve the SYN attack of TCP.
net.ipv4.tcp_max_syn_backlog：
This parameter indicates the maximum length of the receiving SYN request queue during the establishment phase of the TCP three-way handshake. The default is 1024. Setting it larger can prevent Linux from losing the connection request initiated by the client when Nginx is too busy to accept a new connection.
Memory and disk data optimization
client_header_buffer_size 1k;
accessing 1k as a static resource is enough.
The buffer size of the client's request header is 1k by default. When requesting an interface, set an integer multiple of 4k according to the request header data.
4k is the system memory page size, command getconf PAGESIZE memory page size
If client_body_in_single_buffer
is set to on, all http packets will be written to the memory buffer. If the packet body exceeds the size of client_body_buffer_size, it will still be written to the disk file.
client_body_buffer_size
x64 defaults to 16K. Define the memory buffer size of the accepted request body, the request is first written to the cache area, and then written to the temporary file.
client_max_body_size 100m;
Set the maximum allowable size of the client request body, the default is 1m. What is checked is Content-Length. Set to 0 to not check. For specific location configuration. To prevent attacks from being requested, set it where it needs to be increased, and do not set it globally.
sendflie on;
file content reading reduces the copying from the kernel state to the user state, directly from the kernel state to the network card device, which improves the sending efficiency.
open_file_cache
caches the storage information of the file. max indicates the maximum storage quantity, if it exceeds this quantity, it will be eliminated by LRU
inactive specifies the time period, elements that have not been accessed during this time period will be eliminated
open_file_cache max=65535 inactive=20s;
open_file_cache_min_uses
defaults to 1. Used in conjunction with inactive of open_file_cache, if the number of visits exceeds the number of times specified by open_file_cache_min_uses within the time specified by inactive, the cache will not be eliminated.
Increase the size of the file handle to limit
Linux. Everything is a file, but the number of files opened by the process will be limited. The number of file handles can be limited for both users and processes.
File handle, which can be set for users and processes
Global settings /etc/security/limits.conf
* hard nofile 65535
* soft nofile 65535
root hard nofile 65535
nginx process configuration
worker_rlimit_nofile 20480;
log optimization
Log operations can be appropriately reduced. For example, log records for accessing static data, if you feel that it is not important, you can cancel the log records. In this way, when requesting resources, the disk io of the log will be reduced.
# 关闭日志
access_log off;
# 禁用文件未找到的错误到日志中去
log_not_found off;
Reverse proxy optimization
If you use nginx as a proxy server, also reduce disk io reads.
proxy_buffering on;
proxy_buffer_size 4k;
proxy_buffers 16 64k;
proxy_busy_buffers_size 128k;
proxy_temp_file_write_size 128k;
proxy_set_header Host $http_host;
proxy_set_header X-REAL-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Nginx optimization configuration
# 配置 worker 进程所属用户,用户组
user nginx nginx;
# 配置 worker 进程数量，为避免 cpu 切换损耗，配置和系统内核数一样即可,或者 auto
worker_processes auto;
# 配置 cpu 亲和,auto 代表自动绑定
worker_cpu_affinity auto;
# nginx 进程打开文件描述符数目，此值覆盖 ulimit -n 的值。
worker_rlimit_nofile 65535;
events {
# 用这个模型来高效处理异步事件
use epoll;
# 设置为 on worker 进程轮流接受新链接,官方推荐设置为 off.高负载的情况下设置为 on.
accept_mutex on;
# worker进程是否同时接受连接所有新请求。默认为off，表示一次只接受一个新的请求。官方推荐 off
multi_accept on;
# 配置 一个 woker 进程处理的连接数
worker_connections 65535;
}
http {
# 关闭日志
access_log off;
# 隐藏响应头中的有关操作系统和web server（Nginx）版本号的信息，这样对于安全性是有好处的。
server_tokens off;
sendfile on;
# 设置为非零值时，可限制单个 sendfile() 调用时传输的数据量。如果没有限制，一个快速 连接可能会完全占用工作进程。
sendfile_max_chunk 5m;
# tcp_nopush 和 tcp_nodeny 可以一起生效
# 等数据包累计到一定大小发送，启用 sendfile 生效
tcp_nopush on;
# 该选项仅在连接转换到 keep-alive ，长连接状态时启用。让 tcp 尽快发包。
tcp_nodelay on;
# 为了尽快释放连接,可以设置小点. 15 至 30
keepalive_timeout 15;
# 客户端请求头部的缓冲区大小，默认 1k，当请求接口的时候需要设置 4k 整数倍.内存设置为系统内存页大小，命令 getconf PAGESIZE 内存页大小
client_header_buffer_size 4k;
large_client_header_buffers 8 8k;
# 根据需求设置,接口请求可以设置大些
client_body_buffer_size 128k;
# 设置客户端请求体最大允许大小，默认 1m。检查的是 Content-Length。设置为 0 不检查。针对具体 location 配置
client_max_body_size 1m;
# 下面这个参数将为打开文件指定缓存，默认是没有启用的，max指定缓存数量，建议和打开文件数一致，
# inactive是指经过多长时间文件没被请求后删除缓存。
open_file_cache max=262140 inactive=20s;
# 下面这个是指多长时间检查一次缓存的有效信息。
open_file_cache_valid 30s;
# open_file_cache指令中的inactive参数时间内文件的最少访问次数，低于这个数,缓存清除
open_file_cache_min_uses 1;
open_file_cache_errors on;
reset_timedout_connection on;
client_body_timeout 10;
send_timeout 2;
# 限制每个 ip 的连接数
limit_conn_zone $binary_remote_addr zone=conn_limit_per_ip:10m;
# 限制每个 ip 每秒的请求数
limit_req_zone $binary_remote_addr zone=req_limit_per_ip:10m rate=10r/s;
gzip on;
# 在响应头中增加，Vary: Accept-Encoding
gzip_vary on;
# gzip压缩级别1-9，数字越大压缩效果越好，压缩时间也就越长CPU越高
gzip_comp_level 5;
# 申请内存时大小，如果源文件 9k，超过了 8K，那会申请 16*8K。
gzip_buffers 8 128k;
gzip_min_length 5K;
gzip_proxied any;
gzip_disable msie6;
gzip_http_version 1.1;
# 文本（js、text、css、xml、json）压缩比较好，图片已经进行过压缩，在压缩，效果不是很明显，还浪费 cpu
gzip_types text/plain text/css text/xml text/javascript application/javascript application/json application/xml+rss application/rss+xml application/atom+xml image/svg+xml;
# 安全相关 header
add_header X-Frame-Options "SAMEORIGIN" always;
add_header Feature-Policy "autoplay 'none'; camera 'none'" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "no-referrer-when-downgrade" always;
add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'" always;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
server {
listen 80 backlog=262144;
limit_conn conn_limit_per_ip 10;
limit_req zone=req_limit_per_ip burst=10 nodelay;
# assets, media
location ~* \.(?:css(\.map)?|js(\.map)?|jpe?g|png|gif|ico|cur|heic|webp|tiff?|mp3|m4a|aac|ogg|midi?|wav|mp4|mov|webm|mpe?g|avi|ogv|flv|wmv)$ {
# 强缓存，时间为一年，浏览器和 cdn 中间件可以缓存
add_header Cache-Control "max-age=31536000";
etag off;
access_log off;
# 禁用文件未找到的错误到日志中去
log_not_found off;
}
# svg, fonts
location ~* \.(?:svgz?|ttf|ttc|otf|eot|woff2?)$ {
# 强缓存，时间为一年，浏览器和 cdn 中间件可以缓存
add_header Cache-Control "max-age=31536000";
etag off;
access_log off;
# 禁用文件未找到的错误到日志中去
log_not_found off;
}
}
}
Reposted from https://www.zhihu.com/tardis/landing/m/360/art/129327121