Original author: Alan Murphy of F5

Original link:Performance tuning suggestions and techniques

Reprint source:NGINX Chinese official website

NGINX is the only official Chinese community, all at nginx.org.cn

Several NGINX team members contributed to this article, including Valentin Bartenev and Nick Shadrin.

Over the past few years, I have worked with partners for whom NGINX Plus performance was a primary concern. Our conversations often begin with the difficulty of meeting our publishedperformance benchmarks. Issues like this usually arise because they jump straight to a specific use case, such as using existing SSL keys or targeting very large file loads, and then find that NGINX Plus's performance is sub-par.

To some extent, this result is not surprising. I always explain to my partners that, as a software component, NGINX Plus is capable of running at near linear speeds on any available hardware when handling the most basic HTTP use cases. However, to achieve our published theoretical performance in a specific use case, NGINX Plus often requires certain adjustments in configuration, underlying operating system, and hardware setup.

In every use case to date, our partners have been able to achieve a very unique use case by simply configuring operating system components and hardware settings and adjusting how NGINX Plus interacts with those components based on the specific use case scenario. Let NGINX Plus achieve theoretical performance.

After years of review, I have compiled a list of suggestions and tips on how to adjust NGINX configuration, operating system, and hardware, hoping to help our partners and customers improve the performance of NGINX Open Source Edition and NGINX Plus in specific use cases. .

This article provides guidance on only a subset of configuration settings that may affect performance. It does not cover all aspects of NGINX performance optimization, nor is it necessarily suitable for changing every setting discussed below in your environment.

Note: This blog post has been revised since it was first published and will be reviewed further to ensure it is as complete as possible. Welcome to add your changes and additional suggestions in the comment area below, we will adopt them as appropriate.

Tuning workflow

For performance tuning issues with NGINX, I generally recommend the following workflow:

Testing the performance of NGINX Plus on the most common HTTP use cases possible will establish the correct testing baseline for your environment.
Be clear about your use case. For example, if your application requires large file uploads, or you want to handle larger length SSL keys, start by defining the end goal of your use case.
Configure NGINX Plus for your use cases and retest to determine how actual performance differs from theoretical performance in your environment.
Focus on the settings that best fit your use case and adjust one setting at a time. In other words, don't add a new NGINX directive and change a bunch of systemctl settings at the same time. You should start small, especially those features that are most relevant to your use case. For example, if high security is critical to your environment, start by adjusting the SSL key type and length.
If a change doesn't improve performance, change it back to the default settings. In the process of adjusting one by one, you will find that some related settings will have an impact on performance together. Then in future optimization, you can adjust these settings as a whole as needed.

It's important to note that each deployment environment is unique and has its own network and application performance requirements. We do not recommend changing some settings in a production environment. The effects of the configuration adjustments described below may vary significantly depending on the application type and network topology.

Given NGINX's strong foundation in the open source community, many people have contributed feedback and contributions over the years on how to improve performance. This article adds links to some external resources where appropriate, and introduces some specific performance tuning suggestions that people have made after actual production testing.

Adjust NGINX configuration

See the NGINX reference documentation for details on the allowed values, default settings, and setting ranges for each setting.

SSL

This section describes how to remove slow and unnecessary ciphers from OpenSSL and NGINX.

Since SSL performance is so important, it's often necessary to experiment with different key lengths and types in your environment. This can help you find a balance between "long key, high security" and "short key, high performance" while meeting specific security needs. Here's a simple test: switch from a more traditional RSA key to Elliptic Curve Cryptozoology (ECC), which uses a smaller key length and therefore has faster computation while achieving the same level of security. speed.

To quickly generate a self-signed ECC P-256 key for testing, run the following command:

# openssl ecparam -out ./nginx-ecc-p256.key -name prime256v1 -genkey
# openssl req -new -key ./nginx-ecc-p256.key -out ./nginx-ecc-p256-csr.pem -subj '/CN=localhost'
# openssl req -x509 -nodes -days 30 -key ./nginx-ecc-p256.key -in ./nginx-ecc-p256-csr.pem -out ./nginx-ecc-p256.pem

compression

Gzip parameters provide fine-grained control over how NGINX serves content, so setting them incorrectly can cause NGINX performance to degrade. Enabling gzip can save bandwidth and improve page loading speeds on slow connections. (The effects of enabling gzip in synthetic benchmarks run locally may differ from actual usage.) For maximum performance, you can try the following settings:

Enable gzip only for relevant content, such as text, JavaScript, and CSS files;
Do not increase the compression level as this will increase the consumption of CPU resources without the benefit of Matching throughput improvements;
Enable and disable gzip for different types and sizes of content to evaluate the effect of enabling compression;
More on fine-grained control of gzip For more information, see the NGINX gzip module reference documentation.

connection handling

Here are several tuning options related to connection handling. For information on the correct syntax and applicable configuration blocks (http, server, location), see the linked reference documentation.

accept_mutexoff – All worker processes are notified of new connections (default in NGINX 1.11.3 and later, NGINX Plus R10 and later). If this option is enabled, worker processes will take turns accepting new connections.
keepalive 128 – Enables keepalive connections from NGINX Plus to the upstream server, defining the maximum number of idle keepalive connections retained in the cache of each worker process. When this value is exceeded, the least recently used connection is closed. If keepalives are not set, more overhead is incurred and neither the connection nor the ephemeral port is effectively used.

For HTTP traffic, if you use this directive in an upstream block, you must also add the following directives to the configuration to ensure that they are applied in all location blocks of the upstream server group (you can place them in individual location blocks , parent server code block or http level): proxy_http_version 1.1 - NGINX Plus uses HTTP/1.1 to process proxy requests proxy_set_header Connection "" - NGINX Plus strips all Connection request headers from proxy requests

multi_accept off - A worker process only accepts one new connection at a time (default). If enabled, one worker process will receive all new connections at once. We recommend leaving the default value (off) unless you are sure there is a benefit to modifying it. Performance testing with default values helps better measure predictable results.
proxy_buffering on - NGINX Plus will receive responses from the proxy server as quickly as possible and buffer them (default). If disabled, NGINX Plus will synchronously deliver responses to the client as they are received, increasing the load on NGINX Plus. Disabling response buffering is only necessary for applications that require immediate access to the data stream.
listen 80 reuseport - Enable port sharding, which means creating a separate listening socket for each worker process (using the SO_REUSEPORT socket option). This allows the kernel to create separate listening sockets for different worker processes. Distributes received connections. For more information, see our blog post: Socket Sharding in NGINX Release 1.9.1.

log

Logging is an important tool for managing and auditing systems. Logging large amounts of data and storing large amounts of logs can strain system resources, so we recommend that you disable logging only in very specific circumstances or when troubleshooting performance.

access_log off - disable access logging.
access_log /path/to/access.log main buffer=16k - Enable the access log buffering function.

Many open source projects and commercial product manufacturers will provide centralized logging systems based on syslog protocol . This system may bring you convenient. But if you need metrics for NGINX and NGINX Plus servers that consolidate the information originally recorded in the logs, you can use NGINX Amplify .

Thread Pool

The thread pool consists of a task queue and multiple threads that process the queue. When a worker process needs to perform a potentially long operation, it does not handle the operation itself, but places the task in a queue in the thread pool so that any idle thread can pick it up and process it.

To enable thread pooling, add the aio threads directive. Note that how the thread pool is managed may be affected by other buffer-related configuration settings. For complete information on adjusting additional settings to support thread pools, seeour blog post.

CPU Affinity

CPU Affinity specifies which CPU core or cores NGINX Plus uses for each worker process (see the worker_cpu_affinity reference documentation for background information).

In most cases, we recommend using the default automatic argument to the worker_processes directive, which sets the number of worker processes to match the number of available CPU cores.

However, when NGINX Plus is run in a container environment such as Docker, system administrators may choose to allocate fewer kernel resources to the container than are available on the host. In this case, NGINX Plus detects the number of cores available on the host and rotates the worker processes between the cores actually available to the container. In this case, the value of worker_processes should be specified to be the number of cores actually available in the container to reduce the number of worker processes.

Test CPU affinity

Typically when testing, we recommend that you use traffic similar to your production traffic. But for basic testing, you can use a traffic generator like wrk, as described here.

Quickly load wrk sessions for NGINX Plus:

# wrk -t 1 -c 50 -d 20s http://localhost/1k.bin

If necessary, you can create a simple 1k.bin file for testing:

# dd if=/dev/zero of=1kb.bin bs=1024 count=1

Execute the top command in CPU monitoring mode (press 1 after top starts).

You can test linearity by iterating with different numbers of worker processes and affinity binding settings. This is an efficient way to set access restrictions to an appropriate portion of the available cores.

Selection suggestions

Here are some rough selection recommendations that are suitable for ordinary web services and load balancing. These values may not be suitable for VOD streaming or CDN application scenarios.

CPU

Allocate 1 CPU core for every 1-2 Gbps of unencrypted traffic.

Small (1–2 KB) responses and 1 response per connection increase CPU load.

RAM

Allocate 1GB RAM for operating system and other general needs.

The rest is allocated to NGINX Plus buffers, socket buffers, and virtual memory cache, giving a rough estimate of 1MB per connection.

See more details of selection suggestionsNGINX community official website

Magnetic I/O

Disk I/O is limited by I/O operations per second (IOPS).

Many features of NGINX Plus rely on disk I/O and IOPS, such as logging and caching.

NGINX’s only official Chinese community, all at nginx.org.cn

More technical information, interactive Q&A, series of courses, and event resources related to NGINX: Open source community official website | WeChat public account

Performance tuning suggestions and techniques