Getting Started: QUIC Networking and Encryption in NGINX

Original author: Robert Haynes - F5 Technical Marketing Manager

Original link: Getting Started: QUIC Network Connections and Encryption in NGINX

Reprint source: NGINX official website

The only official Chinese community of NGINX, all at nginx.org.cn

Four years ago , QUIC and HTTP/3 were first mentioned on the NGINX blog. Now that the QUIC implementation is finally about to be merged into the mainline version of NGINX open source, we, like everyone, are eagerly awaiting it. Given that the entire waiting process is relatively long, it's understandable if you haven't studied QUIC carefully before.

But now, as a developer or webmaster, it's important for you to understand how QUIC moves some of the specific operations of network connections from the operating system to NGINX (and all HTTP applications). Even if networking isn't your job, using QUIC means that networking is, at least to some extent, your job.

This article takes an in-depth look at the key networking and encryption concepts involved with QUIC, and we've simplified some details and omitted unnecessary information for the sake of clarity. While some details may be overlooked in doing so, our goal is to provide you with enough information to use QUIC effectively in your own environment, or at least to help you get started.

If you don't know anything about QUIC, it is recommended that you read our previous related blog posts first .

For a more detailed and comprehensive understanding of QUIC, it is highly recommended that you read the " Manageability of QUIC Transport Protocol " document carefully compiled by the IETC QUIC Working Group and other materials linked in this article.

Why should you care about QUIC network connections and encryption?

Currently, most users don't need to know the fine details of the network connection between the client and NGINX. After all, in HTTP/1.x and HTTP/2, the operating system was responsible for setting up the Transmission Control Protocol (TCP) connection between the client and NGINX. NGINX just uses the established connection.

But in QUIC, the responsibility for creating, validating, and managing connections is transferred from the underlying operating system to NGINX. Instead of receiving established TCP connections, NGINX now receives User Datagram Protocol (UDP) datagram streams and parses them into client connections and data streams. NGINX now also handles packet loss, reconnection, and congestion control.

Additionally, QUIC combines connection initiation, version negotiation, and encryption key exchange into a single connection establishment operation. Although TLS encryption is handled roughly the same for QUIC+HTTP/3 and TCP+HTTP/1+2, there can be significant differences for downstream devices such as Layer 4 load balancers, firewalls, and security appliances.

Ultimately, the combined effect of these changes is to improve the security, speed, and reliability of the user experience without much changing the configuration or operation of NGINX. However, NGINX administrators need to have at least some basic knowledge of QUIC and NGINX, if only to minimize the mean time to failure investigation when problems arise .

(It is worth mentioning that although this article focuses on HTTP operations because HTTP/3 requires the use of QUIC, QUIC can also be used with other protocols. A typical example is DNS over QUIC - defined in RFC 9250 as " based on private QUIC Connected DNS ".)

Let's dive into some QUIC network connection details.

Comparison of TCP and UDP

QUIC makes significant adjustments to the underlying network protocols used to transmit HTTP application data between clients and servers.

As mentioned earlier, TCP is a protocol used to transport HTTP web application data and is designed to reliably transport data over IP networks. Not only does it have well-defined, easy-to-understand mechanisms for establishing connections and acknowledging data receipt, but it also employs a variety of algorithms and techniques to manage packet loss and latency issues common on unreliable and congested networks.

While TCP ensures transmission reliability, there are tradeoffs in performance and latency. Additionally, data encryption is not built into TCP and must be implemented separately. Improving or extending TCP is also difficult given changing HTTP traffic patterns because TCP processing is performed in the Linux kernel and any changes must be carefully designed and tested to avoid unintended effects on the performance and stability of the entire system. .

Another problem is that in many cases, HTTP traffic between clients and servers flows through multiple TCP processing devices, such as firewalls or load balancers (collectively called "middleboxes"), and middleboxes implement changes in the TCP standard. It can be slow.

Therefore, QUIC uses UDP as the transport protocol. UDP can transmit data over IP networks like TCP
, but it deliberately abandons connection establishment and reliable delivery functions. This simplicity makes it suitable for a large number of applications that focus more on efficiency and speed than reliability.

However, for most web applications, reliable data transmission is crucial. Since the underlying UDP transport layer cannot provide reliable data transfer, these functions need to be provided by QUIC (or the application itself). Fortunately, QUIC has advantages over TCP in this regard:

QUIC processing is performed in Linux user space, where problems with one specific operation pose less risk to the entire system. This facilitates rapid development of new features.
The above-mentioned "middleboxes" usually only perform minimal processing of UDP traffic and therefore do not limit enhancements to the QUIC protocol.

Brief analysis of QUIC network

A QUIC data stream is a logical object containing an HTTP/3 request or response (or any other application data). For transport between network endpoints, they are encapsulated within multiple logical layers as shown in the figure.

Diagram showing components of a QUIC stream: a UDP datagram containing a header and multiple QUIC packets; the components in a QUIC packet (a header and frames); the components in a QUIC header; the components in a frame

Figure 1. QUIC data flow diagram

From outside to inside, there are the following logical layers and objects:

UDP datagram – Contains a request header specifying the source and destination ports (as well as length and checksum data), followed by one or more QUIC packets. A datagram is a unit of information transmitted over a network from a client to a server.
QUIC packet – Contains a QUIC request header and one or more QUIC data frames.
QUIC request header – Contains metadata about the packet. There are two types of request headers:

Long request header, used when establishing a connection.
Short request header, used after the connection is established. It contains (among other data) the connection ID, packet number, and keyphrase (used to track which keys were used to encrypt packets to support key rotation). Packet numbers are unique (and increasing) for a specific connection and keyphrase.

Data Frame – Contains type, stream ID, offset and streaming data. Streaming data is spread across multiple data frames, but can be combined using connection IDs, data flow IDs, and offsets to render chunks of data in the correct order.
Data flow – A unidirectional or bidirectional flow of data within a single QUIC connection. Each QUIC connection can support multiple independent data streams, each with its own data stream ID. If a QUIC packet containing some data flows is lost, the progress of the data flows that are not among the lost packets is not affected (this is the key to avoiding the HTTP/2 head-of-line blocking problem.) The data flow can be bidirectional and controlled by Arbitrary endpoint creation.

Connection established

Establish a TCP connection via the familiar SYN/SYN-ACK/ACK three-way handshake:

Diagram showing the three messages exchanged between client and server in the handshake to establish a TCP connection

Figure 2. Three-way handshake to establish TCP connection

Establishing a QUIC connection follows similar steps, but is more efficient. It also includes address verification in the connection settings as part of the cryptographic handshake. Address validation protects against traffic amplification attacks, preventing attackers from sending packets to the server that contain forged source address information for the intended victim. The attacker wants the server to generate more or larger packets to the victim than the attacker can generate itself, causing a surge in traffic that exceeds the victim's ability to handle. (See RFC 9000 Section 8 " QUIC: Multiplexing and Secure Transport over UDP " for details.)

During the connection establishment process, the client and server provide independent connection IDs, which are encoded in the QUIC request header, providing a simple connection identification that is independent of the client's source IP address.

However, because the initial establishment of a QUIC connection also requires a TLS encryption key exchange operation, the computational workload for the server is higher than the simple SYN-ACK response generated during the establishment of a TCP connection. This also provides a potential vector for distributed denial of service (DDoS) attacks, as the client IP address is not verified before the key exchange operation occurs.

However, you can configure NGINX to verify the client IP address before starting complex encryption operations by quic_retry setting the directive to . onIn this case, NGINX sends the client a retry packet containing a token , which the client must add to the connection setup packet.

Diagram showing the handshake for establishing a QUIC connection, without and with a replay packet

Figure 3. QUIC connection setup with and without retry packets

This mechanism is somewhat similar to TCP's three-way handshake. The key is to make sure that the client owns the source IP address it provided. Without this check, QUIC servers such as NGINX may be vulnerable to DoS attacks using spoofed source IP addresses. (Another QUIC mechanism that mitigates this type of attack is to require that all initial connection packets contain at least 1,200 bytes, increasing the cost of sending these packets.)

Additionally, retry packets encode connection details into the connection ID they send to the client, mitigating SYN attacks similar to TCP flooding attacks (where server resources are exhausted by a large number of open but incomplete handshakes stored in memory ); this also has the advantage that there is no need to preserve server-side information because the connection information can be restored using the connection ID and token subsequently submitted by the client. This technology is similar to TCP SYN cookies. In addition, QUIC servers such as NGINX can provide an expiring token that can be used for future connections from the client, thus speeding up connection recovery.

Using a connection ID makes the connection independent of the underlying transport layer, so network connection changes do not cause the connection to be interrupted. This is discussed in detail in the " Smoothly manage client IP address changes " section below.

Packet loss detection

After the connection is established (and encryption is enabled, as detailed below), HTTP requests and responses can flow back and forth between the client and NGINX. UDP datagrams are constantly being sent and received. However, many factors can cause some of these datagrams to be lost or delayed.

TCP has sophisticated mechanisms to acknowledge packet delivery, detect packet loss or delay, and manage retransmissions of lost packets to deliver correctly sequenced, complete data to the application layer. UDP lacks this mechanism, so congestion control and packet loss detection are implemented at the QUIC layer.

Both clients and servers send explicit acknowledgment messages for every QUIC packet they receive (although packets containing only low-priority data frames do not receive immediate acknowledgment).

If a packet containing a data frame that requires reliable transmission is not acknowledged after the set timeout, it is considered lost.

The timeout depends on the contents of the packet. For example, for packets required to establish encryption and set up a connection, the timeout is shorter because this is related to QUIC handshake performance.

When a packet is considered lost, the lost data frame is retransmitted with a new packet with a new sequence number.
The packet receiver uses the stream ID and offset on the packet to assemble the transmitted data in the correct order. The packet number only determines the order in which the packets are sent, not the order in which the packets are combined.
Because the data combination at the receiving end is independent of the order of transmission, the loss or delay of a certain data packet only affects the individual data streams contained in it, not all data streams in the connection. This eliminates the head-of-line blocking problem affecting HTTP/ 1.x and HTTP/2 because the data flow does not belong to the transport layer.

A complete description of packet loss detection is beyond the scope of this primer. Please refer to RFC 9002 " QUIC Packet Loss Detection and Congestion Control " for details on the timeout determination mechanism and the amount of unacknowledged data allowed in transmission.

Smoothly manage client IP address changes

The client IP address (referred to as the "source IP address" in the context of an app session) may change during the session, for example when the VPN or gateway changes its public address or the smartphone user leaves the WiFi coverage area, forcing a switch to the cellular network. Additionally, network administrators often set timeouts for UDP traffic that are shorter than TCP connections, which increases the chance of Network Address Translation (NAT) rebinding.

QUIC provides two mechanisms to reduce possible interruptions: the client can actively notify the server that its address will change, and the server can smoothly handle unexpected changes in the client's address. Because the connection ID remains consistent throughout the translation process, unacknowledged data frames can be retransmitted to the new IP address.

Changing the source IP address during a QUIC session may impact downstream load balancers (or other Layer 4 networking components) because these load balancers (or other Layer 4 networking components) use the source IP address and port to determine which upstream server Specific UDP datagrams will be received. To ensure proper traffic management, providers of Layer 4 network equipment will need to update them to handle QUIC connection IDs. To learn more about the future of load balancing and QUIC, see the IETF draft " QUIC-LB: Generating routable QUIC connection IDs ."

encryption

In connection establishment , we mentioned that the initial QUIC handshake is more than just establishing the connection. Unlike TCP's TLS handshake, in UDP the exchange of keys and TLS 1.3 encryption parameters is also part of the initial connection. This feature eliminates multiple exchanges and enables zero round-trip time (0-RTT) when the client resumes its previous connection.

Diagram comparing the encryption handshakes for TCP+TLS/1.3 and QUIC

Figure 4. Comparison of encrypted handshake between TCP+TLS/1F.3 and QUIC

In addition to integrating the encrypted handshake into the connection establishment process, QUIC also encrypts more metadata than TCP+TLS. The initial connection packet is encrypted even before the keys are exchanged; although an eavesdropper can still crack the key, it will take more effort than with unencrypted packets. This helps better protect data, such as Server Name Indicators (SNI), which attackers and network inspectors focus on. Figure 5 shows that QUIC encrypts more sensitive metadata (red) compared to TCP+TLS.

Diagram showing how much more data is encrypted in a QUIC datagram than in a TCP packet for HTTP/1 and HTTP/2

Figure 5. Compared with TCP+TLS, QUIC encrypts more sensitive metadata

All data in the QUIC payload is encrypted using TLS 1.3. This has two advantages: it prohibits the use of outdated, vulnerable cipher suites and hashing algorithms, and it requires the use of a forward secrecy (FS) key exchange mechanism. Forward secrecy prevents an attacker from decrypting the data, even if the attacker captures the private key and a copy of the traffic.

Low RTT and zero RTT connections reduce latency

Reducing the number of communication round-trips that must be completed between the client and server before any application data is transmitted can improve application performance, especially on networks with higher latency.

TLS 1.3 establishes an encrypted connection with a single round trip and enables zero round trips for connection recovery, but for TCP this means that a handshake must occur before the TLS Client Hello.

By combining encryption operations with connection setup, QUIC enables true 0-RTT connection reestablishment, where the client can send a request in the first QUIC packet, which eliminates the initial connection establishment time before the first request. round trip, thereby reducing latency.

Diagram showing that TCP+TLS requires 6 messages to re-establish a connection, and QUIC only 3

Figure 6. Comparison of the messages required to reestablish a connection using TCP+TLS and QUIC respectively

In this example, the client sends an HTTP request that is encrypted with the parameters used in the previous connection and, to verify the address, includes a token provided by the server in the previous connection.

Unfortunately, 0-RTT connection recovery does not provide forward secrecy, so the initial client request is not securely encrypted like other traffic in the exchange. All requests and responses except the first are protected by forward secrecy. But the problem is that the initial request is also highly susceptible to a replay attack , where an attacker can capture the initial request and replay it to the server multiple times.

For many apps and websites, the performance gains from 0-RTT connection recovery outweigh these potential vulnerability risks, but this is a trade-off you need to make yourself.

NGINX disables this feature by default. To enable this feature, ssl_early_data set the directive to on.

Convert from HTTP/1.1 to HTTP/3 using Alt-Svc request header

Almost all clients (especially browsers) establish the initial connection via TCP/TLS. If the server supports QUIC+HTTP/3, an Alt-Svch3 HTTP/1.1 response Alt-Svc containing parameters in the request header will be returned. The client can then choose whether to use QUIC+HTTP/3 or continue to use an earlier version of HTTP. (It is worth mentioning that request headers [see RFC 7838 for definition ] predate QUIC and can also be used for other purposes.)

Diagram showing how the server uses the Alt-Svc header to signal to a client that it supports HTTP/3

Figure 7. How to use the Alt-Svc request header to convert the connection from HTTP/1.1 to HTTP/3

Alt-Svc The request header informs the client that an alternate host, protocol, or port (or a combination thereof) provides the same service. Additionally, clients are informed of a conservative estimate of how long the service will be available.

Example:

WeChat screenshot_20231107110557.png

Although not required, in most cases the server will be configured to respond to QUIC connections on the same port as TCP+TLS.

To configure NGINX to include Alt-Svc request headers, use the directive. In this case, the variable means that NGINX accepts QUIC connections on the port where the client sends TCP+TLS requests, where 86,400 represents 24 hours:add_header$server_port

add_header Alt-Svc 'h3=":$server_port"; ma=86400';

Conclusion

This article provides an introduction to QUIC and hopes to give you an overview of QUIC's key network connections and encryption operations.

For a more complete understanding of how to configure NGINX for QUIC + HTTP/3, read our blog post NGINX QUIC+HTTP/3 Preview Binary Packages Now Available or watch our webinar Trying NGINX for Your Hands with QUIC+HTTP/3》.

To learn more about all NGINX directives for QUIC+HTTP/3 and complete instructions for installing prebuilt binaries or building from source, visit the NGINX QUIC website .

The only official Chinese community of NGINX, all at nginx.org.cn

More NGINX-related technical information, interactive Q&A, series of courses, and event resources: Open Source Community Official Website | WeChat Official Account