nginx implements reverse proxy and load balancing
1. Reverse proxy
1.What is a reverse proxy?
The usual proxy server is only used to proxy connection requests from the internal network to the Internet. The client must specify a proxy server and send the http request that is to be sent directly to the Web server to the proxy server. The proxy server then sends the HTTP request to the Internet on the Internet. The server initiates a request and ultimately achieves the purpose of the client accessing the Internet (that is, forward proxy).
The reverse proxy (Reverse Proxy) method refers to using a proxy server to accept connection requests on the Internet, then forwarding the requests to the server on the internal network, and returning the results obtained from the server to the client requesting a connection on the Internet. , at this time the proxy server appears as a reverse proxy server to the outside world.
As shown below:
Nginx only forwards requests. There are multiple http servers in the background to provide services. The function of nginx is to forward the request to the subsequent server and decide who to forward the request to.
2. Load balancing
1. What is load balancing?
Load balancing is built on the existing network structure. It provides a cheap, effective and transparent method to expand the bandwidth of network devices and servers, increase throughput, enhance network data processing capabilities, and improve network flexibility and availability.
Load balancing, whose English name is Load Balance, means to allocate execution to multiple operating units, such as Web servers, FTP servers, enterprise key application servers and other mission-critical servers, so as to jointly complete work tasks.
Nginx load balancing is jointly implemented by the proxy module and the upstream module. Nginx forwards user requests to the upstream server group through the reverse proxy function of the proxy module. The upstream module forwards user requests to the upstream server group through the specified load balancing strategy and related parameter configurations. The request is forwarded to the target server. The upstream module can implement various protocols with Nginx's proxy command (proxy_pass), FastCGI protocol command (fastcgi_pass), uWSGI protocol command (uwsgi_pass), SCGI protocol command (scgi_pass), memcached command (memcached_pass) and gRPC protocol command (grpc_pass) Load balancing of end servers.
Nginx supports a variety of load balancing strategies, such as Round Robin, Consistent Hash, IP Hash, least_conn, etc. The default load balancing strategy of Nginx is the polling strategy, which does not require configuration instructions. The polling strategy can implement a manually assigned weighted polling strategy through the weight parameters of the server.
Load balancing technology is an application technology that distributes a large number of client requests to nodes in the cluster through specific strategies to achieve rapid response. When dealing with highly concurrent application requests, the computing power of a single node's application service is limited and cannot meet the client's response needs. Through load balancing technology, requests can be distributed to multiple nodes in the cluster, allowing multiple nodes to share the high concurrency. Compute the request and quickly complete the client's request response.
1. Polling
The Round Robin strategy is the default load balancing strategy in Nginx configuration. This strategy allocates client requests to back-end server nodes in turn, and allocates servers in the back-end cluster in turn. The polling strategy is absolutely balanced and simple to implement, but it will also affect the processing performance of the entire cluster due to the different processing capabilities of the back-end server.
1) Weighted polling
In Nginx's polling strategy, in order to avoid the impact of differences in server performance in the cluster on the performance of the entire cluster, a weight parameter is added to the polling strategy, allowing users to manually assign requests based on the performance of each server in the cluster. The quantity is allocated to different proxied servers in proportion to the weight.
2) Smooth polling
In the weighted polling strategy, client requests are allocated according to the high and low weights. If the high weights are allocated and then the low weights are allocated, it may happen that the high-weight servers are always busy and the pressure is relatively concentrated. Nginx uses a smooth polling algorithm to enable each server in the upstream server group to participate in the processing of client requests while the total weight distribution remains unchanged, effectively avoiding the centralized allocation of requests to high-end servers within a period of time. Weight server situation occurs.
2. Consistent hashing
Nginx's hash-enabled load balancing policy is set using the hash directive. The hash strategy method can calculate a hash value for the URL accessed by the client. For the same URL request, Nginx can assign it to the same backend server because of the same hash value. When the backend server is a cache server, the hit rate will be greatly improved and the access speed will be improved.
The advantage of consistent hashing is that similar requests from different clients can be sent to the same proxy server. When the proxy server is used as a cache server, the cache hit rate can be greatly improved.
The disadvantage of consistent hashing is that when the number of nodes in the upstream server group changes, all bindings will be recalculated by the hash value of the proxy server, affecting the binding relationship of the entire cluster and generating a large number of back-to-origin requests.
The calculation process of Nginx consistent hashing strategy in the configuration example is as follows.
First calculate the hash value based on $request_uri;
Through the dichotomy method, quickly select the largest virtual node hash value in the range of the hash value in the virtual node list;
By taking the remainder of the virtual node hash value and the total number of virtual node sets, the corresponding server is obtained as an alternative server;
Traverse the list of proxied servers in the polling policy, determine the effectiveness of the alternative servers, and select the server;
If the selection cannot be made after 20 cycles, the polling strategy will be used for selection.
3. IP hash
The IP Hash load balancing strategy calculates a hash value based on the client IP, and then allocates the request to the proxied server corresponding to the value. As long as the hash value remains unchanged and the proxied server is available, requests from the same client will always be distributed to the same proxied server. The IP hash load balancing strategy is often applied in scenarios where sessions are maintained.
When the HTTP client interacts with the server, because the HTTP protocol is stateless, any scenario that requires context logic must use the session persistence mechanism. The session persistence mechanism is to store session information identified by a unique Session ID through the client. , session information will be submitted to the server every time it interacts with the server, and the server will implement the logical association of the client's request context based on the session information.
Session information is usually stored in the memory of the proxied server. If the load balancer distributes the client's session request to other proxied servers, the session logic will be interrupted due to invalid session information. Therefore, in order to ensure that the session is not interrupted, load balancing is required to always send session requests from the same client to the same proxied server, and effective delivery of session information is achieved through session persistence.
The calculation process of Nginx ’s IP hash policy in the configuration example is as follows:
In a multi-layer proxy scenario, please ensure that Nginx can currently obtain the real client source IP;
A hash value is first calculated based on the first three octets of the client's IPv4 address or the entire IPv6 address as the hash key;
Calculate the hash remainder based on the hash value and the total weight of the non-backup state servers in the configuration file;
The initial proxy server is selected according to the polling policy. If the hash remainder is greater than the weight of the initial proxy server, the list of proxy servers in the polling policy is traversed, otherwise the initial proxy server will be selected;
When traversing the list of proxied servers in the polling policy, the hash remainder is used to subtract the weight of the previous proxied server in the polling policy until the hash remainder is less than the weight of a proxied server. to be elected;
If the selection cannot be made after 20 cycles, the polling strategy will be used for selection.
4. Minimum connections
Under the default configuration, the polling algorithm distributes client requests evenly to each proxied server, and the load of each proxied server is roughly the same. The premise of this scenario is that the request processing capabilities of each proxied server are equal. If a server in the cluster takes a long time to process requests, the load on that server will also increase relatively. Under the least connection (least_conn) load balancing strategy, client requests will be assigned to the proxy server with the fewest active connections based on the weight of each server in the upstream server group, thereby effectively improving the usage of the proxy server with high processing performance. .
The calculation process of Nginx minimum connection strategy in the configuration example is as follows:
Traverse the list of proxied servers in the polling policy, compare the ratio of the number of active connections (conns) of each backend to its weight, and select the one with the smallest ratio to allocate client requests;
If server a was selected last time, the current request will choose between servers b and c;
Assume that the number of active connections of b is 100 and the number of active connections of c is 60. Then the ratio of b (conns/weight) is 50 and the ratio of c (conns/weight) is 60, so the current request will be assigned to b.
5. Random load algorithm
In the Nginx cluster environment, each Nginx performs load balancing based on its own knowledge of the upstream server. In this scenario, it is easy for multiple Nginx to allocate requests to the same proxy server at the same time. In this scenario, Known as herd behavior.
Nginx designs a random (Random) load algorithm based on the principle of Power of Two Choices. This algorithm enables Nginx to no longer use the inherent load balancing strategy to select the proxy server based on a one-sided understanding of the situation, but to randomly select two and make the final selection after comparison. The random load algorithm provides a parameter two. When this parameter is specified, Nginx will randomly select two servers taking into account the weight, and then select a server using the following methods.
The minimum number of connections, the configuration instruction is least_conn, the default configuration;
The minimum average response header time, the configuration directive is least_time=header, is only valid for commercial versions;
The minimum average time for complete requests. The configuration directive is least_time=last_byte. It is only valid for commercial versions.