Distributed current limiting solution (gateway current limiting, redis+lua to achieve current limiting, nginx current limiting)
Current limiting algorithm
counter
The counter is relatively simple and rude. For example, we limit the number of requests that can pass in 1s. The idea is to start timing from the first request. In the next 1s, the number of requests for each request will be +1, which exceeds the maximum request. The count request will be rejected, and the count will be cleared after the end of 1s, and the count will be restarted.
This method has a big drawback: for example, the maximum number of requests has been passed in the first 10ms, then the following 990ms requests can only be rejected. This phenomenon is called "spur phenomenon".
Leaky bucket algorithm
That is, the speed of the water coming out of the bottom of the bucket is constant, and the speed of the water entering may vary, but when the amount of water inflow is greater than the amount of water coming out, the water will be packed in the bucket and will not be directly discarded; but the bucket also has a capacity limit. After the bucket is filled with water, the overflow part will still be discarded.
Algorithm implementation: You can prepare a queue to store requests that cannot be processed temporarily, and then periodically obtain requests from the queue for execution through a thread pool.
Token bucket algorithm
The token bucket is a place where access credentials are produced. The production speed is constant. When users access, they can get the credentials when there are credentials in the bucket.
Implementation scheme:
Guava RateLimiter current limit:
Guava RateLimiter is a current limit provided by Google, which is based on the token bucket algorithm and is more suitable for single-instance systems.
Specific realization of current limiting scheme
Gateway current limit
The RequestRateLimiterGatewayFilterFactory class is provided in Spring Cloud Gateway, which is implemented based on the token bucket. It has built-in RedisReteLimiter, which relies on Redis to store current limit configuration and statistics. We can also inherit
org.springframework.cloud.gateway.filter.ratelimit.AbstractRateLimiter or implement
org.springframework.cloud.gateway.filter.ratelimit.RateLimiter interface To implement your own RateLimiter.
The specific implementation is as follows:
-
First introduce dependencies in the gateway service
-
The introduced configuration is as follows
-
Current limitation based on three dimensions can be realized:
The configuration shown above is that the number of tokens generated per second is 1. When our request speed is greater than this value, a status code of 429 will be returned.
At this point, we have realized the current limit at the gateway level.
redis + lua
-
Introduce lua scripts into the service
-
Quote lua
-
Determine whether current limiting related code is needed
-
Above we can see that under normal circumstances, only 3 tokens are generated within 10s, let’s see the effect
It can be seen that only the first three requests are passed within 10 seconds, and the others are rejected. After 10s, there will be three more requests that can pass, which is fully in line with our expected effect.
Nginx current limit
Limit access frequency:
Nginx can limit the access frequency through the parameter ngx_http_limit_req_module module, which is achieved by the leaky bucket algorithm used; you can limit the request processing frequency of a single ip through the limit_req_zone command and the limit_req command.
Limiting the number of connections: The
ngx_http_limit_conn_module module provides the function of the number of concurrent connections, which can be configured using the limit_conn_zone command and limit_conn, which is also implemented based on the leaky bucket algorithm.
In addition to writing articles, the author maintains an open source project behind the technology, so students who have source code needs can pay attention to Xia Mumu, and it is expected to start opening during the Chinese New Year this year.