This article mainly introduces you to relevant information about the commonly used algorithms and characteristics of Spring Boot interface current limiting. The text introduced in the text is very detailed, and has a certain reference learning value for everyone's study or work. Friends who need it are below Let's learn together with the editor. It
is very important to control traffic in a high-concurrency system. When a huge amount of traffic is directly requested to our server, it may not take long for the interface to be unavailable, or even if it is not processed. Cause the entire application to be unavailable.
So what is current limiting? As the name suggests, current limiting is to limit the traffic, just like your broadband packet of 1G of traffic, it will be gone when it is used up. By limiting the current, we can well control the qps of the system, so as to achieve the purpose of protecting the system. This article will introduce the commonly used current limiting algorithms and their respective characteristics.
Algorithm introduction
counter method
The counter method is the simplest and the easiest to implement in the current limiting algorithm. For example, we stipulate that for the A interface, our number of visits per minute cannot exceed 100. Then we can do this: at the beginning, we can set a counter counter, and every time a request comes, the counter will increase by 1, if the value of counter is greater than 100 and the interval between the request and the first request is still Within 1 minute, it means that there are too many requests; if the interval between the request and the first request is greater than 1 minute, and the value of counter is still within the current limit, then reset the counter. The schematic diagram of the specific algorithm is as follows :
The specific pseudo code is as follows:
Although this algorithm is simple, it has a very fatal problem, that is, the critical problem. Let's look at the picture below:
As we can see from the above figure, suppose there is a malicious user who sent 100 requests in an instant at 0:59 and 100 requests in an instant at 1:00. In fact, this user is within 1 second. 200 requests were sent in an instant. What we just stipulated is a maximum of 100 requests per minute, that is, a maximum of 1.7 requests per second. By bursting requests at the reset node of the time window, users can instantly exceed our rate limit. It is possible for users to pass this loophole in the algorithm and instantly overwhelm our application.
Smart friends may have noticed that the problem just now is actually because the accuracy of our statistics is too low. So how to deal with this problem well? In other words, how to reduce the impact of critical issues? We can look at the sliding window algorithm below .
Sliding window
Sliding window, also known as rolling window. To solve this problem, we introduced a sliding window algorithm. If you have learned the TCP network protocol, you must be familiar with the term sliding window. The following picture explains the sliding window algorithm well:
In the above figure, the entire red rectangular box represents a time window. In our example, a time window is one minute. Then we divide the time window. For example, in the figure, we divide the sliding window into 6 grids, so each grid represents 10 seconds. Every 10 seconds, our time window will slide one space to the right. Each grid has its own independent counter. For example, when a request arrives at 0:35 seconds, the counter corresponding to 0:30~0:39 will increase by 1.
So how does the sliding window solve the critical problem just now? We can see the above picture, 100 requests arriving at 0:59 will fall in the gray grid, and requests arriving at 1:00 will fall in the orange grid. When the time reaches 1:00, our window will move one square to the right, then the total number of requests in the time window at this time is 200, which exceeds the limit of 100, so it can be detected that the current limit has been triggered at this time .
Let me review the counter algorithm just now, we can find that the counter algorithm is actually a sliding window algorithm. But it does not further divide the time window, which is 60s.
It can be seen that the more grids of the sliding window are divided, the smoother the scrolling of the sliding window, and the more accurate the current limit statistics.
Leaky bucket algorithm
Leaky bucket algorithm, also known as leaky bucket. In order to understand the leaky bucket algorithm, let's look at the schematic diagram of the algorithm:
Token bucket algorithm
Token bucket algorithm, also known as token bucket. In order to understand the algorithm, let's look at the schematic diagram of the algorithm:
From the figure, we can see that the token bucket algorithm is slightly more complicated than the leaky bucket algorithm. First, we have a fixed-capacity bucket in which tokens are stored. The bucket is empty at the beginning, and the token is filled into the bucket at a fixed rate r until the capacity of the bucket is reached, and the excess tokens will be discarded. Whenever a request comes, it will try to remove a token from the bucket. If there is no token, the request cannot be passed.
The specific pseudo-code implementation is as follows:
RateLimiter implementation
For the code implementation of the token bucket, you can directly use the RateLimiter in the Guava package.
Related variants
A careful study of algorithms, we will find our default token is removed from the bucket does not require time-consuming. If a delay time is set for removing tokens, then the idea of leaky bucket algorithm is actually adopted. The SmoothWarmingUp class under Google's guava library uses this idea.
Critical problem
Let us consider the critical problem scenario again. At 0:59 seconds, because the bucket is full of 100 tokens, these 100 requests can be passed instantly. However, because the tokens are filled at a lower rate, at 1:00, the number of tokens in the bucket cannot reach 100, so it is impossible for 100 more requests to pass at this time. Therefore, the token bucket algorithm can solve the critical problem well. The figure below compares the rate changes of the counter (left) and the token bucket algorithm (right) at the critical point. We can see that although the token bucket algorithm allows burst rates, the next burst rate cannot occur until there are enough tokens in the bucket:
Summary
counter VS sliding window
The counter algorithm is the simplest algorithm and can be seen as a low-precision implementation of the sliding window. Because the sliding window needs to store multiple counters (one for each grid), the sliding window needs more storage space in implementation. In other words, if the accuracy of the sliding window is higher, the storage space required is greater.
Leaky Bucket Algorithm VS Token Bucket Algorithm
The most obvious difference between the leaky bucket algorithm and the token bucket algorithm is that the token bucket algorithm allows a certain degree of burst of traffic. Because of the default token bucket algorithm, it does not take time to remove the token, that is, if there are 100 tokens in the bucket, then 100 requests can be allowed to pass in an instant.
The token bucket algorithm is widely used in the industry because it is simple to implement, allows bursts of certain traffic, and is user-friendly. Of course, we need a specific analysis of the specific situation, there is only the most suitable algorithm, there is no optimal algorithm.
So far, this article on the common algorithms and characteristics of Spring Boot interface current limiting is introduced. For more related Spring Boot interface current limiting algorithms, please follow the editor or add wx notes (csdn) for more information. Hope everyone Support me a lot in the future!