background
I once saw this sentence in the blog of a great god : When developing a high-concurrency system, there are three sharp tools to protect the system: cache, downgrade, and current limiting . So what is current limiting? As the name suggests, current limiting is to limit traffic, just like your broadband package is 1 GB of traffic, and it will be gone when it runs out. By limiting the current, we can control the qps of the system well, so as to achieve the purpose of protecting the system. This article will introduce the commonly used current limiting algorithms and their respective characteristics.
Algorithm introduction
counter method
The counter method is the simplest and easiest to implement in the current limiting algorithm. For example, we stipulate that for interface A, the number of visits we can make in one minute cannot exceed 100. Then we can do this: at the beginning, we can set a counter counter, and each time a request comes, the counter is incremented by 1, if the value of counter is greater than 100 and the interval between the request and the first request is still Within 1 minute, it means that there are too many requests; if the interval between the request and the first request is greater than 1 minute, and the value of counter is still within the current limit range, then reset the counter. The schematic diagram of the specific algorithm is as follows :
The specific pseudocode is as follows:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
public class CounterDemo {
public long timeStamp = getNowTime();
public int reqCount = 0;
public final int limit = 100; // maximum number of requests within the time window
public final long interval = 1000; // time window ms
public boolean grant() {
long now = getNowTime();
if (now < timeStamp + interval) {
// within the time window
reqCount++;
// Determine whether the current time window exceeds the maximum number of request control
return reqCount <= limit;
}
else {
timeStamp = now;
// reset after timeout
reqCount =
1;
return true;
}
}
}
|
Although this algorithm is simple, it has a very fatal problem, that is, the critical problem. Let's look at the following figure:
As we can see from the above figure, suppose there is a malicious user who instantly sends 100 requests at 0:59, and sends 100 requests instantly at 1:00, then in fact this user is within 1 second, 200 requests were sent in an instant. What we just stipulated is a maximum of 100 requests per minute, that is, a maximum of 1.7 requests per second. Users can instantly exceed our rate limit by making burst requests at the reset node of the time window. It is possible for users to overwhelm our application instantly through this loophole in the algorithm.
Smart friends may have seen that the problem just now is because the precision of our statistics is too low. So how to deal with this problem well? In other words, how to reduce the impact of critical issues? We can look at the sliding window algorithm below.
sliding window
Sliding window, also known as rolling window. To solve this problem, we introduce a sliding window algorithm. If you have learned the TCP network protocol, you must be familiar with the term sliding window. The following picture, a good explanation of the sliding window algorithm:
In the image above, the entire red rectangle represents a time window, which in our case is one minute. Then we divide the time window. For example, in the figure, we divide the sliding window into 6 grids, so each grid represents 10 seconds. Every 10 seconds, our time window slides one space to the right. Each grid has its own independent counter. For example, when a request arrives at 0:35 seconds, the counter corresponding to 0:30~0:39 will be incremented by 1.
So how does the sliding window solve the critical problem just now? We can look at the picture above, 100 requests arriving at 0:59 will fall in the gray grid, while requests arriving at 1:00 will fall in the orange grid. When the time reaches 1:00, our window will move one space to the right, then the total number of requests in the time window is 200, which exceeds the limit of 100, so it can be detected that the current limit is triggered at this time. .
Let me review the counter algorithm just now. We can find that the counter algorithm is actually a sliding window algorithm. It's just that it doesn't further divide the time window, so there is only 1 grid.
It can be seen that the more grids of the sliding window are divided, the smoother the scrolling of the sliding window will be, and the more accurate the current limiting statistics will be.
leaky bucket algorithm
Leaky bucket algorithm, also known as leaky bucket. To understand the leaky bucket algorithm, let's take a look at the schematic diagram of the algorithm on Wikipedia:
As we can see from the figure, the whole algorithm is actually very simple. First, we have a fixed capacity bucket with water flowing in and water flowing out. For incoming water, we cannot predict how much water will flow in, nor can we predict how fast the water will flow. But for outgoing water, the bucket can fix the rate at which the water goes out. Also, when the bucket is full, the excess water will overflow.
We replace the water in the algorithm with requests in real applications, and we can see that the leaky bucket algorithm inherently limits the speed of requests. When using the leaky bucket algorithm, we can guarantee that the interface will process requests at a constant rate. Therefore, the leaky bucket algorithm does not inherently have critical problems. The specific pseudo-code implementation is as follows:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
public class LeakyDemo {
public long timeStamp = getNowTime();
public int capacity; // the capacity of the bucket
public int rate; // speed of water leakage
public int water; // current water volume (current cumulative number of requests)
public boolean grant() {
long now = getNowTime();
water = max(
0, water - (now - timeStamp) * rate); // Execute water leakage first and calculate the remaining water amount
timeStamp = now;
if ((water + 1) < capacity) {
// Attempt to add water, and the water is not full
water +=
1;
return true;
}
else {
// full of water, refuse to add water
return false;
}
}
}
|
Token Bucket Algorithm
Token bucket algorithm, also known as token bucket. To understand the algorithm, let's take a look at the schematic diagram of the algorithm on Wikipedia:
From the figure, we can see that the token bucket algorithm is slightly more complicated than the leaky bucket algorithm. First, we have a fixed-capacity bucket that holds tokens. The bucket is empty at the beginning, and tokens are filled into the bucket at a fixed rate r until the capacity of the bucket is reached, and excess tokens will be discarded. Whenever a request comes in, an attempt is made to remove a token from the bucket, if there is no token, the request cannot go through.
The specific pseudo-code implementation is as follows:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
public class TokenBucketDemo {
public long timeStamp = getNowTime();
public int capacity; // 桶的容量
public int rate; // 令牌放入速度
public int tokens; // 当前令牌数量
public boolean grant() {
long now = getNowTime();
// 先添加令牌
tokens = min(capacity, tokens + (now - timeStamp) * rate);
timeStamp = now;
if (tokens < 1) {
// 若不到1个令牌,则拒绝
return false;
}
else {
// 还有令牌，领取令牌
tokens -=
1;
return true;
}
}
}
|
相关变种
If we look closely at the algorithm, we will see that it takes no time for us to remove tokens from the bucket by default. If a delay time is set for removing the token, the idea of the leaky bucket algorithm is actually adopted. The classes under Google's guava library SmoothWarmingUpuse this idea.
critical problem
Let us consider the critical problem scenario again. At 0:59 seconds, since the bucket is full of 100 tokens, these 100 requests can be passed instantly. However, since the tokens are filled at a low rate, at 1:00, the number of tokens in the bucket cannot reach 100, so it is impossible for another 100 requests to pass at this time. So the token bucket algorithm can solve the critical problem well. The graph below compares the rate change at the tipping point for the counter (left) and the token bucket algorithm (right). We can see that although the token bucket algorithm allows burst rates, the next burst rate cannot occur until there are enough tokens in the bucket:
Summarize
Counter vs Sliding Window
The counter algorithm is the simplest algorithm and can be seen as a low-precision implementation of a sliding window. Because the sliding window needs to store multiple counters (one for each grid), the sliding window needs more storage space in implementation. That is, if the precision of the sliding window is higher, the storage space required is larger.
Leaky Bucket Algorithm VS Token Bucket Algorithm
The most obvious difference between the leaky bucket algorithm and the token bucket algorithm is that the token bucket algorithm allows a certain degree of burst of traffic. Because of the default token bucket algorithm, it does not take time to remove tokens. That is to say, if there are 100 tokens in the bucket, 100 requests can be allowed to pass instantly.
The token bucket algorithm is widely used in the industry because it is simple to implement, allows some traffic bursts, and is user-friendly. Of course, we need to analyze the specific situation, there is only the most suitable algorithm, and there is no optimal algorithm.
https://my.oschina.net/sbcagf/blog/783082