Limiting
It is indeed a good thing that the service system has a lot of traffic, but if it is overloaded and the system is down, everyone will suffer.
Therefore, before various promotional activities, it is necessary to conduct a stress test on the system, evaluate the peak QPS of the entire system, and make some current limiting settings. If it exceeds a certain threshold, it will refuse to process or postpone the process to avoid hanging up the system. The situation arises.
What is the difference between current limiting and fusing?
Traffic limiting occurs before traffic comes in, and excess traffic is restricted.
Fuse is a mechanism to deal with failures. It occurs after traffic comes in. If the system fails or is abnormal, the fuse will automatically cut off the request to prevent further expansion of the fault and cause an avalanche of services.
What is the difference between current limiting and peak clipping?
Peak clipping is a smoothing process for traffic that avoids instantaneous overloading of the system by slowly increasing the processing rate of requests.
Peak shaving is probably a reservoir, which stores the flow and flows slowly. Flow limiting is probably a gate, which rejects excess flow.
General flow of throttling
So how to implement specific current limiting? It can be summarized into the following steps:
Current Limiting General Process
-
Statistics of request traffic : record the number or rate of requests, and statistics can be carried out through counters, sliding windows, etc.
-
Determine whether the limit is exceeded : Determine whether the current request traffic exceeds the limit based on the set limit conditions.
-
Execute the current limiting policy : If the request traffic exceeds the limit, implement the current limiting policy, such as rejecting the request, delaying processing, returning error information, etc.
-
Update statistical information : Update statistical information based on the processing results of the request, such as increasing the value of the counter, updating the data of the sliding window, etc.
-
Repeat the above steps : continuously count request traffic, determine whether the limit is exceeded, implement the current limiting policy, and update statistical information.
It should be noted that the implementation of the specific current limiting algorithm may be adjusted and optimized according to different scenarios and requirements, such as using token bucket algorithm, leaky bucket algorithm, etc.
Single machine current limiting and distributed current limiting
We have noticed that in the general process of current limiting, it is necessary to count requests and update statistics, so the statistics and updates of requests must be maintained in a storage.
If it is just a stand-alone environment, it would be easy to store it directly locally.
Single machine vs cluster
But generally speaking, our services are deployed in clusters. How to achieve overall current limiting among multiple machines?
At this time, we can put our statistical information into distributed KV storage such as Tair or Redis.
Four current limiting algorithms and distributed implementation
Next, we began to implement some common current-limiting algorithms. Redis is used here as distributed storage. Needless to say, Redis is the most popular distributed cache DB; Redission is used as a Redis client, and Redission is simply used for distributed storage. Locks are somewhat "foolish", but they are actually very easy to use as a Redis client.
Distributed implementation of five current limiting algorithms
Before we start, let's briefly prepare the environment. We won't go into details about Redis installation and project creation.
-
add dependencies
<dependency>
<groupId>org.redisson</groupId>
<artifactId>redisson</artifactId>
<version>3.16.2</version>
</dependency>
-
Use singleton mode to obtain RedissonClient. It is not registered as a bean here. Running single test is too slow.
public class RedissonConfig {
private static final String REDIS_ADDRESS = "redis://127.0.0.1:6379";
private static volatile RedissonClient redissonClient;
public static RedissonClient getInstance(){
if (redissonClient==null){
synchronized (RedissonConfig.class){
if (redissonClient==null){
Config config = new Config();
config.useSingleServer().setAddress(REDIS_ADDRESS);
redissonClient = Redisson.create(config);
return redissonClient;
}
}
}
return redissonClient;
}
}
1. Fixed window current limiting algorithm
Algorithm principle
Fixed window algorithm, many references are also called counter algorithm, of course, I personally understand that counter algorithm is a special case of fixed window algorithm, of course we don't worry so much.
The fixed window algorithm is a relatively simple current limiting algorithm, which divides time into fixed time windows, and sets a limit on the number of requests allowed in each window. If the number of requests exceeds the upper limit within a time window, current limiting will be triggered.