Recommended reading:
Counter fixed window algorithm
principle
The counter fixed window algorithm is the most basic and simplest current limiting algorithm. The principle is to count the requests within a fixed time window. If the number of requests exceeds the threshold, the request is discarded; if the set threshold is not reached, the request is accepted, and the count is increased by 1. When the time window ends, reset the counter to 0.
Code implementation and testing
It is also relatively simple to implement, as follows:
package project.limiter;
import java.util.concurrent.atomic.AtomicInteger;
/**
* Project: AllForJava
* Title:
* Description:
* Date: 2020-09-07 15:56
* Copyright: Copyright (c) 2020
*
* @公众号: 超悦编程
* @微信号：exzlco
* @author: 超悦人生
* @email: [email protected]
* @version 1.0
**/
public class CounterLimiter {
private int windowSize; //窗口大小，毫秒为单位
private int limit;//窗口内限流大小
private AtomicInteger count;//当前窗口的计数器
private CounterLimiter(){}
public CounterLimiter(int windowSize,int limit){
this.limit = limit;
this.windowSize = windowSize;
count = new AtomicInteger(0);
//开启一个线程，达到窗口结束时清空count
new Thread(new Runnable() {
@Override
public void run() {
while(true){
count.set(0);
try {
Thread.sleep(windowSize);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}).start();
}
//请求到达后先调用本方法，若返回true，则请求通过，否则限流
public boolean tryAcquire(){
int newCount = count.addAndGet(1);
if(newCount > limit){
return false;
}else{
return true;
}
}
//测试
public static void main(String[] args) throws InterruptedException {
//每秒20个请求
CounterLimiter counterLimiter = new CounterLimiter(1000,20);
int count = 0;
//模拟50次请求，看多少能通过
for(int i = 0;i < 50;i ++){
if(counterLimiter.tryAcquire()){
count ++;
}
}
System.out.println("第一拨50次请求中通过：" + count + ",限流：" + (50 - count));
//过一秒再请求
Thread.sleep(1000);
//模拟50次请求，看多少能通过
count = 0;
for(int i = 0;i < 50;i ++){
if(counterLimiter.tryAcquire()){
count ++;
}
}
System.out.println("第二拨50次请求中通过：" + count + ",限流：" + (50 - count));
}
}
The test results are as follows:
It can be seen that only 20 of the 50 requests passed and 30 were restricted, which achieved the expected current restriction effect.
Characteristic analysis
Advantages : simple to implement and easy to understand.
Disadvantages : The flow curve may not be smooth enough, with "spur phenomenon", as shown in the figure below. There will be two problems:
-
The system service is unavailable for a period of time (not exceeding the time window) . For example, the window size is 1s, the current limit size is 100, and then 100 requests happen in the first ms of a window, and then the requests from 2ms to 999ms will be rejected. During this time, users will feel that the system service is unavailable.
-
When the window is switched, a request of twice the threshold traffic may be generated . For example, the window size is 1s, the current limit size is 100, and then 100 requests happen to come in the 999ms of a certain window. There are no requests in the early stage of the window, so all 100 requests will pass. It just so happens that 100 requests came in the first ms of the next window, and all of them passed, that is, 200 requests passed within 2ms, and the threshold we set was 100, and the passed requests reached the threshold. double.
Counter sliding window algorithm
principle
The counter sliding window algorithm is an improvement of the counter fixed window algorithm, which solves the shortcoming of twice the threshold traffic request when the fixed window is switched.
The sliding window algorithm divides a timing window into several small windows on the basis of a fixed window, and then each small window maintains an independent counter. When the requested time is greater than the maximum time of the current window, the timing window is shifted forward by a small window. When panning, discard the data of the first small window, then set the second small window as the first small window, and add a small window at the end, and place the new request in the newly added small window . At the same time, it is necessary to ensure that the number of requests for all small windows in the entire window cannot exceed the set threshold afterwards.
It is not difficult to see from the figure that the sliding window algorithm is an upgraded version of the fixed window. Dividing the timing window into a small window, the sliding window algorithm degenerates into a fixed window algorithm. The sliding window algorithm actually limits the number of requests at a finer granularity. The more windows are divided, the more accurate the current limit.
Code implementation and testing
package project.limiter;
/**
* Project: AllForJava
* Title:
* Description:
* Date: 2020-09-07 18:38
* Copyright: Copyright (c) 2020
*
* @公众号: 超悦编程
* @微信号：exzlco
* @author: 超悦人生
* @email: [email protected]
* @version 1.0
**/
public class CounterSildeWindowLimiter {
private int windowSize; //窗口大小，毫秒为单位
private int limit;//窗口内限流大小
private int splitNum;//切分小窗口的数目大小
private int[] counters;//每个小窗口的计数数组
private int index;//当前小窗口计数器的索引
private long startTime;//窗口开始时间
private CounterSildeWindowLimiter(){}
public CounterSildeWindowLimiter(int windowSize, int limit, int splitNum){
this.limit = limit;
this.windowSize = windowSize;
this.splitNum = splitNum;
counters = new int[splitNum];
index = 0;
startTime = System.currentTimeMillis();
}
//请求到达后先调用本方法，若返回true，则请求通过，否则限流
public synchronized boolean tryAcquire(){
long curTime = System.currentTimeMillis();
long windowsNum = Math.max(curTime - windowSize - startTime,0) / (windowSize / splitNum);//计算滑动小窗口的数量
slideWindow(windowsNum);//滑动窗口
int count = 0;
for(int i = 0;i < splitNum;i ++){
count += counters[i];
}
if(count >= limit){
return false;
}else{
counters[index] ++;
return true;
}
}
private synchronized void slideWindow(long windowsNum){
if(windowsNum == 0)
return;
long slideNum = Math.min(windowsNum,splitNum);
for(int i = 0;i < slideNum;i ++){
index = (index + 1) % splitNum;
counters[index] = 0;
}
startTime = startTime + windowsNum * (windowSize / splitNum);//更新滑动窗口时间
}
//测试
public static void main(String[] args) throws InterruptedException {
//每秒20个请求
int limit = 20;
CounterSildeWindowLimiter counterSildeWindowLimiter = new CounterSildeWindowLimiter(1000,limit,10);
int count = 0;
Thread.sleep(3000);
//计数器滑动窗口算法模拟100组间隔30ms的50次请求
System.out.println("计数器滑动窗口算法测试开始");
System.out.println("开始模拟100组间隔150ms的50次请求");
int faliCount = 0;
for(int j = 0;j < 100;j ++){
count = 0;
for(int i = 0;i < 50;i ++){
if(counterSildeWindowLimiter.tryAcquire()){
count ++;
}
}
Thread.sleep(150);
//模拟50次请求，看多少能通过
for(int i = 0;i < 50;i ++){
if(counterSildeWindowLimiter.tryAcquire()){
count ++;
}
}
if(count > limit){
System.out.println("时间窗口内放过的请求超过阈值，放过的请求数" + count + ",限流：" + limit);
faliCount ++;
}
Thread.sleep((int)(Math.random() * 100));
}
System.out.println("计数器滑动窗口算法测试结束，100组间隔150ms的50次请求模拟完成，限流失败组数：" + faliCount);
System.out.println("===========================================================================================");
//计数器固定窗口算法模拟100组间隔30ms的50次请求
System.out.println("计数器固定窗口算法测试开始");
//模拟100组间隔30ms的50次请求
CounterLimiter counterLimiter = new CounterLimiter(1000,limit);
System.out.println("开始模拟100组间隔150ms的50次请求");
faliCount = 0;
for(int j = 0;j < 100;j ++){
count = 0;
for(int i = 0;i < 50;i ++){
if(counterLimiter.tryAcquire()){
count ++;
}
}
Thread.sleep(150);
//模拟50次请求，看多少能通过
for(int i = 0;i < 50;i ++){
if(counterLimiter.tryAcquire()){
count ++;
}
}
if(count > limit){
System.out.println("时间窗口内放过的请求超过阈值，放过的请求数" + count + ",限流：" + limit);
faliCount ++;
}
Thread.sleep((int)(Math.random() * 100));
}
System.out.println("计数器滑动窗口算法测试结束，100组间隔150ms的50次请求模拟完成，限流失败组数：" + faliCount);
}
}
In the test, the size of the sliding window is 1000/10=100ms, and then 100 sets of 50 requests with an interval of 150ms are simulated. The counter sliding window algorithm is compared with the counter fixed window algorithm, and the following results can be seen:
The fixed window algorithm generates a problem of twice the threshold traffic request when the window is switched, and the sliding window algorithm avoids this problem.
Characteristic analysis
- Avoid the problem that the counter fixed window algorithm may generate twice the threshold traffic request when the fixed window is switched;
- Compared with the funnel algorithm, new requests can also be processed, avoiding the hunger problem of the funnel algorithm.
Funnel algorithm
principle
The principle of the funnel algorithm is also easy to understand. After the request comes, it will first enter the funnel, and then the funnel will process the request outflow at a constant rate, thus smoothing the flow. When the requested traffic is too large, the funnel will overflow when the maximum capacity is reached, and the request is discarded at this time. From the system point of view, we don't know when there will be requests, and we don't know how fast the requests will come, which creates hidden dangers to the security of the system. But if you add a layer of funnel algorithm to limit the current, you can ensure that the request flows out at a constant rate. From the system's point of view, the request always comes at a smooth transmission rate, thus protecting the system.
Code implementation and testing
package project.limiter;
import java.util.Date;
import java.util.LinkedList;
/**
* Project: AllForJava
* Title:
* Description:
* Date: 2020-09-08 16:45
* Copyright: Copyright (c) 2020
*
* @公众号: 超悦编程
* @微信号：exzlco
* @author: 超悦人生
* @email: [email protected]
* @version 1.0
**/
public class LeakyBucketLimiter {
private int capaticy;//漏斗容量
private int rate;//漏斗速率
private int left;//剩余容量
private LinkedList<Request> requestList;
private LeakyBucketLimiter() {}
public LeakyBucketLimiter(int capaticy, int rate) {
this.capaticy = capaticy;
this.rate = rate;
this.left = capaticy;
requestList = new LinkedList<>();
//开启一个定时线程，以固定的速率将漏斗中的请求流出，进行处理
new Thread(new Runnable() {
@Override
public void run() {
while(true){
if(!requestList.isEmpty()){
Request request = requestList.removeFirst();
handleRequest(request);
}
try {
Thread.sleep(1000 / rate); //睡眠
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}).start();
}
/**
* 处理请求
* @param request
*/
private void handleRequest(Request request){
request.setHandleTime(new Date());
System.out.println(request.getCode() + "号请求被处理，请求发起时间："
+ request.getLaunchTime() + ",请求处理时间：" + request.getHandleTime() + ",处理耗时："
+ (request.getHandleTime().getTime() - request.getLaunchTime().getTime()) + "ms");
}
public synchronized boolean tryAcquire(Request request){
if(left <= 0){
return false;
}else{
left --;
requestList.addLast(request);
return true;
}
}
/**
* 请求类，属性包含编号字符串、请求达到时间和请求处理时间
*/
static class Request{
private int code;
private Date launchTime;
private Date handleTime;
private Request() { }
public Request(int code,Date launchTime) {
this.launchTime = launchTime;
this.code = code;
}
public int getCode() {
return code;
}
public void setCode(int code) {
this.code = code;
}
public Date getLaunchTime() {
return launchTime;
}
public void setLaunchTime(Date launchTime) {
this.launchTime = launchTime;
}
public Date getHandleTime() {
return handleTime;
}
public void setHandleTime(Date handleTime) {
this.handleTime = handleTime;
}
}
public static void main(String[] args) {
LeakyBucketLimiter leakyBucketLimiter = new LeakyBucketLimiter(5,2);
for(int i = 1;i <= 10;i ++){
Request request = new Request(i,new Date());
if(leakyBucketLimiter.tryAcquire(request)){
System.out.println(i + "号请求被接受");
}else{
System.out.println(i + "号请求被拒绝");
}
}
}
}
In the test, the capacity of the funnel current limiting algorithm is 5, and the funnel rate is 2 per second, and then 10 consecutive requests, numbered from 1-10, are simulated, and the results are as follows:
It can be seen that requests 1-5 were accepted, while requests 6-10 were rejected, indicating that the funnel has overflowed at this time, which is in line with our expectations.
Let's pay attention to the processing of the five accepted requests. We can see that although the five requests are accepted, the processing is processed one by one (not necessarily in order, depending on the specific implementation), approximately every 500ms processing one. This reflects the characteristics of the funnel algorithm, that is, although the request flow is generated instantaneously, the request flows out at a fixed rate and is processed. Because we set the funnel rate to 2 per second, every 500ms the funnel will leak a request and then process it.
Characteristic analysis
- The leakage rate of the leaky barrel is fixed, which can play a role of rectification . That is to say, although the requested flow may be random, large and small, after the funnel algorithm, it becomes a stable flow with a fixed rate, which protects the downstream system.
- Can not solve the problem of sudden traffic . Take the example we just tested. We set the funnel rate to 2 per second, and then suddenly 10 requests came. Due to the capacity of the funnel, only 5 requests were accepted and the other 5 were rejected. You might say that the funnel rate is 2 per second, and then 5 requests are accepted instantly. Doesn't this solve the problem of sudden traffic? No, these 5 requests were only accepted, but they were not processed immediately. The processing speed is still 2 per second as we set, so the traffic burst problem is not solved. And the token bucket algorithm we are going to talk about can solve the problem of traffic bursts to a certain extent. Readers can compare it.
Token bucket algorithm
principle
The token bucket algorithm is an improvement of the funnel algorithm. In addition to limiting the flow, it also allows a certain degree of traffic burst. In the token bucket algorithm, there is a token bucket, and there is a mechanism in the algorithm to put tokens into the token bucket at a constant rate. The token bucket also has a certain capacity, and if it is full, the token cannot be put in it. When a request comes, it will first go to the token bucket to get the token. If the token is obtained, the request will be processed and the token will be consumed; if the token bucket is empty, the request will be thrown away.
Code implementation and testing
package project.limiter;
import java.util.Date;
/**
* Project: AllForJava
* Title:
* Description:
* Date: 2020-09-08 19:22
* Copyright: Copyright (c) 2020
*
* @公众号: 超悦编程
* @微信号：exzlco
* @author: 超悦人生
* @email: [email protected]
* @version 1.0
**/
public class TokenBucketLimiter {
private int capaticy;//令牌桶容量
private int rate;//令牌产生速率
private int tokenAmount;//令牌数量
public TokenBucketLimiter(int capaticy, int rate) {
this.capaticy = capaticy;
this.rate = rate;
tokenAmount = capaticy;
new Thread(new Runnable() {
@Override
public void run() {
//以恒定速率放令牌
while (true){
synchronized (this){
tokenAmount ++;
if(tokenAmount > capaticy){
tokenAmount = capaticy;
}
}
try {
Thread.sleep(1000 / rate);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}).start();
}
public synchronized boolean tryAcquire(Request request){
if(tokenAmount > 0){
tokenAmount --;
handleRequest(request);
return true;
}else{
return false;
}
}
/**
* 处理请求
* @param request
*/
private void handleRequest(Request request){
request.setHandleTime(new Date());
System.out.println(request.getCode() + "号请求被处理，请求发起时间："
+ request.getLaunchTime() + ",请求处理时间：" + request.getHandleTime() + ",处理耗时："
+ (request.getHandleTime().getTime() - request.getLaunchTime().getTime()) + "ms");
}
/**
* 请求类，属性只包含一个名字字符串
*/
static class Request{
private int code;
private Date launchTime;
private Date handleTime;
private Request() { }
public Request(int code,Date launchTime) {
this.launchTime = launchTime;
this.code = code;
}
public int getCode() {
return code;
}
public void setCode(int code) {
this.code = code;
}
public Date getLaunchTime() {
return launchTime;
}
public void setLaunchTime(Date launchTime) {
this.launchTime = launchTime;
}
public Date getHandleTime() {
return handleTime;
}
public void setHandleTime(Date handleTime) {
this.handleTime = handleTime;
}
}
public static void main(String[] args) throws InterruptedException {
TokenBucketLimiter tokenBucketLimiter = new TokenBucketLimiter(5,2);
for(int i = 1;i <= 10;i ++){
Request request = new Request(i,new Date());
if(tokenBucketLimiter.tryAcquire(request)){
System.out.println(i + "号请求被接受");
}else{
System.out.println(i + "号请求被拒绝");
}
}
}
}
In the test, in order to compare with the funnel current limiting algorithm, the capacity of the token bucket algorithm is also 5, the rate of token generation is 2 per second, and then 10 consecutive requests are simulated, numbered from 1-10, The results are as follows:
It can be seen that for 10 requests, the token bucket algorithm is the same as the funnel algorithm, 5 requests are accepted and 5 requests are rejected. Different from the funnel algorithm, the token bucket algorithm processes these 5 requests immediately, and the processing speed can be considered as 5 per second, which exceeds the rate we set 2 per second, which allows a certain degree of traffic burst. . This is also the main difference from the funnel algorithm, which can be experienced carefully.
Characteristic analysis
The token bucket algorithm is an improvement of the leaky bucket algorithm. It can not only limit the average call rate but also allow a certain degree of traffic burst.
summary
Let's briefly summarize the above four current limiting algorithms.
The counter fixed window algorithm is simple to implement and easy to understand. Compared with the funnel algorithm, new requests can also be processed immediately. However, the flow curve may not be smooth enough, and there may be a "spur phenomenon", and a request for twice the threshold flow may be generated when the window is switched. The counter sliding window algorithm, as an improvement of the counter fixed window algorithm, effectively solves the problem of twice the threshold flow request when the window is switched.
The funnel algorithm can rectify the traffic, allowing random and unstable traffic to flow out at a fixed rate, but it cannot solve the problem of sudden traffic . As an improvement of the funnel algorithm, the token bucket algorithm can not only smooth the flow, but also allows a certain degree of flow burst.
The above four current-limiting algorithms have their own characteristics, and they must be selected according to their own scenarios when using them. There is no best algorithm, only the most suitable algorithm . For example, the token bucket algorithm is generally used to protect its own system, limit the flow of the caller, and protect its own system from burst traffic. If the actual processing capacity of the own system is stronger than the configured flow limit, a certain degree of traffic burst can be allowed, so that the actual processing rate is higher than the configured rate, and system resources are fully utilized. The funnel algorithm is generally used to protect third-party systems. For example, its own system needs to call a third-party interface. In order to protect the third-party system from being overwhelmed by its own calls, the funnel algorithm can be used to limit the flow to ensure its own flow is stable. To the third-party interface.
The algorithm is dead, and the essence of the algorithm is worth learning. In actual scenarios, it can be used flexibly. Again, there is no best algorithm, only the most suitable algorithm .