When a large amount of high concurrent access to data, they would often request the service or the face of skyrocketing not available interfaces, and even lead to a chain reaction leading to the collapse of the entire system. At this point you need to use one of the techniques is limiting, when the request reaches a certain number of concurrent or rate, they were waiting, queuing, demotion, denial of service and so on. When limiting, two common algorithm is leaky bucket and token bucket algorithm algorithm, this paper is to highlight relevant content.
Four common limiting algorithm
1, algorithm counter
2, sliding window algorithm
3, leaky bucket
4, the token bucket algorithm
implemented two ways of limiting control: RateLimiter, Semphore
A counter algorithm
Counter algorithm is an algorithm in the simplest and easiest to implement an algorithm limiting. For example, our provisions for the A interface, we can not for one minute visits over 100. Then we can do this: in the beginning, we can set a counter counter, whenever a request is over, a counter is incremented, if the value of counter 100 and the time interval is greater than the request with the first request further within 1 minute, then that too many requests; request interval if the first request is greater than 1 minute, and also limiting the value of counter flow range, the counter is reset, the following algorithm is a schematic view of the specific :
public class CounterTest {
public long timeStamp = getNowTime();
public int reqCount = 0;
public final int limit = 100; // 时间窗口内最大请求数
public final long interval = 1000; // 时间窗口ms
public boolean grant() {
long now = getNowTime();
if (now < timeStamp + interval) {
// 在时间窗口内
reqCount++;
// 判断当前时间窗口内是否超过最大请求控制数
return reqCount <= limit;
} else {
timeStamp = now;
// 超时后重置
reqCount = 1;
return true;
}
}
public long getNowTime() {
return System.currentTimeMillis();
}
}
This algorithm is simple, but there is a very fatal problem, that is the critical issue, we look:
We can see from the figure above, suppose you have a malicious user, he was in 0:59, and instantly sent 100 request and 1:00 and instantly sends request 100, then the user is actually inside 1 second, request 200 is sent instantly. We have just one minute up to a predetermined request 100, i.e. up to 1.7 per request, the user request burst reset node time window, we can instantly exceeds the rate limit. Possible for the user algorithm by this vulnerability, and instantly overwhelm our application.
Smart friends may have seen, in fact, the problem is just because our statistical accuracy is too low. So how do we deal with this problem nicely it? Or, how would it affect the critical issue of reducing? We can see the following sliding window algorithm.
Second, the sliding window algorithm
In the figure, the red rectangle indicates the whole time window, in our example, a time window is one minute. We then divide the time window, such as drawing, we will become the designated sliding window frame 6, so that each cell is represented by 10 seconds. Every 10 seconds, our time window will slide to the right one space. Each grid has its own separate counter counter, such as when a request is 0:35 seconds, when reached, 0:30 ~ 0:39 corresponding counter is increased by one.
So how to solve the critical problem of the sliding window just do? We can fancy FIG, 0: 59 100 requests arriving gray grid will fall, and the request will fall reaches 1:00 orange grid. When the time reaches 1:00, we will move to the right of a window frame, then at this time the total number of requests within a time window of a total of 200, 100 exceeds the limit, so in this case it is possible to detect the trigger current limiting .
I again look just counter algorithm, we can see that, in fact, counter algorithm sliding window algorithm. It just does not further divide the window of time, so only one grid.
Thus, when more grid divided sliding window, the more smooth scrolling sliding window, limiting the statistics will be more accurate.
Third, the leaky bucket algorithm
Request to the drain into the first bucket, leaky bucket of water at a constant speed, when the water overflows directly request over the General Assembly, it can be seen leaky-bucket algorithm can impose restrictions on the data transmission rate.
In terms of algorithm, a queue may be prepared for storage request, obtained by a further thread pool periodic requests from the queue and executed, can acquire disposable multiple concurrent execution.
This algorithm, after use are also disadvantages: Unable to deal with unexpected traffic for a short time.
Fourth, the token bucket algorithm
The size of the token bucket may be fixed at a constant rate itself continuously produce the token. If the token is not consumed or is consumed to produce the speed is slower, the token will continue to increase until the bucket is full. Token back again generated will overflow from the tub. Finally, the maximum number of tokens in the bucket can be saved will never exceed the bucket size.
This action is the release token is continuous, if the number of tokens in the bucket reaches the upper limit, the token is discarded, so there are cases where the bucket has a large number of available tokens, then the incoming request can be directly get a token to perform, such as setting qps to 100, after initialization is complete current limiter second, bucket, there were already 100 tokens, and then start the service is not completely good, and so when you start to complete the external services, the flow restrictor 100 can withstand instantaneous request . So, not only the token bucket, the request will be waiting, and finally at a rate equivalent to the implementation.
Comparative leaky bucket algorithm and the token bucket algorithm:
both the primary difference is that the "leaky bucket algorithm" can be forced to limit the rate of data transmission, while the "token bucket algorithm" can be restricted in an average data transfer rate, but also allows a certain degree burst transmission. In the "token bucket algorithm", as long as the presence of tokens in the bucket, then allowing the data burst transmission until a threshold is user-configurable, so it is suitable for bursty traffic characteristics.
Fifth, the use of Guava flow control to limit RateLimiter
public class RateLimiterMain {
public static void main(String[] args) {
RateLimiter rateLimiter = RateLimiter.create(10);
for (int i = 0; i < 10; i++) {
new Thread(new Runnable() {
@Override
public void run() {
rateLimiter.acquire()
System.out.println("pass");
}
}).start();
}
}
}
In the above example, creates a second generation token flow restrictor 10, i.e. generates a 100ms, and save up to 10 tokens, the excess will be discarded.
Providing rateLimiter acquire () and to tryAcquire () Interface
1, acquire () method, if the token is not available, it will block until there are enough tokens.
2, to tryAcquire () method, if the token is not available, directly returns false.
3, using the method to tryAcquire () with a timeout, if the token is not available, the timeout period will be determined whether the wait token, if not, it returns false, and if so, to block waiting.
Six, using concurrent fluidic Semphore
Semaphore Java concurrency libraries can easily complete the semaphore control, Semaphore can control a number of resources that can be accessed simultaneously, obtaining a license by acquire (), if not wait, and release () Releases a permit. Semaphore single semaphore mutex object can implement the functions, and may be obtained "locked" by a thread, and then release the "lock" by another thread, which can be used in some applications deadlock recovery. The following Demo affirmed a licensed only 200 Semaphore, while 5000 threads to access the resources, access permissions acquired and released by acquire () and release ():
//请求总数
private static int clientTotal = 5000;
//同时并发执行的线程数
private static int threadTotal = 200;
public static void main(String[] args) throws InterruptedException {
ExecutorService executorService = Executors.newCachedThreadPool();
final Semaphore semaphore = new Semaphore(threadTotal);
for(int i=0; i<clientTotal; i++){
executorService.execute(() -> {
try {
semaphore.acquire();
add();
semaphore.release();
} catch (InterruptedException e) {
e.printStackTrace();
}
});
}
}
To limit the flow control may also be a variety of methods have advantages and disadvantages for different scenarios, for example, by the counter control AtomicLong using MQ message queues traffic peak and the like.
