Table of contents
Application scenarios
1. Prevent system crashes caused by excessive traffic: When traffic suddenly increases, flow limiting can ensure that the system does not run out of resources and crash due to excessive traffic.
2. Prevent malicious requests: Current limiting can be used to limit some malicious requests to prevent these requests from causing damage to the system.
3. Avoid thread pool exhaustion: When there are many requests, the backend thread pool may be exhausted, resulting in many requests not being responded to. Adding throttling can prevent this from happening.
4. Prevent cache breakdown: When the cache fails, the backend system will receive a large number of requests. Throttling ensures that back-end systems are not overwhelmed by cache invalidations.
5. Control traffic concurrency: Some systems will control concurrency based on their own processing capabilities. Current limiting can control traffic and concurrency.
Common strategies
1. Client current limiting: Limit the frequency of client requests to the server. This can be achieved by implementing the current limiting algorithm on the client side.
2. Server-side current limiting: Limit the frequency of server processing requests. This can be achieved by implementing current limiting on the server.
3. Application layer current limiting: The application layer limits the current flow of certain requests based on business logic. For example, limit the frequency of access to important resources.
4. Network layer current limiting: perform traffic control on network devices to limit the frequency of requests arriving at the server.
Commonly used techniques
1. Counter: Use a counter to count the number of requests. If the threshold is reached, the request will be rejected. This is a simple current limiting method that can use Java's AtomicInteger to implement a counter.
2. Sliding window: Maintain a fixed-size window, count the number of requests within the window, and reject the request if it reaches the threshold. This method is more accurate than a simple counter, and Java's LinkedBlockingQueue can be used to implement a sliding window.
3. Leaky bucket algorithm: Maintain the size of a bucket. If the bucket is full, the request will be rejected, otherwise the request will be released. The size of the leaky bucket controls the current limiting rate, and Java's Semaphore can be used to implement the leaky bucket algorithm.
4. Token Bucket Algorithm: Tokens are generated regularly and put into the bucket. If there is a token in the bucket, the request is allowed and one token is consumed, otherwise the request is rejected. The rate of token generation controls the current limiting rate, and Java's ScheduledExecutorService and Semaphore can be used to implement the token bucket algorithm.
5. RateLimiter in Guava: RateLimiter in Google Guava implements the token bucket algorithm and can be used directly. It provides simple and easy-to-use current limiting function and has good support for concurrent requests.
Code
counter
public boolean limit() {
count++;
if (count > threshold) {
return false; // 达到限流阈值,拒绝请求
}
return true;
}sliding window
Deque<Long> window = new ArrayDeque<>();
long now = System.currentTimeMillis();
window.add(now);
if (window.size() > windowSize) {
long oldest = window.pollFirst();
if (now - oldest < interval) {
return false; // 窗口内请求数达到限流阈值,拒绝请求
}
}
return true; leaky bucket algorithm
Semaphore semaphore = new Semaphore(bucketSize);
if (semaphore.tryAcquire()) {
return true; // 获取令牌成功,放行请求
}
return false; // 桶中无令牌,拒绝请求Token Bucket Algorithm
ScheduledExecutorService scheduled = Executors.newScheduledThreadPool(1);
Semaphore semaphore = new Semaphore(0);
scheduled.scheduleAtFixedRate(() -> {
int added = Math.min(bucketSize - semaphore.availablePermits(), addRate);
semaphore.release(added); // 按速率添加令牌
}, 0, 1, TimeUnit.SECONDS);
if (semaphore.tryAcquire()) {
return true; // 获取令牌成功,放行请求
}
return false; // 桶中无令牌,拒绝请求Guava RateLimiter
RateLimiter limiter = RateLimiter.create(2); // 每秒 2 个请求
if (limiter.tryAcquire()) {
return true; // 获取令牌成功,放行请求
}
return false; // 限流,拒绝请求 Advantages and Disadvantages
Each of these current limiting techniques has advantages and disadvantages:
1. Counter:
- Advantages: Simple and easy to implement
- Disadvantages: Not accurate enough, unable to control the rate, window size not configurable
2. Sliding window:
- Advantages: more accurate than counters, configurable window size
- Disadvantages: Implementation complexity is higher than that of counters
3. Leaky bucket algorithm:
- Advantages: can limit the rate, simple to implement
- Disadvantages: Not accurate enough to handle sudden traffic
4. Token bucket algorithm:
- Advantages: can limit the rate and can cope with burst traffic
- Disadvantages: more complex to implement
5. Guava RateLimiter:
- Advantages: simple implementation, can limit the rate, and can cope with sudden traffic
- Disadvantages: Rate control is not flexible enough and cannot implement more complex current limiting strategies.
To sum up, my recommendation is: if you need a simple current limiter, use Guava RateLimiter. It is simple to implement and has sufficient functions. If you need more precise rate control or implement a more complex current limiting strategy, implement your own current limiter based on the token bucket algorithm. If the current limiting accuracy is not high, a simple counter or sliding window can be used. It is very important to choose the appropriate current limiting technology based on the application scenario. No one technology is suitable for all scenarios, so it is critical to understand the advantages and disadvantages of each technology. In addition, current limiting can also be performed from multiple dimensions
for example:
- Limit the flow of a certain user
- Limit the flow of a certain resource
- Limit traffic based on client IP
- etc. So when designing a current limiting system, these factors need to be fully considered.
Application scenarios
1. Counter:
- Scenario: A simple current limiter is required, and the current limiting accuracy is not required to be high.
- For example: Limit the number of accesses to a resource to simply prevent resource theft.
- Framework: Simple application layer current limiting implementation
2. Sliding window:
- Scenario: It is necessary to limit the number of requests within a certain window, and the window size is configurable.
- For example: Limit the number of failed logins within a certain time window to prevent brute force login attempts.
-Framework: Nginx and other proxy servers
3. Leaky bucket algorithm:
- Scenario: Requests need to be limited at a fixed rate to cope with a certain degree of burst traffic.
- For example: limit the rate of interface calls to prevent system overload due to sudden increase in traffic.
- Framework: Hystrix and other fault-tolerant libraries
4. Token bucket algorithm:
- Scenario: It is necessary to control the rate of request passing more accurately and be able to cope with higher burst traffic.
- For example: Limit the access rate to important resources to ensure that resources are not overloaded due to sudden large traffic.
- Framework: Sentinel, Resilience4j and other flow control frameworks
5. Guava RateLimiter:
- Scenario: A simple rate limiter is required, and precise rate control and complex current limiting strategies are not required.
- For example: simply limit the frequency of API calls to prevent excessive server resources from being occupied.
-Framework: Google Guava
In addition, current limiting technology is also used in other products and frameworks:
- AWS WAF: Limit malicious traffic based on rate limiting and web firewall
- Nginx current limiting module: current limiting based on rate limiting and leaky bucket algorithm
- Envoy Router: Limit traffic based on rate limit and token bucket algorithm
- API gateway: limits the flow of API requests, usually based on rate limiting.
- Database agent: Limit the number of database connections and query frequency based on rate limit and leaky bucket algorithm
- Cache proxy: Limit request traffic in cache breakdown scenarios based on rate limit and token bucket algorithm
- etc
It can be seen that these current limiting technologies can be applied to many scenarios, but each technology has its most suitable scenarios. When choosing a current limiting technology, you need to consider:
- Objects that need to be controlled (users, resources, etc.)
- Current limiting accuracy requirements (whether the rate control is accurate)
- Do you need to deal with burst traffic?
- Complexity of current limiting strategy
- etc
Take these factors into consideration before selecting the most suitable current limiting technology.
Tool class encapsulation
This tool class in the code package implements four methods: counter current limiting, sliding window current limiting, leaky bucket current limiting and token bucket current limiting. You can choose the corresponding current limiting method for flow control as needed. Such a tool class that integrates multiple current limiting technologies can be well applied to different current limiting scenarios without having to implement the current limiting algorithm yourself every time.
Note: The current limit of the token bucket and leaky bucket here is the current limit at the thread number level and cannot be used for data speed limit at the business level.