The appearance of the API concept predates the birth of the personal computer, let alone the birth of the network. In the early days of public data processing, in order for an application to interact with other systems, developers have begun to design "access points" that are publicly accessible and clearly described. As early as that time, this approach as a guideline was already the mainstream concept of software development. However, it wasn't until the emergence of distributed systems and even the advent of the network that these basic concepts played their important and amazing effects.
When we look back at the history of the API, one of the stages is very important. It was around 2000, and SOA (Service Oriented Architecture) was under development. A form of API was born in enterprise applications. As one of the great practices of SOA, this form of API has gone out of the field of enterprise applications and found more fertile soil in the world of innovative technology.
Today, we can find countless reasons from a technical perspective to explain why web APIs can be successful in various types and sizes of enterprises, and are even widely welcomed by government agencies. But in reality, technology is not everything. The success of web API is also due to many other factors. Most of these factors are not so eye-catching, so we need to study the history carefully, and after careful observation, we can discover why the pioneers of web API can succeed.
Today, we still have to learn the best practices of the past ten years. When conducting research on pioneers who have successfully provided APIs, including Amazon, Salesforce, Ebay, and Twitter, we cannot ignore any important details. You know, most of the APIs they provide are still running.
1. Scene description
Many people who do service interfaces encounter such scenarios more or less. Due to the limited load capacity of the business application system, in order to prevent unexpected requests from overwhelming the system and drag down the business application system.
That is, how to control the flow when facing a large flow?
The traffic control strategy of the service interface: diversion, downgrade, current limit, etc. This article discusses the lower limit strategy. Although the access frequency and concurrency of the service interface is reduced, it is exchanged for the high availability of the service interface and business application system.
Current limiting strategies commonly used in actual scenarios:
Nginx front-end current limit
Limit the flow at the Nginx level according to certain rules such as account number, IP, system call logic, etc.
Business application system current limit
1. Client current limit
2. Server current limit
Database current limit
Red line area, strive to secure the database
2. Commonly used current limiting algorithms
Commonly used current limiting algorithms are: building bucket algorithm and token bucket algorithm. This article does not specify the principles of the two algorithms in detail, and the principles will be explained in the next article.
1. Leaky bucket algorithm
The Leaky Bucket algorithm is very simple.The water (request) enters the leaky bucket first, and the leaky bucket discharges water at a certain speed (the interface has a response rate). When the water inflow rate is too high, it will overflow directly (the access frequency exceeds the interface response). Rate), and then reject the request, it can be seen that the leaky bucket algorithm can forcibly limit the data transmission rate.
The schematic diagram is as follows:
It can be seen that there are two variables, one is the size of the bucket, which supports how much water can be stored when the traffic increases suddenly (burst), and the other is the size of the bucket hole (rate).
Because the leakage rate of the leaky bucket is a fixed parameter, even if there is no resource conflict in the network (no congestion occurs), the leaky bucket algorithm cannot burst the flow to the port rate. Therefore, the leaky bucket algorithm is very effective in It lacks efficiency in terms of traffic characteristics.
2. Token bucket algorithm
The Token Bucket algorithm (Token Bucket) has the same effect as the Leaky Bucket algorithm but in the opposite direction, which is easier to understand. As time passes, the system will go to the bucket at a constant 1/QPS time interval (if QPS=100, the interval is 10ms) Add Token to it (imagine that there is a faucet that is constantly adding water to the leak). If the bucket is full, no more will be added. When a new request comes, one Token will be taken away, and if there is no Token available, it will be blocked. Or denial of service.
Another advantage of the token bucket is that it can easily change the speed. Once the rate needs to be increased, the rate of tokens put into the bucket will be increased as needed. Generally, a certain number of tokens will be added to the bucket regularly (for example, 100 milliseconds) , Some variant algorithms calculate the number of tokens that should be increased in real time.
Third, the realization based on Redis function
Simple design idea: Assuming that a user (judged by IP) cannot access a certain service interface more than 10 times per minute, then we can create a key in Redis, and at this time we set the key expiration time to 60 seconds , Each user accesses this service interface will increase the key value by 1, and when the key value increases to 10 within 60 seconds, access to the service interface is prohibited. It is still necessary to add access time intervals in certain scenarios.
1) Use the incr command of Redis to use the counter as a Lua script
local current
current = redis.call("incr",KEYS[1])
if tonumber(current) == 1 then
redis.call("expire",KEYS[1],1)
end
The Lua script runs in Redis to ensure the atomicity of the incr and expire operations.
2) Use the list structure of Reids instead of the incr command
FUNCTION LIMIT_API_CALL(ip)
current = LLEN(ip)
IF current > 10 THEN
ERROR "too many requests per second"
ELSE
IF EXISTS(ip) == FALSE
MULTI
RPUSH(ip,ip)
EXPIRE(ip,1)
EXEC
ELSE
RPUSHX (ip, ip)
END
PERFORM_API_CALL()
END
Rate Limit uses Redis lists as containers, LLEN is used to check the number of visits, and a transaction contains two commands RPUSH and EXPIRE, which are used to create a list and set an expiration time when the count is executed for the first time.
RPUSHX performs an increase operation in the subsequent counting operation.
Fourth, based on the implementation of the token bucket algorithm
The token bucket algorithm can well support sudden changes in the amount of traffic, that is, the peak of the number of full token buckets.
import java.io.BufferedWriter;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.util.Random;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.ReentrantLock;
import com.google.common.base.Preconditions;
import com.netease.datastream.util.framework.LifeCycle;
20 public class TokenBucket implements LifeCycle {
// The number of default bucket sizes, that is, the maximum instantaneous flow is 64M
private static final int DEFAULT_BUCKET_SIZE = 1024 * 1024 * 64;
// The unit of a bucket is 1 byte
private int everyTokenSize = 1;
// instantaneous maximum flow
private int maxFlowRate;
// average flow
private int avgFlowRate;
// Queue to buffer the number of buckets: the maximum traffic peak is = everyTokenSize*DEFAULT_BUCKET_SIZE 64M = 1 * 1024 * 1024 * 64
private ArrayBlockingQueue<Byte> tokenQueue = new ArrayBlockingQueue<Byte>(DEFAULT_BUCKET_SIZE);
private ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();
private volatile boolean isStart = false;
private ReentrantLock lock = new ReentrantLock(true);
private static final byte A_CHAR = 'a';
public TokenBucket() {
}
public TokenBucket(int maxFlowRate, int avgFlowRate) {
this.maxFlowRate = maxFlowRate;
this.avgFlowRate = avgFlowRate;
}
public TokenBucket(int everyTokenSize, int maxFlowRate, int avgFlowRate) {
this.everyTokenSize = everyTokenSize;
this.maxFlowRate = maxFlowRate;
this.avgFlowRate = avgFlowRate;
}
public void addTokens(Integer tokenNum) {
// If the bucket is full, there is no longer a new token
for (int i = 0; i < tokenNum; i++) {
tokenQueue.offer(Byte.valueOf(A_CHAR));
}
}
public TokenBucket build() {
start();
return this;
}
/**
· Get enough tokens
·
· @return
· */
· public boolean getTokens(byte[] dataSize) {
Preconditions.checkNotNull (dataSize);
Preconditions.checkArgument(isStart, "please invoke start method first !");
int needTokenNum = dataSize.length / everyTokenSize + 1;// The number of buckets corresponding to the size of the transmitted content
final ReentrantLock lock = this.lock;
lock.lock();
try {
boolean result = needTokenNum <= tokenQueue.size(); // Is there a sufficient number of buckets
if (!result) {
return false;
}
int tokenCount = 0;
for (int i = 0; i < needTokenNum; i++) {
Byte poll = tokenQueue.poll();
if (poll != null) {
tokenCount++;
}
}
return tokenCount == needTokenNum;
} finally {
lock.unlock();
}
}
@Override
public void start() {
// Initialize the bucket queue size
if (maxFlowRate != 0) {
tokenQueue = new ArrayBlockingQueue<Byte>(maxFlowRate);
}
// Initialize the token producer
TokenProducer tokenProducer = new TokenProducer(avgFlowRate, this);
scheduledExecutorService.scheduleAtFixedRate(tokenProducer, 0, 1, TimeUnit.SECONDS);
isStart = true;
}
@Override
public void stop() {
isStart = false;
scheduledExecutorService.shutdown();
}
@Override
public boolean isStarted() {
return isStart;
}
class TokenProducer implements Runnable {
private int avgFlowRate;
private TokenBucket tokenBucket;
public TokenProducer(int avgFlowRate, TokenBucket tokenBucket) {
this.avgFlowRate = avgFlowRate;
this.tokenBucket = tokenBucket;
}
@Override
public void run() {
tokenBucket.addTokens(avgFlowRate);
}
}
public static TokenBucket newBuilder() {
return new TokenBucket();
}
public TokenBucket everyTokenSize(int everyTokenSize) {
this.everyTokenSize = everyTokenSize;
return this;
}
public TokenBucket maxFlowRate(int maxFlowRate) {
this.maxFlowRate = maxFlowRate;
return this;
}
public TokenBucket avgFlowRate(int avgFlowRate) {
this.avgFlowRate = avgFlowRate;
return this;
}
private String stringCopy(String data, int copyNum) {
StringBuilder sbuilder = new StringBuilder(data.length() * copyNum);
for (int i = 0; i < copyNum; i++) {
sbuilder.append(data);
}
return sbuilder.toString();
}
public static void main(String[] args) throws IOException, InterruptedException {
tokenTest ();
}
private static void arrayTest() {
ArrayBlockingQueue<Integer> tokenQueue = new ArrayBlockingQueue<Integer>(10);
tokenQueue.offer(1);
tokenQueue.offer(1);
tokenQueue.offer(1);
System.out.println(tokenQueue.size());
System.out.println(tokenQueue.remainingCapacity());
}
private static void tokenTest() throws InterruptedException, IOException {
TokenBucket tokenBucket = TokenBucket.newBuilder().avgFlowRate(512).maxFlowRate(1024).build();
BufferedWriter bufferedWriter = new BufferedWriter(new OutputStreamWriter(new FileOutputStream("/tmp/ds_test")));
String data = "xxxx";// Four bytes
for (int i = 1; i <= 1000; i++) {
Random random = new Random();
int i1 = random.nextInt(100);
boolean tokens = tokenBucket.getTokens(tokenBucket.stringCopy(data, i1).getBytes());
TimeUnit.MILLISECONDS.sleep(100);
if (tokens) {
bufferedWriter.write("token pass --- index:" + i1);
System.out.println("token pass --- index:" + i1);
} else {
bufferedWriter.write("token rejuect --- index" + i1);
System.out.println("token rejuect --- index" + i1);
}
bufferedWriter.newLine();
bufferedWriter.flush();
}
bufferedWriter.close();
}
}
to sum up
At this point, the current limiting function of the service interface API is over. I hope you can forgive me for the shortcomings! ! If you feel that you have gained something, you can click to follow the collection and forward a wave, thank you for your support. (Blow a wave, 233~~)
Let me share some programming experience with you:
1. Write more code and type more code, good code and solid basic knowledge must be practiced
2. Test, test and test again. If you don't thoroughly test your own code, then you may develop more than just code, you may be infamous.
3. Simplify programming, speed up, and code coquettish. After you finish coding, you should go back and optimize it. In the long run, some improvements here or there will make later support staff easier.