Current limiting algorithm
Taking the fixed time window algorithm and the sliding time window algorithm as examples, the analysis of the two current limiting algorithms is carried out.
Fixed Time Window Algorithm
Within a fixed time window, the setting allows a fixed number of requests to come in. If it exceeds the set threshold, the request is rejected or queued.
Specifically, time is divided into several time windows, and the threshold of the number of requests set in each time window is the same. Once the number of requests in a certain time window reaches the threshold, new requests will be rejected or queued.
The disadvantage is that it is impossible to calculate whether the number of requests across adjacent time windows reaches the threshold.
Sliding Time Window Algorithm
On the basis of a fixed time window, the start and end points of the time window are no longer fixed, but change over time, but the length of each time window (end point - start point) is always fixed, that is to say, the overall Swipe continuously over time.
However, every time the time window is moved, it is necessary to re-count the number of requests, and there will be problems of repeated calculations in some overlapping areas. Therefore, the time window can be divided into finer grains and some sub-time windows, namely sample windows, can be added. In this way, the calculation of the number of requests in the time window will become the calculation of the number of requests in the corresponding sample window, and then summed. If it exceeds the threshold, it will also be limited.
Source code analysis
Taking the currentWindow method of LeapArray inside Sentinel as an example, analyze how to obtain the corresponding sample window according to the specified time.
Process overview
1. Specify the time/time window length (500ms by default), then % the total number of sample windows to get the subscript n corresponding to the new sample window to which it belongs.
2. Then get the start time of the corresponding sample window by specifying the time - specifying the time % of the length of the time window.
3. Obtain the old sample window with the specified subscript n from the cache. If the sample window does not exist, create it and return it.
4. Comparing the start time t1 of the new sample window with the start time t2 of the old sample window, there are three cases.
- If t1 = t2, it means that the new sample window is the same as the old sample window, then return the old sample window.
- If t1 > t2, it means that the state of the old sample window is lagging behind, then reset all the indicators of the old sample window, then use LongAdder to calculate a certain indicator and update it, and finally return to the updated sample window.
- If t1 < t2, it means that the time has reversed (generally it will not happen), then create a new sample window and return.
LeapArray
public WindowWrap<T> currentWindow(long timeMillis) {
if (timeMillis < 0) {
return null;
}
// 计算指定时间所属的样本窗口的下标
int idx = calculateTimeIdx(timeMillis);
// 计算指定时间所属的样本窗口的起始时间点
long windowStart = calculateWindowStart(timeMillis);
/*
* Get bucket item at given time from the array.
*
* (1) Bucket is absent, then just create a new bucket and CAS update to circular array.
* (2) Bucket is up-to-date, then just return the bucket.
* (3) Bucket is deprecated, then reset current bucket and clean all deprecated buckets.
*/
while (true) {
WindowWrap<T> old = array.get(idx);
if (old == null) {
/*
* B0 B1 B2 NULL B4
* ||_______|_______|_______|_______|_______||___
* 200 400 600 800 1000 1200 timestamp
* ^
* time=888
* bucket is empty, so create new and update
*
* If the old bucket is absent, then we create a new bucket at {@code windowStart},
* then try to update circular array via a CAS operation. Only one thread can
* succeed to update, while other threads yield its time slice.
*/
WindowWrap<T> window = new WindowWrap<T>(windowLengthInMs, windowStart, newEmptyBucket(timeMillis));
if (array.compareAndSet(idx, null, window)) {
return window;
} else {
Thread.yield();
}
} else if (windowStart == old.windowStart()) {
/*
* B0 B1 B2 B3 B4
* ||_______|_______|_______|_______|_______||___
* 200 400 600 800 1000 1200 timestamp
* ^
* time=888
* startTime of Bucket 3: 800, so it's up-to-date
*
* If current {@code windowStart} is equal to the start timestamp of old bucket,
* that means the time is within the bucket, so directly return the bucket.
*/
return old;
} else if (windowStart > old.windowStart()) {
/*
* (old)
* B0 B1 B2 NULL B4
* |_______||_______|_______|_______|_______|_______||___
* ... 1200 1400 1600 1800 2000 2200 timestamp
* ^
* time=1676
* startTime of Bucket 2: 400, deprecated, should be reset
*
* If the start timestamp of old bucket is behind provided time, that means
* the bucket is deprecated. We have to reset the bucket to current {@code windowStart}.
* Note that the reset and clean-up operations are hard to be atomic,
* so we need a update lock to guarantee the correctness of bucket update.
*
* The update lock is conditional (tiny scope) and will take effect only when
* bucket is deprecated, so in most cases it won't lead to performance loss.
*/
if (updateLock.tryLock()) {
try {
// 重置所有指标并计算PASS指标
return resetWindowTo(old, windowStart);
} finally {
updateLock.unlock();
}
} else {
Thread.yield();
}
} else if (windowStart < old.windowStart()) {
// 注意：一般不会出现该情况，该情况属于时间倒流
return new WindowWrap<T>(windowLengthInMs, windowStart, newEmptyBucket(timeMillis));
}
}
}
calculateTimeIdx
private int calculateTimeIdx(/*@Valid*/ long timeMillis) {
// windowLengthInMs默认是500
long timeId = timeMillis / windowLengthInMs;
// array数组的长度默认是2
return (int)(timeId % array.length());
}
Computes the subscript of the sample window to which the specified time falls.
calculateWindowStart
protected long calculateWindowStart(/*@Valid*/ long timeMillis) {
return timeMillis - timeMillis % windowLengthInMs;
}
Computes the start time of the sample window to which the specified time falls.
resetWindowTo
Take OccupiableBucketLeapArray as an example.
@Override
protected WindowWrap<MetricBucket> resetWindowTo(WindowWrap<MetricBucket> w, long time) {
// 将windowStart设置为指定时间，即样本窗口的开始时间
w.resetTo(time);
// 获取样本窗口的统计数据
MetricBucket borrowBucket = borrowArray.getWindowValue(time);
if (borrowBucket != null) {
// 重置所有指标
w.value().reset();
// 计算PASS指标
w.value().addPass((int)borrowBucket.pass());
} else {
// 重置所有指标
w.value().reset();
}
return w;
}
MetricBucket#reset
public MetricBucket reset() {
for (MetricEvent event : MetricEvent.values()) {
// 重置所有指标
counters[event.ordinal()].reset();
}
// 初始化minRt属性
initMinRt();
return this;
}
MetricEvent is an enumerated class, including: PASS, BLOCK, EXCEPTION, SUCCESS, RT, OCCUPIED_PASS.
MetricBucket#initMinRt
private void initMinRt() {
// 获取 csp.sentinel.statistic.max.rt 属性值（默认 5000），并初始化minRt属性
this.minRt = SentinelConfig.statisticMaxRt();
}
MetricBucket#addPass
public void addPass(int n) {
// 计算PASS指标
add(MetricEvent.PASS, n);
}
public MetricBucket add(MetricEvent event, long n) {
// 底层使用LongAdder计算指标
counters[event.ordinal()].add(n);
return this;
}