Recently I was curious in the RPC current-limiting fuse downgraded to how to do, hystrix has been more than a year has not been updated, I feel feel to be abandoned, so I put eyes focused to Ali's Sentinel, the way to learn about Ali's source code.
In this chapter I will focus on is FlowRuleManager loaded FlowRule time to do something, the next official about how to control the number of concurrent Sentinel.
Now I give a simplified version of the demo, this demo only single-threaded access, first repeat that process to make it clear multithreaded version.
Flow control initialization rules: limited access to 20 concurrent threads
public class FlowThreadDemo {
private static AtomicInteger pass = new AtomicInteger();
private static AtomicInteger block = new AtomicInteger();
private static AtomicInteger total = new AtomicInteger();
private static AtomicInteger activeThread = new AtomicInteger();
private static volatile boolean stop = false;
private static final int threadCount = 100;
private static int seconds = 60 + 40;
private static volatile int methodBRunningTime = 2000;
public static void main(String[] args) throws Exception {
System.out.println(
"MethodA will call methodB. After running for a while, methodB becomes fast, "
+ "which make methodA also become fast ");
tick();
initFlowRule();
Entry methodA = null;
try {
TimeUnit.MILLISECONDS.sleep(5);
methodA = SphU.entry("methodA");
activeThread.incrementAndGet();
//Entry methodB = SphU.entry("methodB");
TimeUnit.MILLISECONDS.sleep(methodBRunningTime);
//methodB.exit();
pass.addAndGet(1);
} catch (BlockException e1) {
block.incrementAndGet();
} catch (Exception e2) {
// biz exception
} finally {
total.incrementAndGet();
if (methodA != null) {
methodA.exit();
activeThread.decrementAndGet();
}
}
}
private static void initFlowRule() {
List<FlowRule> rules = new ArrayList<FlowRule>();
FlowRule rule1 = new FlowRule();
rule1.setResource("methodA");
// set limit concurrent thread for 'methodA' to 20
rule1.setCount(20);
rule1.setGrade(RuleConstant.FLOW_GRADE_THREAD);
rule1.setLimitApp("default");
rules.add(rule1);
FlowRuleManager.loadRules(rules);
}
private static void tick() {
Thread timer = new Thread(new TimerTask());
timer.setName("sentinel-timer-task");
timer.start();
}
static class TimerTask implements Runnable {
@Override
public void run() {
long start = System.currentTimeMillis();
System.out.println("begin to statistic!!!");
long oldTotal = 0;
long oldPass = 0;
long oldBlock = 0;
while (!stop) {
try {
TimeUnit.SECONDS.sleep(1);
} catch (InterruptedException e) {
}
long globalTotal = total.get();
long oneSecondTotal = globalTotal - oldTotal;
oldTotal = globalTotal;
long globalPass = pass.get();
long oneSecondPass = globalPass - oldPass;
oldPass = globalPass;
long globalBlock = block.get();
long oneSecondBlock = globalBlock - oldBlock;
oldBlock = globalBlock;
System.out.println(seconds + " total qps is: " + oneSecondTotal);
System.out.println(TimeUtil.currentTimeMillis() + ", total:" + oneSecondTotal
+ ", pass:" + oneSecondPass
+ ", block:" + oneSecondBlock
+ " activeThread:" + activeThread.get());
if (seconds-- <= 0) {
stop = true;
}
if (seconds == 40) {
System.out.println("method B is running much faster; more requests are allowed to pass");
methodBRunningTime = 20;
}
}
long cost = System.currentTimeMillis() - start;
System.out.println("time cost: " + cost + " ms");
System.out.println("total:" + total.get() + ", pass:" + pass.get()
+ ", block:" + block.get());
System.exit(0);
}
}
}
FlowRuleManager
In this demo, the first calls FlowRuleManager # loadRules the rules registered
Let's talk about the code rule configuration:
private static void initFlowRule() {
List<FlowRule> rules = new ArrayList<FlowRule>();
FlowRule rule1 = new FlowRule();
rule1.setResource("methodA");
// set limit concurrent thread for 'methodA' to 20
rule1.setCount(20);
rule1.setGrade(RuleConstant.FLOW_GRADE_THREAD);
rule1.setLimitApp("default");
rules.add(rule1);
FlowRuleManager.loadRules(rules);
}This code which define a flow control rule , and then call loadRules register.
FlowRuleManager initialization
FlowRuleManager
FlowRuleManager class there are several static parameters:
//规则集合
private static final Map<String, List<FlowRule>> flowRules = new ConcurrentHashMap<String, List<FlowRule>>();
//监听器
private static final FlowPropertyListener LISTENER = new FlowPropertyListener();
//用来监听配置是否发生变化
private static SentinelProperty<List<FlowRule>> currentProperty = new DynamicSentinelProperty<List<FlowRule>>();
//创建一个延迟的线程池
@SuppressWarnings("PMD.ThreadPoolCreationRule")
private static final ScheduledExecutorService SCHEDULER = Executors.newScheduledThreadPool(1,
new NamedThreadFactory("sentinel-metrics-record-task", true));
static {
//设置监听
currentProperty.addListener(LISTENER);
//每一秒钟调用一次MetricTimerListener的run方法
SCHEDULER.scheduleAtFixedRate(new MetricTimerListener(), 0, 1, TimeUnit.SECONDS);
}Ode will be initialized when the value of a static variable.
In the New MetricTimerListener example when doing a lot of things, let me slowly analysis.
MetricTimerListener
public class MetricTimerListener implements Runnable {
private static final MetricWriter metricWriter = new MetricWriter(SentinelConfig.singleMetricFileSize(),
SentinelConfig.totalMetricFileCount());
....
}When first initialized MetricTimerListener creates a MetricWriter instance. We look at the two passed parameters SentinelConfig. SingleMetricFileSize () and SentinelConfig. TotalMetricFileCount ().
SentinelConfig when first initialized will be static initialization code block:
SentinelConfig
static {
try {
initialize();
loadProps();
resolveAppType();
RecordLog.info("[SentinelConfig] Application type resolved: " + appType);
} catch (Throwable ex) {
RecordLog.warn("[SentinelConfig] Failed to initialize", ex);
ex.printStackTrace();
}
}This static code block is mainly set at the configuration parameters.
SentinelConfig#singleMetricFileSize
SentinelConfig#totalMetricFileCount
public static long singleMetricFileSize() {
try {
//获取的是 1024 * 1024 * 50
return Long.parseLong(props.get(SINGLE_METRIC_FILE_SIZE));
} catch (Throwable throwable) {
RecordLog.warn("[SentinelConfig] Parse singleMetricFileSize fail, use default value: "
+ DEFAULT_SINGLE_METRIC_FILE_SIZE, throwable);
return DEFAULT_SINGLE_METRIC_FILE_SIZE;
}
}
public static int totalMetricFileCount() {
try {
//默认是：6
return Integer.parseInt(props.get(TOTAL_METRIC_FILE_COUNT));
} catch (Throwable throwable) {
RecordLog.warn("[SentinelConfig] Parse totalMetricFileCount fail, use default value: "
+ DEFAULT_TOTAL_METRIC_FILE_COUNT, throwable);
return DEFAULT_TOTAL_METRIC_FILE_COUNT;
}
}
singleMetricFileSize method and totalMetricFileCount mainly acquired SentinelConfig in a static variable in the set get into arguments.
Then we enter into the constructor MetricWriter in:
MetricWriter
public MetricWriter(long singleFileSize, int totalFileCount) {
if (singleFileSize <= 0 || totalFileCount <= 0) {
throw new IllegalArgumentException();
}
RecordLog.info(
"[MetricWriter] Creating new MetricWriter, singleFileSize=" + singleFileSize + ", totalFileCount="
+ totalFileCount);
// /Users/luozhiyun/logs/csp/
this.baseDir = METRIC_BASE_DIR;
File dir = new File(baseDir);
if (!dir.exists()) {
dir.mkdirs();
}
long time = System.currentTimeMillis();
//转换成秒
this.lastSecond = time / 1000;
//singleFileSize = 1024 * 1024 * 50
this.singleFileSize = singleFileSize;
//totalFileCount = 6
this.totalFileCount = totalFileCount;
try {
this.timeSecondBase = df.parse("1970-01-01 00:00:00").getTime() / 1000;
} catch (Exception e) {
RecordLog.warn("[MetricWriter] Create new MetricWriter error", e);
}
}Constructor Create a folder which is mainly disposed single file size, the total number of files, the set time.
Static properties MetricTimerListener finished, and now in terms of the way we run the MetricTimerListener.
MetricTimerListener#run
public void run() {
//这个run方法里面主要是做定时的数据采集，然后写到log文件里去
Map<Long, List<MetricNode>> maps = new TreeMap<Long, List<MetricNode>>();
//遍历集群节点
for (Entry<ResourceWrapper, ClusterNode> e : ClusterBuilderSlot.getClusterNodeMap().entrySet()) {
String name = e.getKey().getName();
ClusterNode node = e.getValue();
Map<Long, MetricNode> metrics = node.metrics();
aggregate(maps, metrics, name);
}
//汇总统计的数据
aggregate(maps, Constants.ENTRY_NODE.metrics(), Constants.TOTAL_IN_RESOURCE_NAME);
if (!maps.isEmpty()) {
for (Entry<Long, List<MetricNode>> entry : maps.entrySet()) {
try {
//写入日志中
metricWriter.write(entry.getKey(), entry.getValue());
} catch (Exception e) {
RecordLog.warn("[MetricTimerListener] Write metric error", e);
}
}
}
}The above method is actually the second run of the statistical data is written to the log go. Which Constants.ENTRY_NODE.metrics()is responsible for statistical data, the following analysis of the methods below us.
Constants.ENTRY_NODEThis code instantiates a ClusterNode example.
ClusterNode inherited StatisticNode, when the statistics are implemented in StatisticNode.
StatisticNode method Metrics method is called.
We take a look at StatisticNode global variables
public class StatisticNode implements Node {
//构建一个统计60s的数据，设置60个滑动窗口，每个窗口1s
//这里创建的是BucketLeapArray实例来进行统计
private transient volatile Metric rollingCounterInSecond = new ArrayMetric(SampleCountProperty.SAMPLE_COUNT,
IntervalProperty.INTERVAL);
//上次统计的时间戳
private long lastFetchTime = -1;
.....
}Then we look at the metrics StatisticNode method:
StatisticNode # metrics
public Map<Long, MetricNode> metrics() {
// The fetch operation is thread-safe under a single-thread scheduler pool.
long currentTime = TimeUtil.currentTimeMillis();
//获取当前时间的滑动窗口的开始时间
currentTime = currentTime - currentTime % 1000;
Map<Long, MetricNode> metrics = new ConcurrentHashMap<>();
//获取滑动窗口里统计的数据
List<MetricNode> nodesOfEverySecond = rollingCounterInMinute.details();
long newLastFetchTime = lastFetchTime;
// Iterate metrics of all resources, filter valid metrics (not-empty and up-to-date).
for (MetricNode node : nodesOfEverySecond) {
//筛选符合的滑动窗口的节点
if (isNodeInTime(node, currentTime) && isValidMetricNode(node)) {
metrics.put(node.getTimestamp(), node);
//选出符合节点里最大的时间戳数据赋值
newLastFetchTime = Math.max(newLastFetchTime, node.getTimestamp());
}
}
//设置成滑动窗口里统计的最大时间
lastFetchTime = newLastFetchTime;
return metrics;
}This method is mainly calls rollingCounterInMinute statistical data, and then screened statistically valid results returned.
We enter into rollingCounterInMinute are examples ArrayMetric, so we enter into the details of the method ArrayMetric
ArrayMetric#details
public List<MetricNode> details() {
List<MetricNode> details = new ArrayList<MetricNode>();
//调用BucketLeapArray
data.currentWindow();
//列出统计结果
List<WindowWrap<MetricBucket>> list = data.list();
for (WindowWrap<MetricBucket> window : list) {
if (window == null) {
continue;
}
//对统计结果进行封装
MetricNode node = new MetricNode();
//代表一秒内被流量控制的请求数量
node.setBlockQps(window.value().block());
//则是一秒内业务本身异常的总和
node.setExceptionQps(window.value().exception());
// 代表一秒内到来到的请求
node.setPassQps(window.value().pass());
//代表一秒内成功处理完的请求；
long successQps = window.value().success();
node.setSuccessQps(successQps);
//代表一秒内该资源的平均响应时间
if (successQps != 0) {
node.setRt(window.value().rt() / successQps);
} else {
node.setRt(window.value().rt());
}
//设置统计窗口的开始时间
node.setTimestamp(window.windowStart());
node.setOccupiedPassQps(window.value().occupiedPass());
details.add(node);
}
return details;
}This method first calls dat.currentWindow（）set the current time window to window list to go. Then call data.list()list all the window data, and then traverse the data is not encapsulated into empty window MetricNode return.
data is an example of BucketLeapArray, BucketLeapArray inherited LeapArray, key statistics are carried out in LeapArray in, so we look LeapArray direct method of currentWindow.
LeapArray#currentWindow
public WindowWrap<T> currentWindow(long timeMillis) {
if (timeMillis < 0) {
return null;
}
//通过当前时间判断属于哪个窗口
int idx = calculateTimeIdx(timeMillis);
//计算出窗口开始时间
// Calculate current bucket start time.
long windowStart = calculateWindowStart(timeMillis);
while (true) {
//获取数组里的老数据
WindowWrap<T> old = array.get(idx);
if (old == null) {
WindowWrap<T> window = new WindowWrap<T>(windowLengthInMs, windowStart, newEmptyBucket(timeMillis));
if (array.compareAndSet(idx, null, window)) {
// Successfully updated, return the created bucket.
return window;
} else {
// Contention failed, the thread will yield its time slice to wait for bucket available.
Thread.yield();
}
// 如果对应时间窗口的开始时间与计算得到的开始时间一样
// 那么代表当前即是我们要找的窗口对象，直接返回
} else if (windowStart == old.windowStart()) {
return old;
} else if (windowStart > old.windowStart()) {
//如果当前的开始时间小于原开始时间，那么就更新到新的开始时间
if (updateLock.tryLock()) {
try {
// Successfully get the update lock, now we reset the bucket.
return resetWindowTo(old, windowStart);
} finally {
updateLock.unlock();
}
} else {
// Contention failed, the thread will yield its time slice to wait for bucket available.
Thread.yield();
}
} else if (windowStart < old.windowStart()) {
//一般来说不会走到这里
// Should not go through here, as the provided time is already behind.
return new WindowWrap<T>(windowLengthInMs, windowStart, newEmptyBucket(timeMillis));
}
}
}This method will first pass in a timeMillis is the current timestamp. Then call calculateTimeIdx
private int calculateTimeIdx(/*@Valid*/ long timeMillis) {
//计算当前时间能够落在array的那个节点上
long timeId = timeMillis / windowLengthInMs;
// Calculate current index so we can map the timestamp to the leap array.
return (int)(timeId % array.length());
}calculateTimeIdx method of dividing the size of each window with the current time stamp, and then modulo data array. a capacity of the data array is an array of 60, 60 seconds represents the statistical small window 60 split.
Examples:
The current = 1567175708975 timeMillis
TimeID = 1567175708 = 1567175708975/1000
TimeID be array.length% () = 1,567,175,708 = 60. 8%
that is the current window is the eighth time.
Then call calculateWindowStart calculate the current time start time
protected long calculateWindowStart(/*@Valid*/ long timeMillis) {
//用当前时间减去窗口大小，计算出窗口开始时间
return timeMillis - timeMillis % windowLengthInMs;
}Then there is a while loop:
Before looking at while we look at the array array loop inside what kind of objects
WindowWrap<T> window = new WindowWrap<T>(windowLengthInMs, windowStart, newEmptyBucket(timeMillis));
WindowWrap wrapper object is a time window, which contains the length of the time window, this is 1000; window start time; within the window data entities, is to call newEmptyBucket method returns a MetricBucket.
MetricBucket
public class MetricBucket {
private final LongAdder[] counters;
//默认4900
private volatile long minRt;
public MetricBucket() {
MetricEvent[] events = MetricEvent.values();
this.counters = new LongAdder[events.length];
for (MetricEvent event : events) {
counters[event.ordinal()] = new LongAdder();
}
//初始化minRt，默认是4900
initMinRt();
}
...
}MetricEvent is an enumeration class:
public enum MetricEvent {
PASS,
BLOCK,
EXCEPTION,
SUCCESS,
RT,
OCCUPIED_PASS
}That is all that MetricBucket statistical data in this window by an internal array of counters for each window.
Next we are speaking about the things done while loop:
- Gets node from the array in bucket
- If the node already exists, with CAS update a new node
- If the node is a new, less direct return
- If a node fails, set the current node, remove all nodes fail
For example:
1. 如果array数据里面的bucket数据如下所示：
B0 B1 B2 NULL B4
||_______|_______|_______|_______|_______||___
200 400 600 800 1000 1200 timestamp
^
time=888
正好当前时间所对应的槽位里面的数据是空的，那么就用CAS更新
2. 如果array里面已经有数据了，并且槽位里面的窗口开始时间和当前的开始时间相等，那么直接返回
B0 B1 B2 B3 B4
||_______|_______|_______|_______|_______||___
200 400 600 800 1000 1200 timestamp
^
time=888
3. 例如当前时间是1676，所对应窗口里面的数据的窗口开始时间小于当前的窗口开始时间，那么加上锁，然后设置槽位的窗口开始时间为当前窗口开始时间，并把槽位里面的数据重置
(old)
B0 B1 B2 NULL B4
|_______||_______|_______|_______|_______|_______||___
... 1200 1400 1600 1800 2000 2200 timestamp
^
time=1676So the above array array something like this:
WindowWrap array array of one example of the composition, WindowWrap statistical data by the examples which MetricBucket.
Then continue to return ArrayMetric details of the method, we finished top data.currentWindow(), now speaking againdata.list()
The method is also called the final list to the list LeapArray methods:
LeapArray # list
public List<WindowWrap<T>> list(long validTime) {
int size = array.length();
List<WindowWrap<T>> result = new ArrayList<WindowWrap<T>>(size);
for (int i = 0; i < size; i++) {
WindowWrap<T> windowWrap = array.get(i);
//如果windowWrap节点为空或者当前时间戳比windowWrap的窗口开始时间大超过60s，那么就跳过
//也就是说只要60s以内的数据
if (windowWrap == null || isWindowDeprecated(validTime, windowWrap)) {
continue;
}
result.add(windowWrap);
}
return result;
}This method is used inside the array are good statistics to find out node, and is not empty, and the current time is within 60 seconds of data.
Finally Constants. ENTRY_NODE .metrics () will return all the eligible statistical data node and pass aggregate method, traversing each node is set Resource MetricNode to TOTAL_IN_RESOURCE_NAME , a good package calls metricWriter.writethrough logs operation.
To sum up the initialization FlowRuleManager of what has been done:
- FlowRuleManager during initialization code block calls the static initialization
- ScheduledExecutorService calling thread pool within a static block, every 1 second run method is called once MetricTimerListener
- MetricTimerListener will call
Constants.ENTRY_NODE.metrics()timing statistics- StatisticNode call statistics, statistics for 60 seconds, and 60 seconds of data is divided into 60 small window
- In setting the current window if there is no data set directly, if the data exists and is the latest direct return, if it is old data, then reset the original statistics
- Each widget encapsulated by the data inside MetricBucket
- Finally, a good statistical data by metricWriter written to the log go
FlowRuleManager load rules
FlowRuleManager calling loadRules the rules loaded:
FlowRuleManager#loadRules
public static void loadRules(List<FlowRule> rules) {
currentProperty.updateValue(rules);
}currentProperty This example is carried out in a static loading block in which FlowRuleManager, we have said above, is an example of generation of DynamicSentinelProperty.
We enter into DynamicSentinelProperty of updateValue in:
public boolean updateValue(T newValue) {
//判断新的元素和旧元素是否相同
if (isEqual(value, newValue)) {
return false;
}
RecordLog.info("[DynamicSentinelProperty] Config will be updated to: " + newValue);
value = newValue;
for (PropertyListener<T> listener : listeners) {
listener.configUpdate(newValue);
}
return true;
}updateValue way is to check what is not the same rules already exist, so if there is no direct value equal to the new set of rules, then notifies all listeners to update the rules a little configuration.
currentProperty Examples disposed inside a listener will FlowPropertyListener listener instance FlowRuleManager initialize static block when, FlowPropertyListener FlowRuleManager inner class is:
private static final class FlowPropertyListener implements PropertyListener<List<FlowRule>> {
@Override
public void configUpdate(List<FlowRule> value) {
Map<String, List<FlowRule>> rules = FlowRuleUtil.buildFlowRuleMap(value);
if (rules != null) {
flowRules.clear();
//这个map的维度是key是Resource
flowRules.putAll(rules);
}
RecordLog.info("[FlowRuleManager] Flow rules received: " + flowRules);
}
....
}configUpdate first calls the FlowRuleUtil.buildFlowRuleMap（）method to all the rules by resource category, and then return to the sort map, then the original rule FlowRuleManager emptied into a new set of rules to flowRules go.
FlowRuleUtil # buildFlowRuleMap
this method will eventually call to another FlowRuleUtil overloaded method:
public static <K> Map<K, List<FlowRule>> buildFlowRuleMap(List<FlowRule> list, Function<FlowRule, K> groupFunction,
Predicate<FlowRule> filter, boolean shouldSort) {
Map<K, List<FlowRule>> newRuleMap = new ConcurrentHashMap<>();
if (list == null || list.isEmpty()) {
return newRuleMap;
}
Map<K, Set<FlowRule>> tmpMap = new ConcurrentHashMap<>();
for (FlowRule rule : list) {
//校验必要字段：资源名，限流阈值， 限流阈值类型，调用关系限流策略，流量控制效果等
if (!isValidRule(rule)) {
RecordLog.warn("[FlowRuleManager] Ignoring invalid flow rule when loading new flow rules: " + rule);
continue;
}
if (filter != null && !filter.test(rule)) {
continue;
}
//应用名，如果没有则会使用default
if (StringUtil.isBlank(rule.getLimitApp())) {
rule.setLimitApp(RuleConstant.LIMIT_APP_DEFAULT);
}
//设置拒绝策略：直接拒绝、Warm Up、匀速排队，默认是DefaultController
TrafficShapingController rater = generateRater(rule);
rule.setRater(rater);
//获取Resource名字
K key = groupFunction.apply(rule);
if (key == null) {
continue;
}
//根据Resource进行分组
Set<FlowRule> flowRules = tmpMap.get(key);
if (flowRules == null) {
// Use hash set here to remove duplicate rules.
flowRules = new HashSet<>();
tmpMap.put(key, flowRules);
}
flowRules.add(rule);
}
//根据ClusterMode LimitApp排序
Comparator<FlowRule> comparator = new FlowRuleComparator();
for (Entry<K, Set<FlowRule>> entries : tmpMap.entrySet()) {
List<FlowRule> rules = new ArrayList<>(entries.getValue());
if (shouldSort) {
// Sort the rules.
Collections.sort(rules, comparator);
}
newRuleMap.put(entries.getKey(), rules);
}
return newRuleMap;
}This method first passed in the check rule set is not empty, then traverse the rule set. Rule to be verified the necessary fields, if the incoming filters then filter check, then filtered rule resource is empty, the last rule of the same resource are put together after returning sorted.
Note that the default generated rater is DefaultController.
Here FlowRuleManager has been analyzed, and relatively long.