6.Sentinel source code analysis -Sentinel is how to dynamically load the configuration of the current limiting?

Sentinel Source resolve Series:
1.Sentinel source code analysis -FlowRuleManager loaded rule did what?

2. Sentinel source code analysis -Sentinel How is traffic statistics?

3. Sentinel source code analysis - QPS traffic control is how to achieve?

4.Sentinel source code analysis - Sentinel is how to do downgrade?

How 5.Sentinel source code analysis -Sentinel adaptive current limit?

---

Sometimes we do not want to limit the flow of time-coded directly in the code inside, and then every time you want to change the rules, or rules increases the time to restart the application can only be resolved. But to be able to dynamically change the configuration, so that in case of an emergency can be configured dynamically modified. Such as two-eleven Taobao other services not 2018 a little problem, never expected to hang up first few minutes shopping cart service, this time it can limit the flow of emergency, to rescue applications.

In fact, after reading the foregoing, the dynamic configuration should be a natural thing, because all configuration changes is through the restrictor manager as FlowRuleManager internal listener to achieve, so long as the dynamic signal to the listener, it can be do dynamic changes to the configuration.

Next we look at Sentinel is how to do. Under normal circumstances, the dynamic configuration, there are two common implementations:

• Pull mode: The client periodically polls the initiative to a rule management center pulling rule, this rule may be the center of RDBMS, files, and even VCS and so on. The way to do this is simple, the disadvantage is unable to get change in a timely manner;
• Push mode: Center for unified rules push the client by way of registered listeners listening for changes, such as the use Nacos, Zookeeper and other distribution center. This approach has better real-time and ensure consistency.

The Sentinel both of which are currently supported:

• Pull-based: file, Consul (since 1.7.0)
• Push-based: ZooKeeper, Redis, Nacos, Apollo

So many supportive way, I am here only to explain the two types of files and ZooKeeper, corresponding to the push-pull mode.

Pull-based: file

First, the last example:
FlowRule.json

[
  {
    "resource": "abc",
    "controlBehavior": 0,
    "count": 20.0,
    "grade": 1,
    "limitApp": "default",
    "strategy": 0
  },
  {
    "resource": "abc1",
    "controlBehavior": 0,
    "count": 20.0,
    "grade": 1,
    "limitApp": "default",
    "strategy": 0
  }
]

SimpleFileDataSourceDemo:

public class SimpleFileDataSourceDemo {

    private static final String KEY = "abc";
    public static void main(String[] args) throws Exception {
        SimpleFileDataSourceDemo simpleFileDataSourceDemo = new SimpleFileDataSourceDemo();
        simpleFileDataSourceDemo.init();
        Entry entry = null;
        try {
            entry = SphU.entry(KEY);
            // dosomething
        } catch (BlockException e1) {
            // dosomething
        } catch (Exception e2) {
            // biz exception
        } finally {
            if (entry != null) {
                entry.exit();
            }
        }
    }
    private void init() throws Exception {
       String flowRulePath = "/Users/luozhiyun/Downloads/test/FlowRule.json";
        // Data source for FlowRule
        FileRefreshableDataSource<List<FlowRule>> flowRuleDataSource = new FileRefreshableDataSource<>(
                flowRulePath, flowRuleListParser);
        FlowRuleManager.register2Property(flowRuleDataSource.getProperty());
    }
    private Converter<String, List<FlowRule>> flowRuleListParser = source -> JSON.parseObject(source,
            new TypeReference<List<FlowRule>>() {});
}

This example is coded into a major resource file, and then read the contents inside the resource file, and then parse the contents of the file by setting rules parser resource custom.

Here we need to analyze is how FileRefreshableDataSource then load the file by FlowRuleManager registered.

FileRefreshableDataSource inheritance:

FileRefreshableDataSource

private static final int MAX_SIZE = 1024 * 1024 * 4;
private static final long DEFAULT_REFRESH_MS = 3000;
private static final int DEFAULT_BUF_SIZE = 1024 * 1024;
private static final Charset DEFAULT_CHAR_SET = Charset.forName("utf-8");

public FileRefreshableDataSource(String fileName, Converter<String, T> configParser) throws FileNotFoundException {
    this(new File(fileName), configParser, DEFAULT_REFRESH_MS, DEFAULT_BUF_SIZE, DEFAULT_CHAR_SET);
}

public FileRefreshableDataSource(File file, Converter<String, T> configParser, long recommendRefreshMs, int bufSize,
                                 Charset charset) throws FileNotFoundException {
    super(configParser, recommendRefreshMs);
    if (bufSize <= 0 || bufSize > MAX_SIZE) {
        throw new IllegalArgumentException("bufSize must between (0, " + MAX_SIZE + "], but " + bufSize + " get");
    }
    if (file == null || file.isDirectory()) {
        throw new IllegalArgumentException("File can't be null or a directory");
    }
    if (charset == null) {
        throw new IllegalArgumentException("charset can't be null");
    }
    this.buf = new byte[bufSize];
    this.file = file;
    this.charset = charset;
    // If the file does not exist, the last modified will be 0.
    this.lastModified = file.lastModified();
    firstLoad();
}

FileRefreshableDataSource constructor which will set various parameters, such as: modified buffer size, character code, the last file, the file update interval time.
This method will call the parent class constructor initializes AutoRefreshDataSource let's look at what has been done.

AutoRefreshDataSource

public AutoRefreshDataSource(Converter<S, T> configParser, final long recommendRefreshMs) {
    super(configParser);
    if (recommendRefreshMs <= 0) {
        throw new IllegalArgumentException("recommendRefreshMs must > 0, but " + recommendRefreshMs + " get");
    }
    this.recommendRefreshMs = recommendRefreshMs;
    startTimerService();
}

AutoRefreshDataSource constructor will start a call to the constructor of the superclass initialized as follows:
AbstractDataSource This

public AbstractDataSource(Converter<S, T> parser) {
    if (parser == null) {
        throw new IllegalArgumentException("parser can't be null");
    }
    this.parser = parser;
    this.property = new DynamicSentinelProperty<T>();
}

AbstractDataSource configuration to the two variables are set to values ​​parser and property, wherein the property is an example DynamicSentinelProperty.

We go back to AutoRefreshDataSource, the calls startTimerService method to open a timed schedule tasks after AutoRefreshDataSource after setting recommendRefreshMs parameter values.
AutoRefreshDataSource # startTimerService

private void startTimerService() {
    service = Executors.newScheduledThreadPool(1,
        new NamedThreadFactory("sentinel-datasource-auto-refresh-task", true));
    service.scheduleAtFixedRate(new Runnable() {
        @Override
        public void run() {
            try {
                if (!isModified()) {
                    return;
                }
                T newValue = loadConfig();
                getProperty().updateValue(newValue);
            } catch (Throwable e) {
                RecordLog.info("loadConfig exception", e);
            }
        }
    }, recommendRefreshMs, recommendRefreshMs, TimeUnit.MILLISECONDS);
}

public SentinelProperty<T> getProperty() {
    return property;
}

This method which will open a thread, the run method is called once every 3000ms. Lane will run method will first check what files have not been modified, if you call loadConfig to load the configuration, then call the getProperty method to get the parent class property is set to update the configuration.
Next we turn to explain these major approaches:

isModified method is a hook, the call is FileRefreshableDataSource isModified method:
FileRefreshableDataSource # isModified

protected boolean isModified() {
    long curLastModified = file.lastModified();
    if (curLastModified != this.lastModified) {
        this.lastModified = curLastModified;
        return true;
    }
    return false;
}

isModified view file every time there is not modified, and record it modified.

Next down is to call loadConfig load the file:
AbstractDataSource # loadConfig

public T loadConfig() throws Exception {
    return loadConfig(readSource());
}

public T loadConfig(S conf) throws Exception {
    T value = parser.convert(conf);
    return value;
}

FileRefreshableDataSource#readSource

public String readSource() throws Exception {
    if (!file.exists()) {
        // Will throw FileNotFoundException later.
        RecordLog.warn(String.format("[FileRefreshableDataSource] File does not exist: %s", file.getAbsolutePath()));
    }
    FileInputStream inputStream = null;
    try {
        inputStream = new FileInputStream(file);
        FileChannel channel = inputStream.getChannel();
        if (channel.size() > buf.length) {
            throw new IllegalStateException(file.getAbsolutePath() + " file size=" + channel.size()
                + ", is bigger than bufSize=" + buf.length + ". Can't read");
        }
        int len = inputStream.read(buf);
        return new String(buf, 0, len, charset);
    } finally {
        if (inputStream != null) {
            try {
                inputStream.close();
            } catch (Exception ignore) {
            }
        }
    }
}

LoadConfig realization method is very clear, first of all it is to call readSource stream file read by io, and then through the contents of the incoming file parser.

Then calls updateValue method DynamicSentinelProperty, walk the listener to update the configuration:
DynamicSentinelProperty # updateValue

public boolean updateValue(T newValue) {
    //判断新的元素和旧元素是否相同
    if (isEqual(value, newValue)) {
        return false;
    }
    RecordLog.info("[DynamicSentinelProperty] Config will be updated to: " + newValue);

    value = newValue;
    for (PropertyListener<T> listener : listeners) {
        listener.configUpdate(newValue);
    }
    return true;
}

Of course, when not loaded FlowRuleManager certainly no listener.

We finished loading the parent class FileRefreshableDataSource, we go back to FileRefreshableDataSource the constructor. Continue to go down calls firstLoad method to load configuration file to initialize the first time.
FileRefreshableDataSource # firstLoad

private void firstLoad() {
    try {
        T newValue = loadConfig();
        getProperty().updateValue(newValue);
    } catch (Throwable e) {
        RecordLog.info("loadConfig exception", e);
    }
}

Let's look at how FlowRuleManager is registered. Registration time will call register2Property method to register:

FlowRuleManager # register2Property

public static void register2Property(SentinelProperty<List<FlowRule>> property) {
    AssertUtil.notNull(property, "property cannot be null");
    synchronized (LISTENER) {
        RecordLog.info("[FlowRuleManager] Registering new property to flow rule manager");
        currentProperty.removeListener(LISTENER);
        property.addListener(LISTENER);
        currentProperty = property;
    }
}

This method is actually added a listener, and then replace FlowRuleManager of currentProperty to property flowRuleDataSource created. Then flowRuleDataSource timing inside thread calls about this every 3 seconds configUpdate method LISTENER refreshed rules, thus achieving a dynamic update rules.

Push-based：ZooKeeper

We first give an example:

public static void main(String[] args) { 
    final String remoteAddress = "127.0.0.1:2181";
    final String path = "/Sentinel-Demo/SYSTEM-CODE-DEMO-FLOW";
    ReadableDataSource<String, List<FlowRule>> flowRuleDataSource = new ZookeeperDataSource<>(remoteAddress, path,
            source -> JSON.parseObject(source, new TypeReference<List<FlowRule>>() {}));
    FlowRuleManager.register2Property(flowRuleDataSource.getProperty()); 
}

Here I define /Sentinel-Demo/SYSTEM-CODE-DEMO-FLOWthis path, if the contents of this path has changed, it will refresh rules.

We look at ZookeeperDataSource inheritance:

ZookeeperDataSource

public ZookeeperDataSource(final String serverAddr, final String path, Converter<String, T> parser) {
    super(parser);
    if (StringUtil.isBlank(serverAddr) || StringUtil.isBlank(path)) {
        throw new IllegalArgumentException(String.format("Bad argument: serverAddr=[%s], path=[%s]", serverAddr, path));
    }
    this.path = path;

    init(serverAddr, null);
}

AbstractDataSource

public AbstractDataSource(Converter<S, T> parser) {
    if (parser == null) {
        throw new IllegalArgumentException("parser can't be null");
    }
    this.parser = parser;
    this.property = new DynamicSentinelProperty<T>();
}

ZookeeperDataSource will first call the parent class parameter set, calls the init method after completion of the check.

ZookeeperDataSource#init

private void init(final String serverAddr, final List<AuthInfo> authInfos) {
    initZookeeperListener(serverAddr, authInfos);
    loadInitialConfig();
}

ZookeeperDataSource#initZookeeperListener

    private void initZookeeperListener(final String serverAddr, final List<AuthInfo> authInfos) {
        try {
            //设置监听
            this.listener = new NodeCacheListener() {
                @Override
                public void nodeChanged() {

                    try {
                        T newValue = loadConfig();
                        RecordLog.info(String.format("[ZookeeperDataSource] New property value received for (%s, %s): %s",
                                serverAddr, path, newValue));
                        // Update the new value to the property.
                        getProperty().updateValue(newValue);
                    } catch (Exception ex) {
                        RecordLog.warn("[ZookeeperDataSource] loadConfig exception", ex);
                    }
                }
            };

            String zkKey = getZkKey(serverAddr, authInfos);
            if (zkClientMap.containsKey(zkKey)) {
                this.zkClient = zkClientMap.get(zkKey);
            } else {
                //如果key不存在，那么就加锁设值
                synchronized (lock) {
                    if (!zkClientMap.containsKey(zkKey)) {
                        CuratorFramework zc = null;
                        //根据不同的条件获取client
                        if (authInfos == null || authInfos.size() == 0) {
                            zc = CuratorFrameworkFactory.newClient(serverAddr, new ExponentialBackoffRetry(SLEEP_TIME, RETRY_TIMES));
                        } else {
                            zc = CuratorFrameworkFactory.builder().
                                    connectString(serverAddr).
                                    retryPolicy(new ExponentialBackoffRetry(SLEEP_TIME, RETRY_TIMES)).
                                    authorization(authInfos).
                                    build();
                        }
                        this.zkClient = zc;
                        this.zkClient.start();
                        Map<String, CuratorFramework> newZkClientMap = new HashMap<>(zkClientMap.size());
                        newZkClientMap.putAll(zkClientMap);
                        newZkClientMap.put(zkKey, zc);
                        zkClientMap = newZkClientMap;
                    } else {
                        this.zkClient = zkClientMap.get(zkKey);
                    }
                }
            }
            //为节点添加watcher
            //监听数据节点的变更，会触发事件
            this.nodeCache = new NodeCache(this.zkClient, this.path);
            this.nodeCache.getListenable().addListener(this.listener, this.pool);
            this.nodeCache.start();
        } catch (Exception e) {
            RecordLog.warn("[ZookeeperDataSource] Error occurred when initializing Zookeeper data source", e);
            e.printStackTrace();
        }
    }

This method is mainly used to create client value and set up monitoring, are conventional operation zk, unfamiliar, you can go and see how Curator is used.

private void loadInitialConfig() {
    try {
        //调用父类的loadConfig方法
        T newValue = loadConfig();
        if (newValue == null) {
            RecordLog.warn("[ZookeeperDataSource] WARN: initial config is null, you may have to check your data source");
        }
        getProperty().updateValue(newValue);
    } catch (Exception ex) {
        RecordLog.warn("[ZookeeperDataSource] Error when loading initial config", ex);
    }
}

UpdateValue called once after the client after setting values ​​zk and monitoring, information node is first loaded.

AbstractDataSource

public T loadConfig() throws Exception {
    return loadConfig(readSource());
}

public T loadConfig(S conf) throws Exception {
    T value = parser.convert(conf);
    return value;
}

LoadConfig parent class will call subclass readSource read configuration information, and then calls the deserialize parser.convert.

ZookeeperDataSource#readSource

public String readSource() throws Exception {
    if (this.zkClient == null) {
        throw new IllegalStateException("Zookeeper has not been initialized or error occurred");
    }
    String configInfo = null;
    ChildData childData = nodeCache.getCurrentData();
    if (null != childData && childData.getData() != null) {

        configInfo = new String(childData.getData());
    }
    return configInfo;
}

This method is used to read the information inside the node zk.

The last method FlowRuleManager.register2Property on and above the dynamic configuration file is the same as the.

java study notes / SENTINEL