Soul网关源码阅读（九）- http长轮询同步数据

学习目标：

熟悉 soul 的数据同步机制中的 http 同步

学习内容：

1.http数据同步原理
2.配置
3.源码分析

学习时间：

2020年1月25号

学习产出：

1. http数据同步原理
   zookeeper、websocket 数据同步的机制比较简单，而 http 同步会相对复杂一些。Soul 借鉴了 Apollo、Nacos 的设计思想，取其精华，自己实现了 http 长轮询数据同步功能。注意，这里并非传统的 ajax 长轮询！
   
   soul-web 网管请求admin的配置服务，读取超是时间为90s，意味着网关层请求配置服务最多会等待90s，这样便于admin配置服务的及时相应变更数据，从而实现准实时推送。
   http请求到达soul-admin后，并非立马响应数据，而是利用Servlet3.0 的异步机制，异步响应数据。首先，将长轮询请求任务LongPollingClient 扔到 BlocingQueue中，并开启调度任务，60s后执行，这样做的目的是60s后将该长轮询请求移除队列，即便是这段时间内没有发生数据变更。因为即便是没有配置变更，也得让网关知道，总不能让其干等吧，而且网关请求配置服务时，也有 90s 的超时时间。

2. 配置
   admin：

   soul:
     sync:
         http:
           enabled: true
   

   soul-bootstrap配置

   soul:
   	  sync:
   	      http:
   	        url : http://localhost:9095
   

   pom依赖：

    <!--soul data sync start use http-->
           <dependency>
               <groupId>org.dromara</groupId>
               <artifactId>soul-spring-boot-starter-sync-data-http</artifactId>
               <version>${project.version}</version>
           </dependency>
   

3. 源码解析
   admin启动
   DataSyncConfiguration初始化对象：HttpLongPollingDataChangedListener

   更新数据信息到本地缓存：

    @Override
   public final void afterPropertiesSet() {
       updateAppAuthCache();
       updatePluginCache();
       updateRuleCache();
       updateSelectorCache();
       updateMetaDataCache();
       afterInitialize();
   }
   
   protected <T> void updateCache(final ConfigGroupEnum group, final List<T> data) {
           String json = GsonUtils.getInstance().toJson(data);
           ConfigDataCache newVal = new ConfigDataCache(group.name(), json, Md5Utils.md5(json), System.currentTimeMillis());
           ConfigDataCache oldVal = CACHE.put(newVal.getGroup(), newVal);
           log.info("update config cache[{}], old: {}, updated: {}", group, oldVal, newVal);
       }
   

   默认5分钟刷新一次本地缓存，可配置

   @Override
       protected void afterInitialize() {
           long syncInterval = httpSyncProperties.getRefreshInterval().toMillis();
           // Periodically check the data for changes and update the cache
           scheduler.scheduleWithFixedDelay(() -> {
               log.info("http sync strategy refresh config start.");
               try {
                   this.refreshLocalCache();
                   log.info("http sync strategy refresh config success.");
               } catch (Exception e) {
                   log.error("http sync strategy refresh config error!", e);
               }
           }, syncInterval, syncInterval, TimeUnit.MILLISECONDS);
           log.info("http sync strategy refresh interval: {}ms", syncInterval);
       }
   

DataChangedEventDispatcher 监听数据变更事件：

```
@Override
    public void afterPropertiesSet() {
        Collection<DataChangedListener> listenerBeans = applicationContext.getBeansOfType(DataChangedListener.class).values();
        this.listeners = Collections.unmodifiableList(new ArrayList<>(listenerBeans));
    }
```

soul-bootstrap 启动
HttpSyncDataConfiguration实例化HttpSyncDataService对象

public HttpSyncDataService(final HttpConfig httpConfig, final PluginDataSubscriber pluginDataSubscriber,
                               final List<MetaDataSubscriber> metaDataSubscribers, final List<AuthDataSubscriber> authDataSubscribers) {
        this.factory = new DataRefreshFactory(pluginDataSubscriber, metaDataSubscribers, authDataSubscribers);
        this.httpConfig = httpConfig;
        this.serverList = Lists.newArrayList(Splitter.on(",").split(httpConfig.getUrl()));
        this.httpClient = createRestTemplate();
        this.start();
    }
	private void start() {
        // It could be initialized multiple times, so you need to control that.
        if (RUNNING.compareAndSet(false, true)) {
            // fetch all group configs.
            this.fetchGroupConfig(ConfigGroupEnum.values());
            int threadSize = serverList.size();
            this.executor = new ThreadPoolExecutor(threadSize, threadSize, 60L, TimeUnit.SECONDS,
                    new LinkedBlockingQueue<>(),
                    SoulThreadFactory.create("http-long-polling", true));
            // start long polling, each server creates a thread to listen for changes.
            this.serverList.forEach(server -> this.executor.execute(new HttpLongPollingTask(server)));
        } else {
            log.info("soul http long polling was started, executor=[{}]", executor);
        }
    }

启动http
修改rule数据
触发： DataChangeTask

```
@Override
    protected void afterRuleChanged(final List<RuleData> changed, final DataEventTypeEnum eventType) {
        scheduler.execute(new DataChangeTask(ConfigGroupEnum.RULE));
    }
```

触发事件：

 private void publishEvent(final RuleDO ruleDO, final List<RuleConditionDTO> ruleConditions) {
        SelectorDO selectorDO = selectorMapper.selectById(ruleDO.getSelectorId());
        PluginDO pluginDO = pluginMapper.selectById(selectorDO.getPluginId());

        List<ConditionData> conditionDataList =
                ruleConditions.stream().map(ConditionTransfer.INSTANCE::mapToRuleDTO).collect(Collectors.toList());
        // publish change event.
        eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.RULE, DataEventTypeEnum.UPDATE,
                Collections.singletonList(RuleDO.transFrom(ruleDO, pluginDO.getName(), conditionDataList))));
    }

监听器：DataChangedEventDispatcher 监听到 RULE变化更新本地缓存

@Override
    @SuppressWarnings("unchecked")
    public void onApplicationEvent(final DataChangedEvent event) {
        for (DataChangedListener listener : listeners) {
            switch (event.getGroupKey()) {
                case APP_AUTH:
                    listener.onAppAuthChanged((List<AppAuthData>) event.getSource(), event.getEventType());
                    break;
                case PLUGIN:
                    listener.onPluginChanged((List<PluginData>) event.getSource(), event.getEventType());
                    break;
                case RULE:
                    listener.onRuleChanged((List<RuleData>) event.getSource(), event.getEventType());
                    break;
                case SELECTOR:
                    listener.onSelectorChanged((List<SelectorData>) event.getSource(), event.getEventType());
                    break;
                case META_DATA:
                    listener.onMetaDataChanged((List<MetaData>) event.getSource(), event.getEventType());
                    break;
                default:
                    throw new IllegalStateException("Unexpected value: " + event.getGroupKey());
            }
        }
    }

更新本地缓存之后HttpLongPollingDataChangedListener 监听到rule变化
触发：

@Override
    protected void afterRuleChanged(final List<RuleData> changed, final DataEventTypeEnum eventType) {
        scheduler.execute(new DataChangeTask(ConfigGroupEnum.RULE));
    }