1. Conclusion:
The dynamic effect mechanism of Apollo configuration center is based on Http long polling request and Spring extension mechanism . During the startup of Spring container, Apollo uses custom BeanPostProcessor and BeanFactoryPostProcessor to include ${…} placeholders and @Value in the parameters. The annotated Bean is registered in the registry defined in the Apollo framework . Then use Http long polling to continuously obtain the configuration information of the server . Once the configuration changes, Apollo will find the corresponding Bean according to the changed configuration Key, and then modify the Bean's attributes , thus realizing the feature of dynamic configuration effect.
It should be noted that Apollo can only modify the properties of the Bean after the configuration changes. For example, the properties of our data source have changed. The newly created Connection object is okay, but the information about the Connection object that has been created in the connection pool cannot be Dynamically modified, so the application still needs to be restarted.
Second, the analysis process
2.1, important concepts in Spring
Before understanding the implementation principle of the Apollo configuration center, we need to be familiar with several important concepts in the Spring framework:
1. BeanDefinition is
used to describe the configuration information of the Bean. Bean configuration generally has three ways:
(1) XML configuration file
(2) ) @Service, @Component and other annotations
(3)
The BeanDefinition implementation class corresponding to the Java Config method is as shown in the figure below. When the Spring container starts, all Bean configuration information will be converted into BeanDefinition objects.
2. BeanDefinitionRegistry
BeanDefinition container, all Bean definitions are registered in the BeanDefinitionRegistry object.
3. PropertySource is
used to store Spring configuration resource information. For example, the properties or yaml file configuration information in the spring project will be stored in the PropertySource object. Spring supports the @PropertySource annotation to load configuration information into the Environment object.
4, ImportBeanDefinitionRegistrar
ImportBeanDefinitionRegistrar is an interface, the implementation of the interface is used in the Spring parsing Bean configuration to generate BeanDefinition object stage. When Spring parses Configuration annotations, additional BeanDefinitions are added to the Spring container.
5. BeanFactoryPostProcessor
Bean factory post processor, used to modify the Bean factory information after the registration of the BeanDefinition object, such as adding or modifying the BeanDefinition object.
6, BeanDefinitionRegistryPostProcessor
It is a special BeanFactoryPostProcessor, used to access, add or modify BeanDefinition information after the registration of the BeanDefinition object is completed.
7, PropertySourcesPlaceholderConfigurer
It is a special BeanFactoryPostProcessor, used to parse the ${...} parameter placeholders in the Bean configuration.
8. BeanPostProcessor
Bean post processor, before and after the bean initialization method is called, the interception logic is executed, the original Bean can be packaged or the proxy object can be created according to the mark interface.
2.2 Review of the startup process of the Spring framework
The Spring framework starts roughly through the following stages:
1. Analyze the Bean configuration information, convert the configuration information into a BeanDefinition object, and register it in the BeanDefinitionRegistry.
2. Execute all the postProcessBeanFactory() methods of BeanFactoryPostProcessor to modify the Bean factory information, including modifying or adding BeanDefinition objects.
Note: If you need to control the execution order of BeanFactoryPostProcessor, you need to implement the PriorityOrdered interface. The smaller the value returned by the getOrder() method, the higher the execution priority.
3. Instantiate all Beans through the BeanDefinition object and inject dependencies.
4. Execute the postProcessBeforeInitialization() method of all BeanPostProcessor objects.
5. Execute the initialization method of the Bean, such as the afterPropertiesSet method of the InitializingBean interface, or the method specified by the init-method attribute.
Execute the postProcessAfterInitialization() method of all BeanPostProcessor objects
2.3, Apollo principle analysis
The Apollo framework is very simple to use. If it is a Spring Boot project, you only need to add the @EnableApolloConfig annotation to the startup class. E.g:
@SpringBootApplication
@EnableApolloConfig
public class Application {
public static void main(String[] args) {
ParserConfig.getGlobalInstance().setAutoTypeSupport(true);
SpringApplication.run(Application.class, args);
}
}So what exactly did the @EnableApolloConfig annotation do? We can look at the definition of the EnableApolloConfig annotation. The code is as follows
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
@Documented
@Import(ApolloConfigRegistrar.class)
public @interface EnableApolloConfig {
/**
* Apollo namespaces to inject configuration into Spring Property Sources.
*
* @return Namespace 名字的集合
*/
String[] value() default {ConfigConsts.NAMESPACE_APPLICATION};
/**
* The order of the apollo config, default is {@link Ordered#LOWEST_PRECEDENCE}, which is Integer.MAX_VALUE.
* If there are properties with the same name in different apollo configs, the apollo config with smaller order wins.
* @return 优先级
*/
int order() default Ordered.LOWEST_PRECEDENCE;
}As shown in the code above, in the EnableApolloConfig annotation, an ApolloConfigRegistrar is imported through the @Import annotation . Next, let’s take a look:
public class ApolloConfigRegistrar implements ImportBeanDefinitionRegistrar {
private ApolloConfigRegistrarHelper helper = ServiceBootstrap.loadPrimary(ApolloConfigRegistrarHelper.class);
@Override
public void registerBeanDefinitions(AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry registry) {
helper.registerBeanDefinitions(importingClassMetadata, registry);
}
}Next, look at the implementation of helper.registerBeanDefinitions(importingClassMetadata, registry), which is the method of DefaultApolloConfigRegistrarHelper, the code is as follows:
@Override
public void registerBeanDefinitions(AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry registry) {
// 解析 @EnableApolloConfig 注解
AnnotationAttributes attributes = AnnotationAttributes
.fromMap(importingClassMetadata.getAnnotationAttributes(EnableApolloConfig.class.getName()));
String[] namespaces = attributes.getStringArray("value");
int order = attributes.getNumber("order");
// 添加到 PropertySourcesProcessor 中
PropertySourcesProcessor.addNamespaces(Lists.newArrayList(namespaces), order);
Map<String, Object> propertySourcesPlaceholderPropertyValues = new HashMap<>();
// to make sure the default PropertySourcesPlaceholderConfigurer's priority is higher than PropertyPlaceholderConfigurer
propertySourcesPlaceholderPropertyValues.put("order", 0);
// 注册 PropertySourcesPlaceholderConfigurer 到 BeanDefinitionRegistry 中，替换 PlaceHolder 为对应的属性值，
// 参考文章 https://leokongwq.github.io/2016/12/28/spring-PropertyPlaceholderConfigurer.html
BeanRegistrationUtil.registerBeanDefinitionIfNotExists(registry, PropertySourcesPlaceholderConfigurer.class.getName(),
PropertySourcesPlaceholderConfigurer.class, propertySourcesPlaceholderPropertyValues);
//【差异】注册 PropertySourcesProcessor 到 BeanDefinitionRegistry 中，因为可能存在 XML 配置的 Bean ，用于 PlaceHolder 自动更新机制
BeanRegistrationUtil.registerBeanDefinitionIfNotExists(registry, PropertySourcesProcessor.class.getName(),
PropertySourcesProcessor.class);
// 注册 ApolloAnnotationProcessor 到 BeanDefinitionRegistry 中，解析 @ApolloConfig 和 @ApolloConfigChangeListener 注解。
BeanRegistrationUtil.registerBeanDefinitionIfNotExists(registry, ApolloAnnotationProcessor.class.getName(),
ApolloAnnotationProcessor.class);
// 注册 SpringValueProcessor 到 BeanDefinitionRegistry 中，用于 PlaceHolder 自动更新机制
BeanRegistrationUtil.registerBeanDefinitionIfNotExists(registry, SpringValueProcessor.class.getName(),
SpringValueProcessor.class);
//【差异】注册 SpringValueDefinitionProcessor 到 BeanDefinitionRegistry 中，因为可能存在 XML 配置的 Bean ，用于 PlaceHolder 自动更新机制
BeanRegistrationUtil.registerBeanDefinitionIfNotExists(registry, SpringValueDefinitionProcessor.class.getName(),
SpringValueDefinitionProcessor.class);
// 注册 ApolloJsonValueProcessor 到 BeanDefinitionRegistry 中，解析 @ApolloJsonValue 注解。
BeanRegistrationUtil.registerBeanDefinitionIfNotExists(registry, ApolloJsonValueProcessor.class.getName(),
ApolloJsonValueProcessor.class);
}As shown in the code above, ApolloConfigRegistrar implements the ImportBeanDefinitionRegistrar interface. As mentioned earlier, the implementation class of the ImportBeanDefinitionRegistrar interface is used in the phase of Spring parsing Bean configuration to generate BeanDefinition objects . When Spring parses Configuration annotations, add additional BeanDefinition to the Spring container. .
Several BeanDefinitions are registered in ApolloConfigRegistrar, as follows:
1. PropertySourcesPlaceholderConfigurer -------->BeanFactoryPostProcessor
2. PropertySourcesProcessor -------->BeanFactoryPostProcessor
3. ApolloAnnotationProcessor -------->BeanPostProcessor
4. SpringValueProcessor -------->BeanFactoryPostProcessor and BeanPostProcessor
5. SpringValueDefinitionProcessor-------->BeanDefinitionRegistryPostProcessor (ie BeanFactoryPostProcessor)
6. ApolloJsonValueProcessor -------->BeanPostProcessor
These classes either implement the BeanFactoryPostProcessor interface or the BeanPostProcessor interface. As mentioned earlier, BeanFactoryPostProcessor and BeanPostProcessor are extension mechanisms provided by Spring, and BeanFactoryPostProcessor must be executed before BeanPostProcessor.
Next, let's take a look at the execution order of these custom BeanFactoryPostProcessor and BeanPostProcessor, and what they do.
Custom BeanFactoryPostProcessor
1, SpringValueDefinitionProcessor
To traverse all BeanDefinitions, add the attributes containing the ${...} parameter placeholders in the attributes to the Apollo attribute registry. The specific structure of the Apollo attribute registry is as follows:
2. PropertySourcesProcessor
(1) Obtain configuration information from the configuration center according to the namespace, and create RemoteConfigRepository and LocalFileConfigRepository objects. RemoteConfigRepository represents remote configuration center resources, and LocalFileConfigRepository represents local cache configuration resources.
(2) The LocalFileConfigRepository object caches configuration information to the C:\opt\data or /opt/data directory.
(3) RemoteConfigRepository opens the HTTP long polling request timing task, which is requested once every 2s by default.
(4) Convert the local cache configuration information into a PropertySource object (Apollo customizes Spring's PropertySource) and load it into Spring's Environment object.
(5) Register the custom ConfigPropertySource as an observer. Once RemoteConfigRepository finds that the remote configuration center information has changed, the ConfigPropertySource object will be notified.
3. The PropertySourcesPlaceholderConfigurer
loads the local Properties file and replaces the ${...} parameter placeholders with specific values.
4. The SpringValueProcessor is
only to obtain the BeanDefinition containing the ${...} parameter placeholders obtained in the SpringValueDefinitionProcessor. (From the perspective of the object-oriented design principle, it does not conform to the single responsibility principle. It can be registered in the Guice container and then obtained from the Guice container.)
Custom BeanPostProcessor
5, ApolloJsonValueProcessor
processes ApolloJsonValue annotations, properties or methods include ApolloJsonValue annotated beans, the property values will also change according to the configuration center configuration modification, so it also needs to be added to the configuration center configurable container
6. ApolloAnnotationProcessor
processes the ApolloConfigChangeListener annotation, and the ApolloConfigChangeListener annotation is used to register a configuration change listener.
7. SpringValueProcessor
processes the Value annotations in Spring, and adds the Bean information containing Value annotations in properties or methods to the Apollo property registry.
Long polling
8. Initialize in the constructor of the RemoteConfigRepository class as follows:
## RemoteConfigRepository 的方法
private void scheduleLongPollingRefresh() {
remoteConfigLongPollService.submit(m_namespace, this);
}
## 进入 RemoteConfigLongPollService 的submit 方法
public boolean submit(String namespace, RemoteConfigRepository remoteConfigRepository) {
// 添加到 m_longPollNamespaces 中
boolean added = m_longPollNamespaces.put(namespace, remoteConfigRepository);
// 添加到 m_notifications 中
m_notifications.putIfAbsent(namespace, INIT_NOTIFICATION_ID);
// 若未启动长轮询定时任务，进行启动
if (!m_longPollStarted.get()) {
startLongPolling();
}
return added;
}The whole process is shown in the figure below:
2.4, springboot integrated Apollo loading analysis
After springboot is started, the ApolloApplicationContextInitializer class will be loaded , and the initialize() method will be called . The source code is located in apollo-client . The calling process is as follows:
/**
* springboot 加载 Apollo 的入口
* @param context
*/
@Override
public void initialize(ConfigurableApplicationContext context) {
ConfigurableEnvironment environment = context.getEnvironment();
String enabled = environment.getProperty(PropertySourcesConstants.APOLLO_BOOTSTRAP_ENABLED, "false");
if (!Boolean.valueOf(enabled)) {
logger.debug("Apollo bootstrap config is not enabled for context {}, see property: ${
{}}", context, PropertySourcesConstants.APOLLO_BOOTSTRAP_ENABLED);
return;
}
logger.debug("Apollo bootstrap config is enabled for context {}", context);
initialize(environment);
}
/**
* Initialize Apollo Configurations Just after environment is ready.
*
* @param environment
*/
protected void initialize(ConfigurableEnvironment environment) {
if (environment.getPropertySources().contains(PropertySourcesConstants.APOLLO_BOOTSTRAP_PROPERTY_SOURCE_NAME)) {
//already initialized
return;
}
String namespaces = environment.getProperty(PropertySourcesConstants.APOLLO_BOOTSTRAP_NAMESPACES, ConfigConsts.NAMESPACE_APPLICATION);
logger.debug("Apollo bootstrap namespaces: {}", namespaces);
List<String> namespaceList = NAMESPACE_SPLITTER.splitToList(namespaces);
CompositePropertySource composite = new CompositePropertySource(PropertySourcesConstants.APOLLO_BOOTSTRAP_PROPERTY_SOURCE_NAME);
for (String namespace : namespaceList) {
// 获取配置
Config config = ConfigService.getConfig(namespace);
composite.addPropertySource(configPropertySourceFactory.getConfigPropertySource(namespace, config));
}
environment.getPropertySources().addFirst(composite);
}Note here Config config = ConfigService.getConfig(namespace); code
1. Call ConfigService .getConfig
/**
* Get the config instance for the namespace. 1.调用ConfigService.getService
*
* @param namespace the namespace of the config
* @return config instance
*/
public static Config getConfig(String namespace) {
return s_instance.getManager().getConfig(namespace);
}2. The ConfigManager interface pointed to by s_instance.getManager() is implemented by DefaultConfigManager.getConfig ():
/**
* 2.DefaultConfigManager.getConfig
* @param namespace the namespace
* @return
*/
@Override
public Config getConfig(String namespace) {
Config config = m_configs.get(namespace);
if (config == null) {
synchronized (this) {
config = m_configs.get(namespace);
if (config == null) {
ConfigFactory factory = m_factoryManager.getFactory(namespace);
config = factory.create(namespace);
m_configs.put(namespace, config);
}
}
}
return config;
}3. The factory of factory.create(namespace) points to the ConfigFactory interface, which is implemented by DefaultConfigFactory.create (String namespace):
/**
* 3.DefaultConfigFactory.create(String namespace)
* @param namespace the namespace
* @return
*/
@Override
public Config create(String namespace) {
ConfigFileFormat format = determineFileFormat(namespace);
if (ConfigFileFormat.isPropertiesCompatible(format)) {
return new DefaultConfig(namespace, createPropertiesCompatibleFileConfigRepository(namespace, format));
}
// 4.createLocalConfigRepository-->new LocalFileConfigRepository(namespace, createRemoteConfigRepository(namespace));
return new DefaultConfig(namespace, createLocalConfigRepository(namespace));
}4. createLocalConfigRepository-->new LocalFileConfigRepository (namespace, createRemoteConfigRepository (namespace)); There are two constructors, which are executed according to the specific scenario.
/**
* 4.createLocalConfigRepository-->new LocalFileConfigRepository(namespace, createRemoteConfigRepository(namespace));
* @param namespace
* @return
*/
LocalFileConfigRepository createLocalConfigRepository(String namespace) {
if (m_configUtil.isInLocalMode()) {
logger.warn(
"==== Apollo is in local mode! Won't pull configs from remote server for namespace {} ! ====",
namespace);
return new LocalFileConfigRepository(namespace);
}
//5.调用 LocalFileConfigRepository的构造方法 --> RemoteConfigRepository
return new LocalFileConfigRepository(namespace, createRemoteConfigRepository(namespace));
}
//============================================================================
public LocalFileConfigRepository(String namespace) {
this(namespace, null);
}
// ==========================================================================
public LocalFileConfigRepository(String namespace, ConfigRepository upstream) {
m_namespace = namespace;
m_configUtil = ApolloInjector.getInstance(ConfigUtil.class);
this.setLocalCacheDir(findLocalCacheDir(), false);
//
this.setUpstreamRepository(upstream);
// 尝试同步
this.trySync();
}
5. At this time, see createRemoteConfigRepository (namespace), call the localFileConfigRepository construction method --> RemoteConfigRepository
// DefaultConfigFactory 类
RemoteConfigRepository createRemoteConfigRepository(String namespace) {
return new RemoteConfigRepository(namespace);
}
6. RemoteConfigRepository constructor call, performs synchronous operation and timing of the operation
/**
* Constructor. RemoteConfigRepository 类
*
* @param namespace the namespace
*/
public RemoteConfigRepository(String namespace) {
m_namespace = namespace;
m_configCache = new AtomicReference<>();
m_configUtil = ApolloInjector.getInstance(ConfigUtil.class);
m_httpUtil = ApolloInjector.getInstance(HttpUtil.class);
m_serviceLocator = ApolloInjector.getInstance(ConfigServiceLocator.class);
remoteConfigLongPollService = ApolloInjector.getInstance(RemoteConfigLongPollService.class);
m_longPollServiceDto = new AtomicReference<>();
m_remoteMessages = new AtomicReference<>();
m_loadConfigRateLimiter = RateLimiter.create(m_configUtil.getLoadConfigQPS());
m_configNeedForceRefresh = new AtomicBoolean(true);
m_loadConfigFailSchedulePolicy = new ExponentialSchedulePolicy(m_configUtil.getOnErrorRetryInterval(),
m_configUtil.getOnErrorRetryInterval() * 8);
gson = new Gson();
this.trySync(); // 尝试同步配置
this.schedulePeriodicRefresh(); // 初始化定时刷新配置的任务
//长轮询的开启入口
this.scheduleLongPollingRefresh(); // 注册自己到 RemoteConfigLongPollService 中，实现配置更新的实时通知
}Finally, it began long polling of implementation.
Summary: The
dynamic effect mechanism of Apollo configuration center is based on Http long polling request and Spring extension mechanism. During the startup process of Spring container, Apollo uses custom BeanPostProcessor and BeanFactoryPostProcessor to include ${...} placeholders and The @Value annotated bean is registered in the registry defined in the Apollo framework. Then use Http long polling to continuously obtain the configuration information of the server. Once the configuration changes, Apollo will find the corresponding Bean according to the changed configuration Key, and then modify the Bean's attributes, thus realizing the feature of dynamic configuration effect.
It should be noted that Apollo can only modify the properties of the Bean after the configuration changes. For example, the properties of our data source have changed. The newly created Connection object is no problem, but the related information of the Connection object that has been created in the connection pool cannot be changed. Dynamically modified, so the application still needs to be restarted .
Three, feel
The conclusion is pretty good, so I'm hereby extracting it for notes. . . . . . At the same time, debug it yourself with dugger. . . . .
https://my.oschina.net/u/4277049/blog/3638370