Author: Debugger
Link: https://juejin.cn/post/7181342523728592955
0 background
During the daily development of the company's SpringBoot project, it was found that the service startup process was extremely slow, and it often took 6-7 minutes to expose the port, seriously reducing development efficiency. Through the investigation of SpringBoot SpringApplicationRunListener, principle and source code debugging, it is found that there are big performance bottlenecks in the two stages of Bean scanning and Bean injection.BeanPostProcessor
Registering beans through JavaConfig reduces the scanning path of SpringBoot, and at the same time optimizes and transforms third-party dependencies based on the Springboot automatic configuration principle, reducing the local startup time of services from 7 minutes to about 40s. This article will cover the following knowledge points:
- Observe the SpringBoot startup run method based on the principle of SpringApplicationRunListener;
- Based on the principle of BeanPostProcessor, monitor the time-consuming of Bean injection;
- SpringBoot Cache automatic configuration principle;
- SpringBoot automatic configuration principle and starter transformation;
1 Time-consuming troubleshooting
SpringBoot service startup time-consuming troubleshooting, there are currently two ideas:
- Check the startup process of the SpringBoot service;
- Check the initialization time of beans;
The basics of Spring Boot will not be introduced. It is recommended to watch this free tutorial:
1.1 Observe SpringBoot start run method
The project uses XxBoot, an internal microservice component transformed based on SpringBoot, as the server implementation. Its startup process is similar to SpringBoot, and is divided ApplicationContextinto ApplicationContexttwo parts: construction and startup, that is, instantiate ApplicationContextthe object and call its runmethod to start the service:
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
public static ConfigurableApplicationContext run(Class<?>[] primarySources, String[] args) {
return new SpringApplication(primarySources).run(args);
}ApplicationContextThe object construction process mainly does custom banner settings, application type inference, configuration source settings, etc. Without special extensions, most projects are similar, and it is unlikely to cause time-consuming problems. This can also be verified by breaking the breakpoint in runthe method and running to the breakpoint position soon after startup.
The next step is to focus on troubleshooting. What are the performance bottlenecks during the startup process of runthe method ? The startup process of SpringBoot is very complicated. Fortunately, some mechanisms provided by SpringBoot itself divide the startup process of SpringBoot into multiple stages. The process of this stage division is reflected in SpringApplicationRunListenerthe interface ApplicationContext.run :
public interface SpringApplicationRunListener {
// run 方法第一次被执行时调用，早期初始化工作
void starting();
// environment 创建后，ApplicationContext 创建前
void environmentPrepared(ConfigurableEnvironment environment);
// ApplicationContext 实例创建，部分属性设置了
void contextPrepared(ConfigurableApplicationContext context);
// ApplicationContext 加载后，refresh 前
void contextLoaded(ConfigurableApplicationContext context);
// refresh 后
void started(ConfigurableApplicationContext context);
// 所有初始化完成后，run 结束前
void running(ConfigurableApplicationContext context);
// 初始化失败后
void failed(ConfigurableApplicationContext context, Throwable exception);
}At present, the built-in SpringApplicationRunListenerinterface has only one implementation class: EventPublishingRunListener, the function of this implementation class: through the event mechanism of the observer mode, events are triggered at different stages of runthe method Event, ApplicationListenerand the implementation classes trigger different business processes by listening to different Eventevent objects logic.
By customizing
ApplicationListenerthe implementation class, certain processing can be achieved at different stages of SpringBoot startup. It can be seen thatSpringApplicationRunListenerthe interfaceSpringBootbrings scalability to .
Here we don't need to delve into the functions EventPublishingRunListenerof , but we can use SpringApplicationRunListenerthe principle to add a custom implementation class, print the current time at the end of different stages, and calculate the running time of different stages to roughly locate which stages are more time-consuming , and then focus on troubleshooting the code in these stages.
First look SpringApplicationRunListenerat the implementation principle of , the logic of dividing different stages is reflected in the methodApplicationContext of :run
public ConfigurableApplicationContext run(String... args) {
...
// 加载所有 SpringApplicationRunListener 的实现类
SpringApplicationRunListeners listeners = getRunListeners(args);
// 调用了 starting
listeners.starting();
try {
ApplicationArguments applicationArguments = new DefaultApplicationArguments(args);
// 调用了 environmentPrepared
ConfigurableEnvironment environment = prepareEnvironment(listeners, applicationArguments);
configureIgnoreBeanInfo(environment);
Banner printedBanner = printBanner(environment);
context = createApplicationContext();
exceptionReporters = getSpringFactoriesInstances(SpringBootExceptionReporter.class, new Class[] { ConfigurableApplicationContext.class }, context);
// 内部调用了 contextPrepared、contextLoaded
prepareContext(context, environment, listeners, applicationArguments, printedBanner);
refreshContext(context);
afterRefresh(context, applicationArguments);
stopWatch.stop();
if (this.logStartupInfo) {
new StartupInfoLogger(this.mainApplicationClass).logStarted(getApplicationLog(), stopWatch);
}
// 调用了 started
listeners.started(context);
callRunners(context, applicationArguments);
}
catch (Throwable ex) {
// 内部调用了 failed
handleRunFailure(context, ex, exceptionReporters, listeners);
throw new IllegalStateException(ex);
}
try {
// 调用了 running
listeners.running(context);
}
catch (Throwable ex) {
handleRunFailure(context, ex, exceptionReporters, null);
throw new IllegalStateException(ex);
}
return context;
}rungetRunListeners(args)In the method, all the implementation classes configured in are loaded bySpringFactoriesLoader , instantiated by reflection, and stored in the local variable, whose type is ; then in different stages of the method the stage method calls of all stage methods of .classpathMETA-INF/spring.factotriesSpringApplicationRunListenerlistenersSpringApplicationRunListenersrunlistenersSpringApplicationRunListener
Therefore, as long as you write SpringApplicationRunListenera custom implementation class of , print the current time when implementing the methods in different stages of the interface; and META-INF/spring.factotriesafter configuring the class in , the class will also be instantiated and saved listenersin ; and print the end time at the end of different stages , to evaluate the execution time of different stages.
Add the implementation class to the project MySpringApplicationRunListener:
@Slf4j
public class MySpringApplicationRunListener implements SpringApplicationRunListener {
// 这个构造函数不能少，否则反射生成实例会报错
public MySpringApplicationRunListener(SpringApplication sa, String[] args) {
}
@Override
public void starting() {
log.info("starting {}", LocalDateTime.now());
}
@Override
public void environmentPrepared(ConfigurableEnvironment environment) {
log.info("environmentPrepared {}", LocalDateTime.now());
}
@Override
public void contextPrepared(ConfigurableApplicationContext context) {
log.info("contextPrepared {}", LocalDateTime.now());
}
@Override
public void contextLoaded(ConfigurableApplicationContext context) {
log.info("contextLoaded {}", LocalDateTime.now());
}
@Override
public void started(ConfigurableApplicationContext context) {
log.info("started {}", LocalDateTime.now());
}
@Override
public void running(ConfigurableApplicationContext context) {
log.info("running {}", LocalDateTime.now());
}
@Override
public void failed(ConfigurableApplicationContext context, Throwable exception) {
log.info("failed {}", LocalDateTime.now());
}
}Here (SpringApplication sa, String[] args) there must be no less constructors of the parameter type , because the source code restricts the use of constructors of this parameter type to generate instances through reflection.
Configure this class in the file under resourcesthe file :META-INF/spring.factotries
# Run Listeners
org.springframework.boot.SpringApplicationRunListener=\
com.xxx.ad.diagnostic.tools.api.MySpringApplicationRunListener
runIn the methodgetSpringFactoriesInstances, the implementation class configuredMETA-INF/spring.factotriesunder is obtained through the method, and the bottom layer is to rely on to obtain the fully qualified class name of the configured class, and then reflect to generate an instance;SpringApplicationRunListenerSpringFactoriesLoaderThis method is used a lot in SpringBoot, such as
EnableAutoConfiguration,ApplicationListener,ApplicationContextInitializerand so on.
Restart the service, observe the log output MySpringApplicationRunListenerof , and find that the main time-consuming is between the twocontextLoaded stages of and , and two methods are called between these two stages: and method , and the underlying call is one of the core methods of Spring initialization context One is this .startedrefreshContextafterRefreshrefreshContextAbstractApplicationContext#refreshrefresh
So far, it can be basically concluded that the reason for the high time consumption is to initialize the Spring context. However, this method is still very complicated. Fortunately, the refresh method also organizes the process of initializing the Spring context, and explains the role of each step in detail:
Through simple debugging, the cause of high time-consuming was quickly located:
- In
invokeBeanFactoryPostProcessors(beanFactory)the method , all registeredBeanFactorypost-processors are called; - Among them,
ConfigurationClassPostProcessorthis post-processor contributed most of the time-consuming; - Check relevant information, the post-processor is very important, mainly responsible for the analysis of annotations such as
@Configuration,@ComponentScan,@Import, and so on;@Bean - Continue to debug and find that the main time-consuming is spent on
@ComponentScanparsing , and the main time-consuming is still parsing attributesbasePackages;
That is, the attribute of @SpringBootApplicationthe annotation scanBasePackages:
Through the JavaDoc of this method and viewing the relevant codes, it is generally understood that the process is recursively scanning and parsing classes under basePackagesall paths, and generating objects that can be used as Beans BeanDefinition; if encountering @Configurationannotated configuration classes, recursively parsing them @ComponentScan. So far, the reason for the slow startup of the service has been found:
- As a data platform, our services refer to many third-party dependent services. These dependencies often provide complete functions of the corresponding business, so the provided jar package is very large;
- Scanning the classes under these package paths is very time-consuming, and many classes do not provide beans, but it still takes time to scan;
- Every time a service dependency is added, the scanning time will increase linearly;
After figuring out the reason for the time-consuming, I have 2 questions:
- Do all classes need to be scanned, can only those classes that provide Bean be scanned?
- Are all scanned beans needed? I only access one function, but all beans are injected, which seems unreasonable?
1.2 Monitor Bean injection time-consuming
The second optimization idea is to monitor the time-consuming initialization of all Bean objects, that is, the time spent on instantiation, initialization, and registration of each Bean object. Are there any Bean objects that are particularly time-consuming?
Similarly, we can use BeanPostProcessorthe interface to monitor the time-consuming of Bean injection, BeanPostProcessorwhich is the IOC hook provided by Spring before and after Bean initialization, which is used to execute some custom logic before and after Bean initialization:
public interface BeanPostProcessor {
// 初始化前
default Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
return bean;
}
// 初始化后
default Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
return bean;
}
}For the implementation class of BeanPostProcessorthe interface , its pre- and post-processing process is reflected in AbstractAutowireCapableBeanFactory#doCreateBeanthe method. This is also a very important method in Spring, which is used to instantiate the Bean object, and can be found by debugging all the way through BeanFactory#getBeanthe method . In this method initializeBeanthe method :
protected Object initializeBean(String beanName, Object bean, @Nullable RootBeanDefinition mbd) {
...
Object wrappedBean = bean;
if (mbd == null || !mbd.isSynthetic()) {
// 应用所有 BeanPostProcessor 的前置方法
wrappedBean = applyBeanPostProcessorsBeforeInitialization(wrappedBean, beanName);
}
try {
invokeInitMethods(beanName, wrappedBean, mbd);
}
catch (Throwable ex) {
throw new BeanCreationException(
(mbd != null ? mbd.getResourceDescription() : null),
beanName, "Invocation of init method failed", ex);
}
if (mbd == null || !mbd.isSynthetic()) {
// 应用所有 BeanPostProcessor 的后置方法
wrappedBean = applyBeanPostProcessorsAfterInitialization(wrappedBean, beanName);
}
return wrappedBean;
}Through BeanPostProcessorthe principle , record the current time during pre-processing, and subtract the pre-processing time from the current time during post-processing to know the initialization time of each bean. The following is my implementation:
@Component
public class TimeCostBeanPostProcessor implements BeanPostProcessor {
private Map<String, Long> costMap = Maps.newConcurrentMap();
@Override
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
costMap.put(beanName, System.currentTimeMillis());
return bean;
}
@Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
if (costMap.containsKey(beanName)) {
Long start = costMap.get(beanName);
long cost = System.currentTimeMillis() - start;
if (cost > 0) {
costMap.put(beanName, cost);
System.out.println("bean: " + beanName + "\ttime: " + cost);
}
}
return bean;
}
}BeanPostProcessorThe logic of is processed after Beanfactoryit is ready , so there is no need to SpringFactoriesLoaderload it through , it can be @Componentinjected .
Restart the service, check the Bean initialization process through the above methods, and really find something:
The initialization of this bean takes 43s. Looking at the initialization method of this bean, it is found that a large amount of configuration metadata will be queried from the database and updated to the Redis cache, so the initialization is very slow:
In addition, some service and controller objects that are not the project's own services are also found. These beans come from third-party dependencies: UPM services, which are not required in the project:
In fact, the reason has been mentioned above: I only access one function, but I inject all the beans under the service path, that is to say, the service injects beans of other services that are useless to itself.
The basics of Spring Boot will not be introduced. It is recommended to watch this free tutorial:
2 Optimization scheme
2.1 How to solve too many scanning paths?
The solution that comes to mind is relatively simple and crude:
Sort out the beans to be imported, delete the scanning path on the main configuration class, and use JavaConfig to explicitly inject manually.
Taking UPM's dependency as an example, the previous injection method is that the project depends on its UpmResourceClient object, and the Pom has referenced its Maven coordinates, and added its service path scanBasePackagesin : "com.xxx.ad.upm" , by scanning the classes under the entire service path, find UpmResourceClient and inject it. Because this class is annotated @Service, it will be injected into the Spring context of the service. The source code fragment and main configuration class of UpmResourceClient are as follows:
The transformation method using JavaConfig is: no longer scan the service path of UPM, but actively inject it. Delete "com.xxx.ad.upm", and add the following configuration class under the service path:
@Configuration
public class ThirdPartyBeanConfig {
@Bean
public UpmResourceClient upmResourceClient() {
return new UpmResourceClient();
}
}Tips: If the bean also depends on other beans, you need to inject all the dependent beans; it is more troublesome to sort out the scenarios with complex bean dependencies. Fortunately, the service bean dependencies used in the project are relatively simple, and some dependencies For complex services, it is observed that the path scanning time is not very high, so it will not be processed.
At the same time, through the on-demand injection of JavaConfig, there will be no redundant beans, and it will also help reduce the memory consumption of the service; it solves the above problem of introducing irrelevant upmService and upmController.
2.2 How to solve the high time consumption of Bean initialization?
Bean initialization takes a long time, so it needs to be handled case by case. For example, the problem of initializing configuration metadata encountered in the project can be solved by submitting the task to the thread pool for asynchronous processing or lazy loading.
3 new questions
After completing the above optimizations, the local startup time is reduced from about 7 minutes to 40 seconds, and the effect is still very significant. After the local self-test passed, it was released to the pre-release for verification. During the verification process, some students found that the Redis cache component connected to the project was invalid.
The access method of this component is similar to the access method described above. By adding the root path "com.xxx.ad.rediscache" of the scanning service, the corresponding Bean object is injected; check the source code of the cache component project and find that under this path There is a config class that injects a cache management object CacheManager, and its implementation class is RedisCacheManager:
Cache component code snippet:
In this optimization, I deleted one scan path each time , and after starting the service, sorted out and added dependent beans one by one according to the missing and error information in the startup log to ensure the normal startup of the service, and deleted " com.xxx.ad.rediscache", there is no abnormality in starting the service, so there is no further operation, and the pre-delivery verification is directly performed. This is strange, since the root path of the business code of the component is not scanned, and CacheManagerthe object , why is there no error reported where the cache is used?
Try to get an object of type ApplicationContextfrom CacheManagerto see if it exists? It turns out that RedisCacheManagerthe object :
In fact, the previous analysis is not wrong. The generated after deleting the scanning path RedisCacheManageris not configured in the cache component code, but generated by SpringBoot's automatic configuration, which means that the object is not what we want, and it does not meet expectations. , the reasons for which are described below.
3.1 SpringBoot automatic assembly is hard to guard against
Check the relevant information of SpringBoot Cache and find that SpringBoot Cache has done some automatic inference and injection work. It turns out that it is the pot of SpringBoot automatic assembly. Next, analyze the principle of SpringBoot Cache to clarify the reasons for the above problems.
The automatic configuration of SpringBoot is reflected @SpringBootApplicationin @EnableAutoConfiguration, which enables the automatic configuration function of SpringBoot. In this annotation, configure a series of *AutoConfiguration configuration classes under@Import(AutoConfigurationImportSelector.class) loading , infer from the existing conditions, and configure the required beans for us as much as possible. META-INF/spring.factotriesThese configuration classes are responsible for the automatic configuration of various functions, among which the automatic configuration class for SpringBoot Cache is CacheAutoConfiguration, and then focus on analyzing this configuration class.
@SpringBootApplicationThree very important annotations are integrated in the composite annotation:@SpringBootConfiguration,@EnableAutoConfiguration,@ComponentScan, among which@EnableAutoConfigurationis responsible for enabling the automatic configuration function;There are many annotations in SpringBoot , all of which are used to enable a certain aspect of the function, and the implementation principle is similar: through screening and importing automatic configuration classes that meet the conditions.
@EnableXXX@Import
You can see that there are many annotations CacheAutoConfigurationon , focus on it @Import({CacheConfigurationImportSelector.class}), and CacheConfigurationImportSelectorimplement ImportSelectorthe interface , which is used to dynamically select the configuration class to be imported. This is CacheConfigurationImportSelectorused to import the automatic configuration class of different types of Cache:
Through debugging, CacheConfigurationImportSelectorit is found that according to the cache type (CacheType) supported by SpringBoot, 10 kinds of cache automatic configuration classes are provided, sorted by priority, and only one takes effect in the end, and this project is exactly that RedisCacheConfiguration, which is provided internally RedisCacheManager, and the introduction of the first The three-party cache components are the same, so it causes confusion:
Take a look RedisCacheConfigurationat the implementation of :
There are many conditional annotations on this configuration class. When these conditions are met, this automatic configuration class will take effect, and this project happens to be satisfied. At the same time, the main configuration class of the project is also added, and the caching function is enabled, even if the cache @EnableCachingcomponent If it does not take effect, SpringBoot will automatically generate a cache management object;
That is: if the cache component service scan path exists, the code in the cache component generates a cache management object and @ConditionalOnMissingBean(CacheManager.class)fails; if the scan path does not exist, SpringBoot automatically generates a cache management object by inference.
This is also very easy to verify. If you break the point RedisCacheConfigurationin , if you don’t delete the scanning path, you can’t go to the SpringBoot automatic assembly process here (the cache component has been explicitly generated), and you can delete the scanning path (SpringBoot automatically generate).
@Import has been mentioned many times above, which is an important annotation in SpringBoot. It mainly has the following functions: 1. Import the class of
@Configurationthe annotation ; 2. Import the class that implements orImportSelector; 3. Import ordinary POJO.ImportBeanDefinitionRegistrar
3.2 Use the starter mechanism, out of the box
After understanding the cause of the cache failure, there is a solution. Because it is a component of your own team, there is no need to modify it by explicitly manually importing it through JavaConfig. Instead, you can optimize the implementation of the cache component through SpringBoot’s starter mechanism. Do automatic injection, out of the box.
Just modify the code of the cache component resources, add a META-INF/spring.factotriesfile in the file, and configure a EnableAutoConfigurationbelow , so that the project will also scan spring.factotriesthe file in this jar when it starts, and automatically import the XxxAdCacheConfigurationconfiguration class without scanning "com. xxx.ad.rediscache" the entire path:
# EnableAutoConfigurations
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
com.xxx.ad.rediscache.XxxAdCacheConfigurationThe
EnableAutoConfigurationautomatic is still relatively complicated. Before loading the automatic configuration class, the metadata of the automatic configuration must be loaded first, and the validity of all automatic configuration classes should be screened. For details, please refer to the relevant code of EnableAutoConfiguration;
Recent hot article recommendation:
1. 1,000+ Java interview questions and answers (2022 latest version)
2. Brilliant! Java coroutines are coming. . .
3. Spring Boot 2.x tutorial, too comprehensive!
5. The latest release of "Java Development Manual (Songshan Edition)", download quickly!
Feel good, don't forget to like + forward!