【Soul源码阅读】番外1.WebFlux服务器启动流程

Soul 网关是基于 WebFlux 编写的，一直想搞懂响应式编程服务器到底是怎么运行的，找了本书看了下，又 debug 了下源码，才有此文章，记录如下。

WebFlux 服务器启动流程

结合 Spring Boot 的启动流程讲解 WebFlux 服务启动流程，首先看一下启动时序图：

1.run

    // SpringApplication.java
	/**
	 * Static helper that can be used to run a {@link SpringApplication} from the
	 * specified sources using default settings and user supplied arguments.
	 * @param primarySources the primary sources to load
	 * @param args the application arguments (usually passed from a Java main method)
	 * @return the running {@link ApplicationContext}
	 */
	public static ConfigurableApplicationContext run(Class<?>[] primarySources, String[] args) {
		return new SpringApplication(primarySources).run(args);
	}

    // SpringApplication.java
    /**
	 * Run the Spring application, creating and refreshing a new
	 * {@link ApplicationContext}.
	 * @param args the application arguments (usually passed from a Java main method)
	 * @return a running {@link ApplicationContext}
	 */
	public ConfigurableApplicationContext run(String... args) {
		StopWatch stopWatch = new StopWatch();
		stopWatch.start();
		ConfigurableApplicationContext context = null;
		Collection<SpringBootExceptionReporter> exceptionReporters = new ArrayList<>();
		configureHeadlessProperty();
		SpringApplicationRunListeners listeners = getRunListeners(args);
		listeners.starting();
		try {
			ApplicationArguments applicationArguments = new DefaultApplicationArguments(args);
			ConfigurableEnvironment environment = prepareEnvironment(listeners, applicationArguments);
			configureIgnoreBeanInfo(environment);
			Banner printedBanner = printBanner(environment);
			context = createApplicationContext();
			exceptionReporters = getSpringFactoriesInstances(SpringBootExceptionReporter.class,
					new Class[] { ConfigurableApplicationContext.class }, context);
			prepareContext(context, environment, listeners, applicationArguments, printedBanner);
			refreshContext(context);
			afterRefresh(context, applicationArguments);
			stopWatch.stop();
			if (this.logStartupInfo) {
				new StartupInfoLogger(this.mainApplicationClass).logStarted(getApplicationLog(), stopWatch);
			}
			listeners.started(context);
			callRunners(context, applicationArguments);
		}
		catch (Throwable ex) {
			handleRunFailure(context, ex, exceptionReporters, listeners);
			throw new IllegalStateException(ex);
		}

		try {
			listeners.running(context);
		}
		catch (Throwable ex) {
			handleRunFailure(context, ex, exceptionReporters, null);
			throw new IllegalStateException(ex);
		}
		return context;
	}

2.createApplicationContext

21行，createApplicationContext()，创建应用程序上下文 ConfigurableApplicationContext

    // SpringApplication.java
    /**
	 * Strategy method used to create the {@link ApplicationContext}. By default this
	 * method will respect any explicitly set application context or application context
	 * class before falling back to a suitable default.
	 * @return the application context (not yet refreshed)
	 * @see #setApplicationContextClass(Class)
	 */
	protected ConfigurableApplicationContext createApplicationContext() {
		Class<?> contextClass = this.applicationContextClass;
		if (contextClass == null) {
			try {
                // 环境类型
				switch (this.webApplicationType) {
				case SERVLET:
                    // Web servlet环境
					contextClass = Class.forName(DEFAULT_SERVLET_WEB_CONTEXT_CLASS);
					break;
				case REACTIVE:
                    //  Web Reactive环境
					contextClass = Class.forName(DEFAULT_REACTIVE_WEB_CONTEXT_CLASS);
					break;
				default:
                    // 非Web环境
					contextClass = Class.forName(DEFAULT_CONTEXT_CLASS);
				}
			}
			catch (ClassNotFoundException ex) {
				throw new IllegalStateException(
						"Unable create a default ApplicationContext, please specify an ApplicationContextClass", ex);
			}
		}
		return (ConfigurableApplicationContext) BeanUtils.instantiateClass(contextClass);
	}

	/**
	 * The class name of application context that will be used by default for web
	 * environments.
	 */
	public static final String DEFAULT_SERVLET_WEB_CONTEXT_CLASS = "org.springframework.boot."
			+ "web.servlet.context.AnnotationConfigServletWebServerApplicationContext";

	/**
	 * The class name of application context that will be used by default for reactive web
	 * environments.
	 */
	public static final String DEFAULT_REACTIVE_WEB_CONTEXT_CLASS = "org.springframework."
			+ "boot.web.reactive.context.AnnotationConfigReactiveWebServerApplicationContext";

	/**
	 * The class name of application context that will be used by default for non-web
	 * environments.
	 */
	public static final String DEFAULT_CONTEXT_CLASS = "org.springframework.context."
			+ "annotation.AnnotationConfigApplicationContext";

如上述代码所示，创建容器应用程序上下文时应根据环境类型的不同而创建不同的应用程序上下文。

我们使用的是反应式Web环境，所以创建的应用程序上下文是 AnnotationConfigReactiveWebServerApplicationContext 的实例。

AnnotationConfigReactiveWebServerApplicationContext 的继承关系如下，后面会用到：

那么环境类型 webApplicationType 是如何确定的呢？其实是在创建 SpringApplication 的构造函数内确定的：

	/**
	 * Create a new {@link SpringApplication} instance. The application context will load
	 * beans from the specified primary sources (see {@link SpringApplication class-level}
	 * documentation for details. The instance can be customized before calling
	 * {@link #run(String...)}.
	 * @param resourceLoader the resource loader to use
	 * @param primarySources the primary bean sources
	 * @see #run(Class, String[])
	 * @see #setSources(Set)
	 */
	@SuppressWarnings({ "unchecked", "rawtypes" })
	public SpringApplication(ResourceLoader resourceLoader, Class<?>... primarySources) {
		this.resourceLoader = resourceLoader;
		Assert.notNull(primarySources, "PrimarySources must not be null");
		this.primarySources = new LinkedHashSet<>(Arrays.asList(primarySources));
        // 确定环境类型
		this.webApplicationType = WebApplicationType.deduceFromClasspath();
		setInitializers((Collection) getSpringFactoriesInstances(ApplicationContextInitializer.class));
		setListeners((Collection) getSpringFactoriesInstances(ApplicationListener.class));
		this.mainApplicationClass = deduceMainApplicationClass();
	}

紧接着再来看下 WebApplicationType.deduceFromClasspath() 这个方法：

扫描二维码关注公众号，回复： 12417638 查看本文章

    // WebApplicationType.java
	static WebApplicationType deduceFromClasspath() {
        // 判断是不是 REACTIVE 类型
		if (ClassUtils.isPresent(WEBFLUX_INDICATOR_CLASS, null) && !ClassUtils.isPresent(WEBMVC_INDICATOR_CLASS, null)
				&& !ClassUtils.isPresent(JERSEY_INDICATOR_CLASS, null)) {
			return WebApplicationType.REACTIVE;
		}
        // 判断是不是非 Web 类型
		for (String className : SERVLET_INDICATOR_CLASSES) {
			if (!ClassUtils.isPresent(className, null)) {
				return WebApplicationType.NONE;
			}
		}
        // SERVLET 环境
		return WebApplicationType.SERVLET;
	}
    
    // Servlet 容器所需要的类
	private static final String[] SERVLET_INDICATOR_CLASSES = { "javax.servlet.Servlet",
			"org.springframework.web.context.ConfigurableWebApplicationContext" };

    // spring mvc 分派器
	private static final String WEBMVC_INDICATOR_CLASS = "org.springframework.web.servlet.DispatcherServlet";

    // reactive web分派器
	private static final String WEBFLUX_INDICATOR_CLASS = "org.springframework.web.reactive.DispatcherHandler";

    // Jersey Web 项目容器类
	private static final String JERSEY_INDICATOR_CLASS = "org.glassfish.jersey.servlet.ServletContainer";

如上述代码所示，deduceFromClasspath 根据 classpath 下是否有对应的 class 字节码存在，来决定当前的环境的。

3.refreshContext

第1步中，run 方法内 25 行的 refreshContext 方法

	// SpringApplication.java
    private void refreshContext(ConfigurableApplicationContext context) {
		refresh(context);
		if (this.registerShutdownHook) {
			try {
				context.registerShutdownHook();
			}
			catch (AccessControlException ex) {
				// Not allowed in some environments.
			}
		}
	}

    // SpringApplication.java
	/**
	 * Refresh the underlying {@link ApplicationContext}.
	 * @param applicationContext the application context to refresh
	 */
	protected void refresh(ApplicationContext applicationContext) {
		Assert.isInstanceOf(AbstractApplicationContext.class, applicationContext);
		((AbstractApplicationContext) applicationContext).refresh();
	}

4.refresh

从前文，我们知道此时的 applicationContext 是 AnnotationConfigReactiveWebServerApplicationContext 的实例，是 AbstractApplicationContext 的子类，所以跳转到 AbstractApplicationContext 的 refresh 方法里：

    // AbstractApplicationContext.java
    @Override
	public void refresh() throws BeansException, IllegalStateException {
		synchronized (this.startupShutdownMonitor) {
			// Prepare this context for refreshing.
			prepareRefresh();

			// Tell the subclass to refresh the internal bean factory.
			ConfigurableListableBeanFactory beanFactory = obtainFreshBeanFactory();

			// Prepare the bean factory for use in this context.
			prepareBeanFactory(beanFactory);

			try {
				// Allows post-processing of the bean factory in context subclasses.
				postProcessBeanFactory(beanFactory);

				// Invoke factory processors registered as beans in the context.
				invokeBeanFactoryPostProcessors(beanFactory);

				// Register bean processors that intercept bean creation.
				registerBeanPostProcessors(beanFactory);

				// Initialize message source for this context.
				initMessageSource();

				// Initialize event multicaster for this context.
				initApplicationEventMulticaster();

				// Initialize other special beans in specific context subclasses.
				onRefresh();

				// Check for listener beans and register them.
				registerListeners();

				// Instantiate all remaining (non-lazy-init) singletons.
				finishBeanFactoryInitialization(beanFactory);

				// Last step: publish corresponding event.
				finishRefresh();
			}

			catch (BeansException ex) {
				if (logger.isWarnEnabled()) {
					logger.warn("Exception encountered during context initialization - " +
							"cancelling refresh attempt: " + ex);
				}

				// Destroy already created singletons to avoid dangling resources.
				destroyBeans();

				// Reset 'active' flag.
				cancelRefresh(ex);

				// Propagate exception to caller.
				throw ex;
			}

			finally {
				// Reset common introspection caches in Spring's core, since we
				// might not ever need metadata for singleton beans anymore...
				resetCommonCaches();
			}
		}
	}

抽象类 + protected 方法，模板方法模式，这个方法就是模板方法，具体的方法逻辑会写到对应的子类中。

5.onRefresh

创建并启动HTTP服务器是在 onRefresh 方法中完成的：

    // ReactiveWebServerApplicationContext.java	    
    @Override
	protected void onRefresh() {
		super.onRefresh();
		try {
			createWebServer();
		}
		catch (Throwable ex) {
			throw new ApplicationContextException("Unable to start reactive web server", ex);
		}
	}

6.createWebServer

    // ReactiveWebServerApplicationContext.java	 
	private void createWebServer() {
		ServerManager serverManager = this.serverManager;
		if (serverManager == null) {
			String webServerFactoryBeanName = getWebServerFactoryBeanName();
			ReactiveWebServerFactory webServerFactory = getWebServerFactory(webServerFactoryBeanName);
			boolean lazyInit = getBeanFactory().getBeanDefinition(webServerFactoryBeanName).isLazyInit();
			this.serverManager = ServerManager.get(webServerFactory, lazyInit);
		}
		initPropertySources();
	}

7.getWebServerFactory

    // ReactiveWebServerApplicationContext.java
	protected String getWebServerFactoryBeanName() {
        // 从bean工厂中获取所有ReactiveWebServerFactory类型的Bean实例的名字
		// Use bean names so that we don't consider the hierarchy
		String[] beanNames = getBeanFactory().getBeanNamesForType(ReactiveWebServerFactory.class);
		if (beanNames.length == 0) {
			throw new ApplicationContextException(
					"Unable to start ReactiveWebApplicationContext due to missing ReactiveWebServerFactory bean.");
		}
		if (beanNames.length > 1) {
			throw new ApplicationContextException("Unable to start ReactiveWebApplicationContext due to multiple "
					+ "ReactiveWebServerFactory beans : " + StringUtils.arrayToCommaDelimitedString(beanNames));
		}
        // 只有正好是一个时获取实例
		return beanNames[0];
	}

	protected ReactiveWebServerFactory getWebServerFactory(String factoryBeanName) {
		return getBeanFactory().getBean(factoryBeanName, ReactiveWebServerFactory.class);
	}

如上述代码所示，从应用程序上下文对应的 Bean 工厂中获取对应的 ReactiveWebServerFactory 实例，以便后续创建 Web 服务器。

那么问题来了，ReactiveWebServerFactory 的实现类，什么时候注入上下文容器的呢？

这里借助了 Spring Boot 的 autoconfigure 机制来实现的，代码如下：

/**
 * Configuration classes for reactive web servers
 * <p>
 * Those should be {@code @Import} in a regular auto-configuration class to guarantee
 * their order of execution.
 *
 * @author Brian Clozel
 * @author Raheela Aslam
 * @author Sergey Serdyuk
 */
abstract class ReactiveWebServerFactoryConfiguration {

    // 1.将NettyReactiveWebServerFactory注入容器
	@Configuration(proxyBeanMethods = false)
	@ConditionalOnMissingBean(ReactiveWebServerFactory.class)
	@ConditionalOnClass({ HttpServer.class })
	static class EmbeddedNetty {
		@Bean
		@ConditionalOnMissingBean
		ReactorResourceFactory reactorServerResourceFactory() {
			return new ReactorResourceFactory();
		}
		@Bean
		NettyReactiveWebServerFactory nettyReactiveWebServerFactory(ReactorResourceFactory resourceFactory,
				ObjectProvider<NettyRouteProvider> routes, ObjectProvider<NettyServerCustomizer> serverCustomizers) {
			NettyReactiveWebServerFactory serverFactory = new NettyReactiveWebServerFactory();
			serverFactory.setResourceFactory(resourceFactory);
			routes.orderedStream().forEach(serverFactory::addRouteProviders);
			serverFactory.getServerCustomizers().addAll(serverCustomizers.orderedStream().collect(Collectors.toList()));
			return serverFactory;
		}
	}

    // 2.注入TomcatReactiveWebServerFactory实例
	@Configuration(proxyBeanMethods = false)
	@ConditionalOnMissingBean(ReactiveWebServerFactory.class)
	@ConditionalOnClass({ org.apache.catalina.startup.Tomcat.class })
	static class EmbeddedTomcat {
		@Bean
		TomcatReactiveWebServerFactory tomcatReactiveWebServerFactory(
				ObjectProvider<TomcatConnectorCustomizer> connectorCustomizers,
				ObjectProvider<TomcatContextCustomizer> contextCustomizers,
				ObjectProvider<TomcatProtocolHandlerCustomizer<?>> protocolHandlerCustomizers) {
			TomcatReactiveWebServerFactory factory = new TomcatReactiveWebServerFactory();
			factory.getTomcatConnectorCustomizers()
					.addAll(connectorCustomizers.orderedStream().collect(Collectors.toList()));
			factory.getTomcatContextCustomizers()
					.addAll(contextCustomizers.orderedStream().collect(Collectors.toList()));
			factory.getTomcatProtocolHandlerCustomizers()
					.addAll(protocolHandlerCustomizers.orderedStream().collect(Collectors.toList()));
			return factory;
		}
	}

    // 3.注入JettyReactiveWebServerFactory实例
	@Configuration(proxyBeanMethods = false)
	@ConditionalOnMissingBean(ReactiveWebServerFactory.class)
	@ConditionalOnClass({ org.eclipse.jetty.server.Server.class })
	static class EmbeddedJetty {
		@Bean
		@ConditionalOnMissingBean
		JettyResourceFactory jettyServerResourceFactory() {
			return new JettyResourceFactory();
		}
		@Bean
		JettyReactiveWebServerFactory jettyReactiveWebServerFactory(JettyResourceFactory resourceFactory,
				ObjectProvider<JettyServerCustomizer> serverCustomizers) {
			JettyReactiveWebServerFactory serverFactory = new JettyReactiveWebServerFactory();
			serverFactory.getServerCustomizers().addAll(serverCustomizers.orderedStream().collect(Collectors.toList()));
			serverFactory.setResourceFactory(resourceFactory);
			return serverFactory;
		}
	}

    // 4.注入UndertowReactiveWebServerFactory实例
	@Configuration(proxyBeanMethods = false)
	@ConditionalOnMissingBean(ReactiveWebServerFactory.class)
	@ConditionalOnClass({ Undertow.class })
	static class EmbeddedUndertow {
		@Bean
		UndertowReactiveWebServerFactory undertowReactiveWebServerFactory(
				ObjectProvider<UndertowBuilderCustomizer> builderCustomizers) {
			UndertowReactiveWebServerFactory factory = new UndertowReactiveWebServerFactory();
			factory.getBuilderCustomizers().addAll(builderCustomizers.orderedStream().collect(Collectors.toList()));
			return factory;
		}
	}
}

1.如果当前容器上下文中不存在 ReactiveWebServerFactory 的实例，并且 classpath 下存在 HttpServer 的class 文件，则说明当前环境为 Reactive 环境，则注入 NettyReactiveWebServerFactory 到容器。
2.如果当前容器上下文中不存在 ReactiveWebServerFactory 的实例，并且 classpath 下存在 org.apache.catalina.startup.Tomcat 的 class 文件，则说明当前环境为 Servlet 环境，并且 Servlet 容器为 tomcat，则将 TomcatReactiveWebServerFactory 实例注入容器。
3-4 情况类似，以此类推。

8.get

找到对应的 ReactiveWebServerFactory 工厂实例后，步骤8创建了 ServerManager 的实例，代码如下：

    // ReactiveWebServerApplicationContext.java
		static ServerManager get(ReactiveWebServerFactory factory, boolean lazyInit) {
			return new ServerManager(factory, lazyInit);
		}

ServerManager 构造方法如下

    // ReactiveWebServerApplicationContext.java
		private ServerManager(ReactiveWebServerFactory factory, boolean lazyInit) {
			this.handler = this::handleUninitialized;
			this.server = factory.getWebServer(this);
			this.lazyInit = lazyInit;
		}

9.getWebServer

由上可知，调用 NettyReactiveWebServerFactory 的 getWebServer 方法创建了Web服务器，其代码如下：

    // NettyReactiveWebServerFactory.java
	@Override
	public WebServer getWebServer(HttpHandler httpHandler) {
        // 创建了 HTTPServer
		HttpServer httpServer = createHttpServer();
        // 创建了与 Netty 对应的适配器类 ReactorHttpHandlerAdapter
		ReactorHttpHandlerAdapter handlerAdapter = new ReactorHttpHandlerAdapter(httpHandler);
        // 创建了一个 NettyWebServer 的实例，其包装了适配器和 HTTPserver 实例
		NettyWebServer webServer = new NettyWebServer(httpServer, handlerAdapter, this.lifecycleTimeout);
		webServer.setRouteProviders(this.routeProviders);
		return webServer;
	}

10.createHttpServer

如上代码所示，其通过 createHttpServer 方法创建了 HTTPServer，其代码如下：

    // NettyReactiveWebServerFactory.java
    // 使用 reactor Netty 的 API 创建了 HTTPServer
	private HttpServer createHttpServer() {
		HttpServer server = HttpServer.create();
		if (this.resourceFactory != null) {
			LoopResources resources = this.resourceFactory.getLoopResources();
			Assert.notNull(resources, "No LoopResources: is ReactorResourceFactory not initialized yet?");
			server = server.tcpConfiguration(
					(tcpServer) -> tcpServer.runOn(resources).addressSupplier(this::getListenAddress));
		}
		else {
			server = server.tcpConfiguration((tcpServer) -> tcpServer.addressSupplier(this::getListenAddress));
		}
		if (getSsl() != null && getSsl().isEnabled()) {
			SslServerCustomizer sslServerCustomizer = new SslServerCustomizer(getSsl(), getHttp2(),
					getSslStoreProvider());
			server = sslServerCustomizer.apply(server);
		}
		if (getCompression() != null && getCompression().getEnabled()) {
			CompressionCustomizer compressionCustomizer = new CompressionCustomizer(getCompression());
			server = compressionCustomizer.apply(server);
		}
		server = server.protocol(listProtocols()).forwarded(this.useForwardHeaders);
		return applyCustomizers(server);
	}

到这里如何创建 HTTPServer 的逻辑就结束了。

下面看下启动服务的逻辑。

从第4步 refresh 的模板方法中，在 onRefresh 后面 的 finishRefresh 方法开始。

11.finishRefresh

    // ReactiveWebServerApplicationContext.java
	@Override
	protected void finishRefresh() {
		super.finishRefresh();
		WebServer webServer = startReactiveWebServer();
		if (webServer != null) {
			publishEvent(new ReactiveWebServerInitializedEvent(webServer, this));
		}
	}

12.startReactiveWebServer

    // ReactiveWebServerApplicationContext.java
	private WebServer startReactiveWebServer() {
		ServerManager serverManager = this.serverManager;
		ServerManager.start(serverManager, this::getHttpHandler);
		return ServerManager.getWebServer(serverManager);
	}

如上代码所示，首先调用了getHttpHandler来获取处理器

    // ReactiveWebServerApplicationContext.java
	/**
	 * Return the {@link HttpHandler} that should be used to process the reactive web
	 * server. By default this method searches for a suitable bean in the context itself.
	 * @return a {@link HttpHandler} (never {@code null}
	 */
	protected HttpHandler getHttpHandler() {
		// Use bean names so that we don't consider the hierarchy
		String[] beanNames = getBeanFactory().getBeanNamesForType(HttpHandler.class);
		if (beanNames.length == 0) {
			throw new ApplicationContextException(
					"Unable to start ReactiveWebApplicationContext due to missing HttpHandler bean.");
		}
		if (beanNames.length > 1) {
			throw new ApplicationContextException(
					"Unable to start ReactiveWebApplicationContext due to multiple HttpHandler beans : "
							+ StringUtils.arrayToCommaDelimitedString(beanNames));
		}
		return getBeanFactory().getBean(beanNames[0], HttpHandler.class);
	}

如上代码所示，其中获取了应用程序上下文中 HttpHandler 的实现类，这里为 HttpWebHandlerAdapter。

13.start

然后调用ServerManager.start启动了服务，其代码如下：

    // ReactiveWebServerApplicationContext.java
		static void start(ServerManager manager, Supplier<HttpHandler> handlerSupplier) {
			if (manager != null && manager.server != null) {
				manager.handler = manager.lazyInit ? new LazyHttpHandler(Mono.fromSupplier(handlerSupplier))
						: handlerSupplier.get();
                // 启动服务
				manager.server.start();
			}
		}

如上代码所示，首先把 HttpWebHandlerAdapter 实例保存到了 ServerManager 内部，然后启动 ServerManager 中的 NettyWebServer 服务器。

14.start

NettyWebServer 的 start 方法代码如下：

    // NettyWebServer.java
	@Override
	public void start() throws WebServerException {
		if (this.disposableServer == null) {
			try {
                // 具体启动服务
				this.disposableServer = startHttpServer();
			}
			catch (Exception ex) {
				ChannelBindException bindException = findBindException(ex);
				if (bindException != null) {
					throw new PortInUseException(bindException.localPort());
				}
				throw new WebServerException("Unable to start Netty", ex);
			}
            // 开启deamon线程同步等待服务终止
			logger.info("Netty started on port(s): " + getPort());
			startDaemonAwaitThread(this.disposableServer);
		}
	}

15.startHttpServer

    // NettyWebServer.java
	private DisposableServer startHttpServer() {
		HttpServer server = this.httpServer;
		if (this.routeProviders.isEmpty()) {
			server = server.handle(this.handlerAdapter);
		}
		else {
			server = server.route(this::applyRouteProviders);
		}
		if (this.lifecycleTimeout != null) {
			return server.bindNow(this.lifecycleTimeout);
		}
		return server.bindNow();
	}

	private void startDaemonAwaitThread(DisposableServer disposableServer) {
		Thread awaitThread = new Thread("server") {

			@Override
			public void run() {
				disposableServer.onDispose().block();
			}

		};
        // 设置线程为 demaon，并启动
		awaitThread.setContextClassLoader(getClass().getClassLoader());
		awaitThread.setDaemon(false);
		awaitThread.start();
	}

这里之所以开启线程来异步等待服务终止，是因为这样不会阻塞调用线程，如果调用线程被阻塞了，则整个 Spring Boot 应用就运行不起来了。

至此，WebFlux 服务就启动了。