As a reverse proxy, the api gateway provides a unified service access entry for external systems, performs access control processing such as authentication flow for requests, and forwards the request route to the back-end service after passing. In high concurrency and potentially high latency scenarios, a basic requirement for API gateways to achieve high performance and high throughput is full-link asynchrony without blocking threads. Previously, it was based on servlet, using asynchronous servlet filter + asynchronous servlet + servlet asynchronous io + asynchronous httpclient as a full-link asynchronous http/rest api gateway. It should be emphasized that after using asynchronous servlet, it is only asynchronous in process, which does not mean io. Automatic is asynchronous, and servlet asynchronous io can be used to achieve high throughput of network io. Asynchronous httpclient used the netty-based async-http-client library at the time.
Spring finally provides reactive programming support in version 5, which perfectly supports asynchronous non-blocking programming. The previous spring systems, including spring mvc, are mostly synchronous blocking programming modes. Even if the @Async annotation is added to the spring mvc controller method or the result of type DeferredResult and Callable is returned, it is still just encapsulating the synchronous call of the method into the task queue of the thread pool. The pseudo-asynchrony realized by the thread pool is why? It is called pseudo-asynchronous, because when there is a high concurrent request, the number of threads in the thread pool needs to be increased (maybe dozens or hundreds of threads), because it is still a synchronous blocking task, plus thread scheduling overhead, this kind of Pseudo-asynchrony is actually impossible to achieve ideal high concurrency. The root cause of this problem is that most of Java's previous synchronous blocking libraries or specifications are provided, and there is no officially supported coroutine similar to golang. For example, there is no asynchronous JDBC api. so. However, these are improving. For example, the Loom project in openJDK plans to implement the coroutine function, and Alibaba's Wisp coroutine technology has been used in Alibaba's production environment. These two (Loom and Wisp) may be merged into the future. an item. Asynchronous JDBC should be ready for release within two years. True asynchronous non-blocking task processing does not require too many threads to achieve ideal high concurrency. Too many threads will have side effects. Usually, the number of threads is not more than twice the number of CPU cores, which is generally the number of CPU cores.
Under the impact of node.js non-blocking asynchronous programming and vertx, spring 5's support for reactive programming can be said to keep pace with vertx, which was at the forefront, so spring fans can breathe a sigh of relief. Spring5 provides webflux module based on reactor3 to support web reactive programming. Reactive advocates asynchronous and non-blocking. Back pressure will not be discussed here. Asynchronous servlet is implemented by patching the original servlet synchronization model, which is different from the newly designed webflux of spring5.
The main component of the webflux-based gateway is the webfilter filter (webfilter and servlet filter in webflux are two different things. Although the concepts are similar, the underlying default runtime is based on netty, not tomcat, although it can also be adapted to traditional tomcat or jetty. Container. Webflux does not use the programming model of servlet. HttpServletRequest and HttpServletResponse in servlet cannot be used in webflux), and multiple filters respectively implement authentication and authorization, current limiting, request routing and forwarding, and normal response write-back (the backend The response of the service is written back to the front-end system calling the gateway), etc. If protocol conversion is required when calling back-end services, for example, the front-end request is http(s) and the back-end service is rpc, then protocol conversion is required. For this, please refer to the controller method parameter parser of spring mvc and spring webflux and The return value processor, in special cases, can provide a configuration interface to set the attribute mapping relationship between the two protocol data. The protocol conversion component is preferably in the form of a plug-in.
Reference Architecture
The front-end here is not the usual UI front-end, but refers to roles. These systems need to access the gateway to call the back-end services. They are service consumers and are in front of the gateway.
code repository
The code is on the github demo-spring-webflux-api-gateway repository, with two branches: master and dev.
use technology
Reactive & asynchronous non-blocking io: springboot2+spring reactor 3+spring5 webflux includes webclient, webfilter, and controller.
This is to pull out the basic shelf from the production project as a demo, it's no problem to run, build with maven.
service routing
后端服务的url前缀(域名和http schema)是固定的，在application.properties文件中由backend.service.url.prefix属性来配置，例如backend.service.url.prefix=http://127.0.0.1:8080 。
假设前端应用(服务消费者)用如下url调用网关(网关端口为9988)
http://api.gateway.demo:9988/orders/1234
如果请求过滤通过，网关最后会将请求路由转发到http://127.0.0.1:8080/orders/1234，这个url提供订单服务。
在实际的生产环境中，一般是后端有多个服务，部署在多个不同的服务域名上，特别是采用微服务架构后，网关需要从服务注册中心获取后端服务的endpoint信息(如果后端服务是http，主要是url前缀部分，例如http schema(http、https)+服务域名或ip+端口)，而不是这种演示情况下使用固定前缀的方式。如果以网关为中心，注册中心可以是网关的一个模块或子系统，也可是独立的系统。
在有多个后端服务域名的情况下，为了方便网关进行路由，可以在前端请求中携带路由提示信息。这个路由提示信息的粒度可大可小：可以是细粒度的服务名，它对应一个url模式+http方法(考虑restful风格)，根据服务名可以映射到服务所在的endpoint；也可是粗粒度的服务名，这个粗粒度的服务名其实是一个应用名(app name)或逻辑域，这个应用上可有多个细粒度的服务，应用名再映射到真实的endpoint上。服务名可放在http请求头中或在url中靠前的位置，例如http://api.gateway.demo:9988/order-center/orders/1234 这个url中的order-center就是粗粒度的服务名(应用名)，网关可根据这个服务名从注册中心找到对应的后端服务域名和http schema等url endpoint信息。
如果众多请求url模式+http方法不冲突，能确保网关根据请求信息(主要是请求url和http方法)找到对应的路由endpoint地址，也可以不加服务名。
相关类介绍
1）main class为com.demo.http.api.gateway.main.RestApiGateWayApplication。
在IDE中直接运行这个类即可跑起来，或mvn package打包，再java -jar运行jar包。
2）过滤器类位于com.demo.http.api.gateway.access.filter包中，已经有两个过滤器：RequestAuthFilter和RateLimitFilter，分别用于鉴权和限流。过滤器类上有Order注解，决定该过滤器在过滤器链中的调用先后顺序，order数值小的先调用。作为演示，鉴权只是看请求中有没有appKey的http头，限流是放过所有请求不进行限流。
3）控制器类：转发请求到后端服务是由com.demo.http.api.gateway.access.controller.ApiProxyController控制器来实现的，也可用过滤器来实现，只是生产项目当时写的顺手用控制器来做了，没改过来。对后端http服务的调用使用webflux模块提供的WebClient,它支持异步和同步两种调用方式。使用反应式编程，对http client不推荐使用spring先前的RestTemplate和AsyncRestTemplate。其他几个控制器用来生成异常情况下的响应数据发给前端，这些也可不需要，而是直接在过滤器中(过滤器验证未通过或后端服务网络异常)生成异常数据并发给前端，在代码中有直接的，也有转发到控制器的。
前端请求中除了要有appKey http头，大多还需要携带类似token头和请求数据签名，如何获取token和对token进行验证这里不介绍。
4）服务类：com.demo.http.api.gateway.service.AppInfoProvider 根据appKey来获取调用方的应用信息(应用密钥)，演示情况下没有调用AppInfoProvider的方法，只是把AppInfoProvider实例注入到RequestAuthFilter类中。
5）过滤器父类
dev分支中的过滤器均继承父类com.demo.http.api.gateway.access.filter.base.AbstractGatewayWebFilter，这个类采用模板方法设计模式。
子类只要实现两个模板方法doFilter和doDenyResponse。doFilter实现过滤器的业务逻辑，有两种过滤结果：布尔值类型TRUE和FALSE(代码中其实是布尔类型的反应式版本Mono<Boolean>)。TRUE表示过滤通过(pass)，请求控制流会转到过滤器链中的下个filter或controller； 如果是FALSE，未通过(deny)，会调用doDenyResponse方法，在这个方法中一般是生成deny响应数据发给前端系统，结束该请求的处理。这两个方法在dev分支的过滤器中有实现，特别是RequestAuthFilter。
该类中还有个skipProcess方法，filter可调用这个方法指示后面的filter跳过对该请求的处理，也就是不执行doFilter。
6）服务层和dao层
com.demo.http.api.gateway.service.AppInfoProvider提供方法用来根据appKey(应用名)获取该应用的密钥等app信息，app信息使用了spring caffeine缓存。com.demo.http.api.gateway.dao.mapper.AppInfoMapper是app信息的dao类，使用myibatis。
7）工具类
com.demo.http.api.gateway.util.WebfluxForwardingUtil，提供forward方法来让filter在需要的时候forward请求到网关内部的其他目标url。controller中如何forward在先前博文中有介绍。
配置文件
application.properties内容如下：
#spring.profiles.active=dev
#网关服务端口
server.port=9988
#网关url前缀，目前没有使用
gateway.url.prefix=/api
#调用后端服务的超时时间，单位毫秒。生产项目应该是根据具体的服务取对应的超时配置
backend.service.timeout.inmillis=10000
#后端服务endpoint，生产项目通常应该从注册中心获取服务和它对应的endpoint关系
backend.service.url.prefix=http://127.0.0.1:8080
#数据库配置，作为演示项目，没有调用数据库，不需要配置下面的属性
datasource.mysql.jdbcUrl=jdbc:mysql://127.0.0.1:3308/gateway?useUnicode=true&characterEncoding=utf-8
datasource.mysql.user=
datasource.mysql.password=
datasource.mysql.driverClass=com.mysql.jdbc.Driver
datasource.mysql.driverClass.type=com.mchange.v2.c3p0.ComboPooledDataSource
datasource.mysql.testWhileIdle = true
datasource.mysql.preferredTestQuery=SELECT 1
datasource.mysql.testConnectionOnCheckin = true
datasource.mysql.idleConnectionTestPeriod = 400
datasource.mysql.maxIdleTime = 500
后记
编写高性能api网关也可采用协程，当有阻塞时，阻塞在协程上而不是阻塞在线程上。和线程相比，协程是很轻量级的资源，在一个线程中可有几千个协程，协程的调度在用户级，调度开销也比线程小得多，所以采用协程也能实现网络io或io密集型系统的高吞吐量。反应式的组合链式调用风格可极大地减轻回调地狱问题(callback hell)，而协程提供同步风格的代码编写体验，同步风格符合开发人员的心智模型和习惯，提高了编码效率，同时同步风格完全消除了回调地狱，显著改善代码的可维护性。
webflux包括其中的webclient目前还不支持http2,今年下半年应该就可以了，http2在一个http连接上能真正并发多个请求，这样在高并发情况下连接池中只要不多的连接资源，也避免了高并发时等待空闲连接的窘境或新建连接的时间。