Preface
This article will use the SpringCloud Gateway gateway component with Nacos to implement grayscale publishing (canary publishing)
Environment setup
Create submodule service provider provider, gateway module gateway
Parent project
pom.xmlConfiguration
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>spring-gateway-demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>pom</packaging>
<name>spring-gateway-demo</name>
<description>spring-gateway-demo</description>
<properties>
<java.version>11</java.version>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<maven.compiler.plugin>3.8.1</maven.compiler.plugin>
<spring-boot.version>2.3.7.RELEASE</spring-boot.version>
<spring-cloud-alibaba.version>2.2.2.RELEASE</spring-cloud-alibaba.version>
<spring-cloud.version>Hoxton.SR9</spring-cloud.version>
<spring-cloud-starter-alibaba-nacos-config>2.2.0.RELEASE</spring-cloud-starter-alibaba-nacos-config>
</properties>
<modules>
<module>provider</module>
<module>gateway</module>
</modules>
<dependencies>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId>
</dependency>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${spring-boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-alibaba-dependencies</artifactId>
<version>${spring-cloud-alibaba.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
</project>
service providerprovider
Here we plan to introduce it , so first create nacosa nacos configuration file , using the default namespace and default grouping here.dataIdprovider.propertiespublicDEFAULT_GROUP
version=2
provider's pom configuration dependencies
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
application.yml
server:
port: 9001
spring:
application:
name: provider
cloud:
nacos:
config:
server-addr: 127.0.0.1:8848
discovery:
server-addr: 127.0.0.1:8848
Add the @EnableDiscoveryClient annotation to the startup class
@EnableDiscoveryClient
@SpringBootApplication
public class ProviderApplication {
public static void main(String[] args) {
SpringApplication.run(ProviderApplication.class, args);
}
}
Then add the test controller
@RefreshScope
@RestController
@RequestMapping("/test")
public class TestController {
@Autowired
private Environment env;
@Value("${version:0}")
private String version;
/**
* http://localhost:9001/test/port
* @return
*/
@GetMapping("/port")
public Object port() {
return String.format("port=%s, version=%s", env.getProperty("local.server.port"), version);
}
}
Note that when configuring nacos here, you need to configure the following two files provider.propertiesand provider, and then actually configure the provider file used by nacos. Otherwise, the backend console will continue to output 400 errors. It may be a problem with the new version. Other versions are not yet clear ( The same applies to the gateway configuration later)
From the backend console output, it can also be seen that two
[fixed-localhost_8848] [subscribe] provider.properties+DEFAULT_GROUP
[fixed-localhost_8848] [add-listener] ok, tenant=, dataId=provider.properties, group=DEFAULT_GROUP, cnt=1
[fixed-localhost_8848] [subscribe] provider+DEFAULT_GROUP
[fixed-localhost_8848] [add-listener] ok, tenant=, dataId=provider, group=DEFAULT_GROUP, cnt=1
Nacos version: 2.3.0-BETA
gatewaygateway
The pom dependency configuration of the gateway service is as follows:
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
</dependencies>
application.yml
# 应用服务 WEB 访问端口
server:
port: 9000
# 应用名称
spring:
application:
name: gateway
cloud:
nacos:
config:
server-addr: 127.0.0.1:8848
discovery:
server-addr: 127.0.0.1:8848
gateway:
routes: # http://127.0.0.1:9000/actuator/gateway/routes
- id: provider # 路由 ID，保持唯一
uri: lb://provider # uri指目标服务地址，lb代表从注册中心获取服务
predicates:
- Path=/provider/** # http://127.0.0.1:9000/provider/port 会转发到 http://localhost:9001/provider/port, 和预期不符合, 需要StripPrefix来处理
filters:
- StripPrefix=1 # StripPrefix=1就代表截取路径的个数为1, 这样请求 http://127.0.0.1:9000/provider/test/port 会转发到 http://localhost:9001/test/port
management:
endpoint:
gateway:
enabled: true
endpoints:
web:
exposure:
include: gateway
Also add the @EnableDiscoveryClient annotation to the startup class
View all routes: /actuator/gateway/routes
View specified routes (GET): /actuator/gateway/routes/{id}
View global filters: /actuator/gateway/globalfilters
View route filters: /actuator/gateway/routefilters
POST Method to refresh the routing cache:/actuator/gateway/refresh
test
curl http://127.0.0.1:9001/test/port
port=9001, version=2
curl http://127.0.0.1:9000/provider/test/port
port=9001, version=2
dynamic routing
There are two ways to implement dynamic routing, one is to rewrite RouteDefinitionRepository(the actual test fails), and the other is nacos to dynamically update the value based on the listener RouteDefinitionRepository . The implementation logic is similar
The following classes are responsible for loading routing information in Spring Cloud Gateway:
1. PropertiesRouteDefinitionLocator: Reading routing information from configuration files (such as YML, Properties, etc.)
2. RouteDefinitionRepository: Reading routing information from storage (such as memory, configuration center , Redis, MySQL, etc.)
3. DiscoveryClientRouteDefinitionLocator: Read routing information from the registration center (such as Nacos, Eurka, Zookeeper, etc.)
Next use RouteDefinitionRepository to configure dynamic routing
gateway-router.json
[{
"id": "provider",
"predicates": [{
"name": "Path",
"args": {
"_genkey_0": "/provider/**"
}
}],
"filters": [{
"name": "StripPrefix",
"args": {
"_genkey_0": "1"
}
}],
"uri": "lb://provider",
"order": 0
}]
NacosRouteDefinitionRepositoryConfiguration class
@Component
public class NacosRouteDefinitionRepository implements RouteDefinitionRepository, ApplicationEventPublisherAware {
private static final Logger log = LoggerFactory.getLogger(NacosRouteDefinitionRepository.class);
@Autowired
private NacosConfigManager nacosConfigManager;
// 更新路由信息需要的
private ApplicationEventPublisher applicationEventPublisher;
private String dataId = "gateway-router.json";
private String group = "DEFAULT_GROUP";
@Value("${spring.cloud.nacos.config.server-addr}")
private String serverAddr;
private ObjectMapper objectMapper = new ObjectMapper();
@PostConstruct
public void dynamicRouteByNacosListener() {
try {
nacosConfigManager.getConfigService().addListener(dataId, group, new Listener() {
public void receiveConfigInfo(String configInfo) {
log.info("自动更新配置...\r\n{}", configInfo);
applicationEventPublisher.publishEvent(new RefreshRoutesEvent(this));
}
public Executor getExecutor() {
return null;
}
});
} catch (NacosException e) {
e.printStackTrace();
}
}
@Override
public Flux<RouteDefinition> getRouteDefinitions() {
try {
String configInfo = nacosConfigManager.getConfigService().getConfig(dataId, group, 5000);
List<RouteDefinition> gatewayRouteDefinitions = objectMapper.readValue(configInfo, new TypeReference<List<RouteDefinition>>() {
});
return Flux.fromIterable(gatewayRouteDefinitions);
} catch (NacosException e) {
e.printStackTrace();
} catch (JsonMappingException e) {
e.printStackTrace();
} catch (JsonProcessingException e) {
e.printStackTrace();
}
return Flux.fromIterable(Lists.newArrayList());
}
@Override
public Mono<Void> save(Mono<RouteDefinition> route) {
return null;
}
@Override
public Mono<Void> delete(Mono<String> routeId) {
return null;
}
@Override
public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) {
this.applicationEventPublisher = applicationEventPublisher;
}
}
Then restart the gateway and visit http://127.0.0.1:9000/actuator/gateway/routes to see if it takes effect.
[
{
"predicate": "Paths: [/provider/**], match trailing slash: true",
"route_id": "provider",
"filters": [
"[[StripPrefix parts = 1], order = 1]"
],
"uri": "lb://provider",
"order": 0
}
]
Grayscale release
First, you need to understand the grayscale scenario. Because there are different versions of services that need to coexist, there will inevitably be differences in code and configuration when new nodes are upgraded. Therefore, we use this difference to determine whether the service version is a new version or an online stable version. . Here we use prod and gray to identify the 2 versions.
The overall idea of implementation:
- Write grayscale routing with version number (load balancing strategy)
- Write a custom filter
- The nacos service configuration requires the metadata information and weight of the grayscale published service (configured in the service jar)
Note that you should first modify the nacos configuration to implement dynamic routing, and then upgrade the grayscale nodes. This case is just a simple example of the grayscale principle.
Next, configure the gateway
Depend on configuration
First exclude the default ribbon dependency
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</exclusion>
</exclusions>
</dependency>
Introducing the official new load balancing package
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>
Load balancing strategy
public class VersionGrayLoadBalancer implements ReactorServiceInstanceLoadBalancer {
private ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider;
private String serviceId;
private final AtomicInteger position;
public VersionGrayLoadBalancer(ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider, String serviceId) {
this(serviceInstanceListSupplierProvider, serviceId, new Random().nextInt(1000));
}
public VersionGrayLoadBalancer(ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider, String serviceId, int seedPosition) {
this.serviceId = serviceId;
this.serviceInstanceListSupplierProvider = serviceInstanceListSupplierProvider;
this.position = new AtomicInteger(seedPosition);
}
@Override
public Mono<Response<ServiceInstance>> choose(Request request) {
HttpHeaders headers = (HttpHeaders) request.getContext();
ServiceInstanceListSupplier supplier = this.serviceInstanceListSupplierProvider.getIfAvailable(NoopServiceInstanceListSupplier::new);
return ((Flux) supplier.get()).next().map(list -> processInstanceResponse((List<ServiceInstance>) list, headers));
}
private Response<ServiceInstance> processInstanceResponse(List<ServiceInstance> instances, HttpHeaders headers) {
if (instances.isEmpty()) {
return new EmptyResponse();
} else {
String reqVersion = headers.getFirst("version");
if (StringUtils.isEmpty(reqVersion)) {
return processRibbonInstanceResponse(instances);
}
List<ServiceInstance> serviceInstances = instances.stream()
.filter(instance -> reqVersion.equals(instance.getMetadata().get("version")))
.collect(Collectors.toList());
if (serviceInstances.size() > 0) {
return processRibbonInstanceResponse(serviceInstances);
} else {
return processRibbonInstanceResponse(instances);
}
}
}
/**
* 负载均衡器
* 参考 org.springframework.cloud.loadbalancer.core.RoundRobinLoadBalancer#getInstanceResponse
*
* @author javadaily
*/
private Response<ServiceInstance> processRibbonInstanceResponse(List<ServiceInstance> instances) {
int pos = Math.abs(this.position.incrementAndGet());
ServiceInstance instance = instances.get(pos % instances.size());
return new DefaultResponse(instance);
}
}
Filter loading load balancing
public class GrayReactiveLoadBalancerClientFilter implements GlobalFilter, Ordered {
private static final Log log = LogFactory.getLog(ReactiveLoadBalancerClientFilter.class);
private static final int LOAD_BALANCER_CLIENT_FILTER_ORDER = 10150;
private final LoadBalancerClientFactory clientFactory;
private LoadBalancerProperties properties;
public GrayReactiveLoadBalancerClientFilter(LoadBalancerClientFactory clientFactory, LoadBalancerProperties properties) {
this.clientFactory = clientFactory;
this.properties = properties;
}
@Override
public int getOrder() {
return LOAD_BALANCER_CLIENT_FILTER_ORDER;
}
@Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
URI url = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR);
String schemePrefix = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_SCHEME_PREFIX_ATTR);
if (url != null && ("grayLb".equals(url.getScheme()) || "grayLb".equals(schemePrefix))) {
ServerWebExchangeUtils.addOriginalRequestUrl(exchange, url);
if (log.isTraceEnabled()) {
log.trace(ReactiveLoadBalancerClientFilter.class.getSimpleName() + " url before: " + url);
}
return this.choose(exchange).doOnNext((response) -> {
if (!response.hasServer()) {
throw NotFoundException.create(this.properties.isUse404(), "Unable to find instance for " + url.getHost());
} else {
URI uri = exchange.getRequest().getURI();
String overrideScheme = null;
if (schemePrefix != null) {
overrideScheme = url.getScheme();
}
DelegatingServiceInstance serviceInstance = new DelegatingServiceInstance((ServiceInstance) response.getServer(), overrideScheme);
URI requestUrl = this.reconstructURI(serviceInstance, uri);
if (log.isTraceEnabled()) {
log.trace("LoadBalancerClientFilter url chosen: " + requestUrl);
}
exchange.getAttributes().put(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR, requestUrl);
}
}).then(chain.filter(exchange));
} else {
return chain.filter(exchange);
}
}
protected URI reconstructURI(ServiceInstance serviceInstance, URI original) {
return LoadBalancerUriTools.reconstructURI(serviceInstance, original);
}
private Mono<Response<ServiceInstance>> choose(ServerWebExchange exchange) {
URI uri = (URI) exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR);
VersionGrayLoadBalancer loadBalancer = new VersionGrayLoadBalancer(clientFactory.getLazyProvider(uri.getHost(), ServiceInstanceListSupplier.class), uri.getHost());
if (loadBalancer == null) {
throw new NotFoundException("No loadbalancer available for " + uri.getHost());
} else {
return loadBalancer.choose(this.createRequest(exchange));
}
}
private Request createRequest(ServerWebExchange exchange) {
HttpHeaders headers = exchange.getRequest().getHeaders();
Request<HttpHeaders> request = new DefaultRequest<>(headers);
return request;
}
}
Inject filter
@Configuration
public class GrayGatewayReactiveLoadBalancerClientAutoConfiguration {
@Bean
@ConditionalOnMissingBean({
GrayReactiveLoadBalancerClientFilter.class})
public GrayReactiveLoadBalancerClientFilter grayReactiveLoadBalancerClientFilter(LoadBalancerClientFactory clientFactory, LoadBalancerProperties properties) {
return new GrayReactiveLoadBalancerClientFilter(clientFactory, properties);
}
}
Publish grayscale services
Production environment configuration fileapplication-prod.yml
server:
port: 9002
spring:
application:
name: provider
cloud:
nacos:
config:
server-addr: 127.0.0.1:8848
discovery:
metadata:
version: prod
server-addr: 127.0.0.1:8848
Grayscale environment configuration fileapplication-gray.yml
server:
port: 9003
spring:
application:
name: provider
cloud:
nacos:
config:
server-addr: 127.0.0.1:8848
discovery:
metadata:
version: gray
server-addr: 127.0.0.1:8848
idea startup parameters specify configuration file
Also pay attention to configuring the nacos files of the two environments ( prod version: 4, gray version: 5 )
test
Then start three services respectively: 9000 port gateway service, 9002 port production environment provider-prod service, 9003 port gray environment provider-gray service
E:\Nacos\nacos>curl http://127.0.0.1:9000/provider/test/port
port=9003, version=5
E:\Nacos\nacos>curl -X GET -H "version:prod" http://127.0.0.1:9000/provider/test/port
port=9003, version=5
E:\Nacos\nacos>curl -X GET -H "version:gray" http://127.0.0.1:9000/provider/test/port
port=9002, version=4
Warehouse Address
If you have any questions about the code in the article, you can directly check the author's repository below
Warehouse address: ReturnTmp/spring-gateway-demo: gateway configuration + grayscale release + configuration center sample warehouse (github.com)
Reference link
- Spring Cloud Gateway+nacos grayscale release-Nuggets (juejin.cn)
- SpringCloud gateway Actuator - shigp1 - Blog Park (cnblogs.com)
- Spring Cloud Gateway monitors Zhou Li's blog
- Front-end grayscale release implementation plan-Tencent Cloud Developer Community-Tencent Cloud (tencent.com)
- Backend - Grayscale release architecture design! Worth tasting
This article is published by OpenWrite, a blog that publishes multiple articles !