Spring Cloud Framework Learning-Spring Cloud Config + Bus Implementation Configuration Center

1. Introduction to Spring Cloud Config

Spring Cloud Config is a distributed system configuration management solution. Spring Cloud Config is divided into server and client. The server is also called the distributed configuration center, which is an independent microservice application used to connect to the configuration warehouse and provide access interfaces for the client to obtain configuration information, encryption/decryption information, etc. The client is each microservice application or infrastructure in the microservice architecture. They manage the configuration content of application resources and business through the designated configuration center, and obtain and load configuration information from the configuration center when they start.

The main functions of Spring Cloud Config:

• Centrally manage the configuration files of each environment and each microservice
• Provide server and client support
• After the configuration file is modified, it can take effect quickly
• Configuration files are managed through Git/SVn, which naturally supports version rollback.
• Supports high-concurrency queries and multiple development languages

2. Spring Cloud Config infrastructure

The infrastructure diagram of Spring Cloud Config is as follows:

Description of the elements of the Spring Cloud Config infrastructure:

• Remote Git repository : where configuration files are stored
• Config Server : Distributed configuration center, corresponding to the config-server project created above, in which the location of the Git warehouse to be connected and connection information such as account and password are specified.
• Local Git warehouse : In the file system of Config Server, every time a client requests to obtain configuration information, Config Server obtains the latest configuration from the Git warehouse to the local, and then reads and returns it from the local Git warehouse. When the remote warehouse cannot be obtained, the local content is returned directly.
• ServiceA-1, ServiceA-2 : Specific microservice applications, they specify the address of the Config Server, so as to obtain the configuration information used by the application itself from externalization. When these applications are started, they will request configuration information from the Config Server for loading.
• bus-refresh : bus-refresh This interface is used to notify the Config Client to refresh the configuration file after the Config Server and the Config Client are connected to the Spring Cloud Bus.

The execution flow of the client application to obtain configuration information from configuration management:

1. When the application starts, it requests configuration information from the Config Server according to the application name {application}, environment name {profile}, and branch name {label} configured in bootstrap.properties.
2. Config Server searches for configuration files based on the Git warehouse information it maintains and the configuration location information passed by the client .
3. git cloneDownload the found configuration information to the file system of the Config Server through the command
4. Config Server creates Spring's ApplicationContext instance, loads configuration files from the Git local warehouse, and finally reads the configuration content and returns it to the client application.
5. The client application loads the ApplicationContext instance after obtaining the external configuration file,The priority of this configuration content is higher than the configuration content inside the client Jar package, so the duplicate content in the Jar package will no longer be loaded。

Config Server cleverly git clonestores the configuration information locally to act as a cache. Even when the Git server cannot be accessed, the cached content in Config Server can still be used.

3. Example of use

The following demonstrates how to build a distributed configuration center based on Git storage, and demonstrates in the client how to specify the configuration center of the microservice application through configuration, and enable it to obtain configuration information from the configuration center and bind it to the code.

3.1 Preparations

Create a project on GitHub, the project name is: configRepo, and the client1-dev.properties, client1-prod.properties, and client1-test.properties configuration files are stored in the client1 directory of the project.

3.2 Create a configuration center

Based on the Spring Boot 2.2.6.RELEASE version, create a basic Spring Boot project named ConfigServer as the configuration center, and add the following dependencies to pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.6.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>top.javahai.learn.springcloud</groupId>
    <artifactId>config-server</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>config-server</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>1.8</java.version>
        <spring-cloud.version>Hoxton.SR6</spring-cloud.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-config-server</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

Add annotations to the project startup class @EnableConfigServerto enable the server function of Spring Cloud Config.

@SpringBootApplication
@EnableConfigServer
public class ConfigServerApplication {
    
    

    public static void main(String[] args) {
    
    
        SpringApplication.run(ConfigServerApplication.class, args);
    }
}

Configure basic service information and Git warehouse information for storing configuration files in the application.properties configuration file

spring.application.name=config-server
server.port=8081
# 配置文件仓库地址
spring.cloud.config.server.git.uri=https://github.com/JustCoding-Hai/configRepo.git
# 仓库路径下配置文件的目录，可以配置多个
spring.cloud.config.server.git.search-paths=client1
# 访问Git仓库的用户名和密码
spring.cloud.config.server.git.username=12598161612@aliyun.com
spring.cloud.config.server.git.password=

After starting the project, visit the address: http://localhost:8081/client1/dev, you can read the configuration file information.

The access address of the request configuration file has the following rules:

/{
    
    application}/{
    
    profile}/[{
    
    label}]
/{
    
    application}-{
    
    profile}.yml
/{
    
    application}-{
    
    profile}.properties
/{
    
    label}/{
    
    application}-{
    
    profile}.yml
/{
    
    label}/{
    
    application}-{
    
    profile}.properties

Parameter Description:

• applicaiton indicates the configuration file name
• profile indicates the configuration file profile, such as test, dev, prod
• label indicates the git branch, the parameter is optional, the default is master

Next, you can modify the configuration file and submit it to the GitHub repository, and then refresh the ConfigServer interface to get the latest configuration content in time.

3.3 Client configuration

Based on Spring Boot 2.2.6.RELEASE version, create a Spring Boot project named ConfigClient to use the configuration file corresponding to ConfigServer. pom.xml adds the following dependencies:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.6.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>top.javahai.learn.springcloud</groupId>
    <artifactId>config-client</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>config-client</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>1.8</java.version>
        <spring-cloud.version>Hoxton.SR6</spring-cloud.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

After the project is successfully created, add the bootstrap.properties configuration in the resources directory. The loading priority of the bootstrap.properties configuration file is higher than that of the application.properties. Use the bootstrap.properties configuration to obtain the externalized configuration first.

Add content The config-server location for obtaining configuration files is as follows:

server.port=8082
spring.application.name=client1
spring.cloud.config.profile=dev
spring.cloud.config.label=master
spring.cloud.config.uri=http://localhost:8081

Description of configuration items:

• spring.application.name: corresponds to the {application} part of the configuration file rule
• spring.cloud.config.profile: corresponds to the {profile} part of the configuration file rule
• spring.cloud.config.label: corresponds to the {label} part of the configuration file rule
• spring.cloud.config.uri: The address of the configuration center config-server

Create class HelloController and provide Hello interface for testing

@RestController
public class HelloController {
    
    


    /**
     * 获取配置文件的env值
     */
    @Value("${env}")
    String value;

    @GetMapping("/hello")
    public String hello() {
    
    
        return value;
    }

}

Start the project below, and visit http://localhost:8082/hello in the browser, and you can see that config-client has obtained the configuration information of config-server.

3.4 Common Configurations

3.4.1 Placeholder configuration

We can flexibly control the query directory using placeholders. For example, modify the config-server configuration file:

spring.cloud.config.server.git.search-paths={
    
    application}

spring.application.nameThe {application} placeholder here represents the application name, and represents the value of the attribute corresponding to the corresponding Config Client connected to the Config Server . When the Config Client client requests to obtain the configuration, the Config Server will spring.application.namefill the {application} placeholder according to the value of the client to locate the storage location of the configuration resource, so as to dynamically obtain the configuration of different locations according to the properties of the microservice application. Realize the effect that a microservice application corresponds to a directory.

In config-server, {profile} can also be used to represent the client spring.cloud.config.profile, and {label} can also be used to represent the client's spring.cloud.config.label

3.4.2 Configure local warehouse

After using the Git or SVN warehouse, the files will be stored in the local file system of the Config Server. By default, these files will be stored in a temporary directory prefixed with <Git warehouse name>, such as /tmp/<Git warehouse name>-<random number> directory.

We can configure the fixed location of the local warehouse through the following properties:

spring.cloud.config.server.git.basedir=
或者
spring.cloud.config.server.svn.basedir=

In addition, we can add the following configuration in config-server to indicate the location of the specified configuration file:

spring.cloud.config.server.native.search-locations=file://E:/properties/

Through the above file://method of setting the file address, Config Server will run as a local warehouse , which is convenient for us to develop and debug locally.

3.4.3 Configure the local file system

Although in actual development, configuration files are generally placed in the Git repository, but config-server also supports placing configuration files under the classpath (src/main/resource). The following configuration needs to be added to config-server:

spring.profiles.active=native

The above configuration means to let config-server search for configuration files from classpath instead of remote warehouse.
Properties specifying paths to search for configuration files:

spring.cloud.config.server.native.searchLocations=

3.4.4 Property Override

There is a "property override" feature in Config Server. The parameters configured through the following properties will not be modified by the Config Client. And when the Spring Cloud client obtains the configuration information from the Config Server, it will obtain the configuration information. For example, configure the name attribute:

spring.cloud.config.server.overrides.name=123

Through the "property override" feature, it is convenient to configure some common properties or default properties for Spring Cloud applications.

4. Security management

In order to prevent users from seeing the content of the configuration file directly by accessing config-server, we can use Spring Security to protect the interface of config-server. First add Spring Security dependency in config-server:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
        </dependency>

After adding dependencies, the configuration center can be protected without any other changes. You need to complete the login before accessing the config-server interface below.

When the configuration center starts, a random password will be generated and printed out. The default automatically generated password is not easy to remember, so we can configure the username and password in config-server. Add the following configuration to the bootstrap.properties configuration file of the config-server service:

spring.security.user.name=myadmin
spring.security.user.password=123

In addition, the following configuration needs to be added to the bootstrap.properties configuration file of the config-client service.

spring.cloud.config.username=myadmin
spring.cloud.config.password=123

5. Configuration file encryption and decryption

5.1 Common encryption schemes

Encryption schemes can generally be divided into two types: irreversible encryption and reversible encryption. Irreversible encryption means that it is theoretically impossible to calculate the plaintext from the encrypted ciphertext. Generally used in password encryption, common algorithms such as MD5 message digest algorithm, SHA secure hash algorithm.

Reversible encryption, just look at the name, you can infer the encryption method of the plaintext based on the encrypted ciphertext. Reversible encryption is generally divided into two types:

• Symmetric encryption
• asymmetric encryption

Symmetric encryption means that the encryption key and the decryption key are the same. Common algorithms include DES, 3DES, and AES. Asymmetric encryption means that the encryption key is different from the decryption key. The encryption key is called the public key, which can be told to anyone, and the decryption key is called the private key, which is kept for use by oneself. A common asymmetric encryption algorithm is RSA.

5.2 Symmetric encryption configuration

First of all, we need to install the unlimited length JCE (Unlimited Strength Java Cryptography Extension) version in the running environment of the configuration center. The JCE provided by default in the JRE is a limited-length version.

The download address of the unlimited-length JCE version corresponding to Java8 version https://www.oracle.com/cn/java/technologies/javase-jce8-downloads.html .

Unzip the downloaded file, and copy the decompressed jar package to the jdk installation directory: %JDK_HOME%\jre\lib\securityoverwrite the original file in the directory.

Then in the bootstrap.properties configuration file of config-server, add the following configuration key:

encrypt.key=test-key

Then, start config-server and visit the following address to check whether the key configuration is successful: http://localhost:8081/encrypt/status

Then, visit: http://localhost:8081/encrypt, this interface requires a POST request to access this address, which is used to encrypt plaintext, and use PostMan to test.
Store the encrypted plaintext in the Git repository, prefixed with {cipher}. The {cipher} prefix is ​​used to mark that the content is an encrypted value.

The unique endpoints for encryption and decryption provided by the configuration center are as follows:

• /encrypt/status: Endpoint to view the status of encryption functions
• /key: Endpoint to view keys
• /encrypt: The endpoint that encrypts the requested body content
• /decrypt: The endpoint to decrypt the requested body content

5.3 Asymmetric encryption configuration

Compared with symmetric encryption, the key generation and configuration of asymmetric encryption is relatively more complicated, but the security is higher than that of symmetric encryption. Asymmetric encryption requires us to first generate a key pair through the keytool tool.

keytool is a key and certificate management tool in the JDK that enables users to manage their own public/private key pairs and their associated certificates for self-authentication through digital signatures (users authenticate themselves to other users/services) or data Integrity authentication service.

Execute the following command on the cmd command line to generate the keystore:

keytool -genkeypair -alias config-server -keyalg RSA -keystore D:\springcloud\config-server.keystore

After the command is executed, copy the generated keystore file to the resources directory of the config-server. Then in the bootstrap.properties directory of config-server, add the following configuration:

#encrypt.key=test-key
encrypt.key-store.location=config-server.keystore
encrypt.key-store.alias=config-server
encrypt.key-store.password=111111
encrypt.key-store.secret=111111

Note that in the build node of pom.xml, add the following configuration to prevent the keystore file from being filtered out.

<resources>
<resource>
<directory>src/main/resources</directory>
<includes>
<include>**/*.properties</include>
<include>**/*.keystore</include>
</includes>
</resource>
</resources>

Restart the config-server service, and the test method is consistent with symmetric encryption.

6. Servitization

In the previous configurations, the address of config-server is directly written in config-client. The following implementation first registers config-server and config-client to the Eureka registration center, first starts Eureka, and then adds the following dependencies to these two services in order to allow both config-server and config-client to register with Eureka.

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>

Then configure the registration information in the application.properties configuration files of config-server and config-client.

eureka.client.service-url.defaultZone=http://localhost:1111/eureka

Add annotations to the config-server and config-client startup classes @EnableDiscoveryClientto register these two services with the Eureka registry.

Then modify the bootstrap.properties configuration file of config-client

# 下面三行配置，分别对应 config-server 中的 {application}、{profile}以及{label}占位符
spring.application.name=client1
spring.cloud.config.profile=dev
spring.cloud.config.label=master
server.port=8082

#开启服务发现
spring.cloud.config.discovery.enabled=true
# 配置 config-server 服务名称
spring.cloud.config.discovery.service-id=config-server

spring.security.user.name=myadmin
spring.security.user.password=123

7. Request failure fast response and retry

When config-client calls config-server, the request may fail. At this time, we can configure a request retry function.

To add the retry function to the client config-client, you only need to add the following dependencies to pom.xml:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-aop</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.retry</groupId>
            <artifactId>spring-retry</artifactId>
        </dependency>

Then modify the configuration file to enable quick response to failure. After the failure fast response is enabled, the client will first judge whether the Config Server acquisition is normal, and quickly respond to the failure content.

# 开启失败快速响应
spring.cloud.config.fail-fast=true

In order to deal with the request Config Server failure caused by intermittent problems such as network fluctuations, we can configure request retries through the following attributes to ensure service availability:

# 请求重试的初识间隔时间，单位为毫秒，默认为1000毫秒
spring.cloud.config.retry.initial-interval=1000
# 重试时间间隔乘数，默认为1.1，如初始间隔为1000毫秒，下一次间隔为1100毫秒
spring.cloud.config.retry.multiplier=1.1
# 最大重试次数，默认为6次
spring.cloud.config.retry.max-attempts=6
# 最大间隔时间，默认为2000毫秒
spring.cloud.config.retry.max-interval=2000

8. Dynamic refresh configuration

When the configuration file changes, config-server can detect the change in time, but config-client will not detect the change in time
. By default, config-client can only load the latest configuration file after restarting.

First add the following dependencies to config-client:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

Then add configuration to the application.properties configuration file to expose the refresh endpoint:

management.endpoints.web.exposure.include=refresh

The refresh endpoint is used to reacquire and refresh client application configuration information.

Finally, add the @RefreshScope annotation to the place where the configuration file is used in config-client, so that when the configuration changes, you only need to
call the refresh endpoint, and the configuration in config-client can be automatically refreshed.

@RefreshScope
@RestController
public class HelloController {
    
    


    /**
     * 获取配置文件的env值
     */
    @Value("${env}")
    String value;

    @GetMapping("/hello")
    public String hello() {
    
    
        return value;
    }

}

Restart the config-client service, as long as the configuration file changes, send a POST request to call this interface: /actuator/refresh. The configuration file will be refreshed automatically.

Through the dynamic refresh achieved by the above method, when the configuration file is changed, it is necessary to manually call/refresh requests one by one for all microservice applications to realize the real-time update of the configuration. When there are many microservice applications, this operation is more troublesome and easy error. We can use Spring Cloud Bus to implement notification of configuration changes in the form of a message bus, and complete batch configuration on the cluster.

9. Integrate Spring Cloud Bus

We can use Spring Cloud Bus to build a message bus in the microservice architecture. The message bus refers to the message topic that all microservice instances are connected to in the microservice architecture. We usually use a lightweight message broker (message middleware) to build a common message topic, and the messages generated in this topic will be monitored and consumed by all instances.

Because the instances on the bus can easily broadcast some messages that need to be known to other instances connected to the topic, we can integrate Spring Cloud Bus in Spring Cloud Config to implement real-time updates of configuration files.

9.1 Broadcast configuration file update

Spring Cloud Bus connects microservices through a lightweight message broker, which can be used to broadcast configuration file changes or manage service
monitoring.

First add Spring Cloud Bus dependencies to config-server and config-client respectively.

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bus-amqp</artifactId>
        </dependency>

Then configure the RabbitMQ connection in the application.properties configuration files of the config-server and config-client services.

spring.rabbitmq.host=192.168.91.128
spring.rabbitmq.port=5672
spring.rabbitmq.username=guest
spring.rabbitmq.password=guest

Add actuator dependency in config-server service

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

Then add the bus-refresh endpoint to the configuration file of config-server:

management.endpoints.web.exposure.include=bus-refresh

Since protection is added to all interfaces in config-server, the refresh interface will not be directly accessible. At this time, you can modify the permissions
of the endpoint by modifying the Security configuration:

@Configuration
public class SecurityConfig extends WebSecurityConfigurerAdapter {
    
    

    @Override
    protected void configure(HttpSecurity http) throws Exception {
    
    
        http.authorizeRequests()
                .anyRequest().authenticated()
                .and()
                .httpBasic()
                .and()
                .csrf().disable();
    }
}

In this configuration, HttpBasic login is enabled, so that when sending a refresh request, the authentication
information can be configured directly through HttpBasic. Finally, start config-server and config-client respectively, then modify the configuration information and submit it to GitHub, refresh the config-client interface, and check whether there is any change.

Then, send a POST request like this: http://localhost:8081/actuator/bus-refresh. This POST request is for config-server, and Config Server will pass the refresh instruction to RabbitMQ, and then RabbitMQ will pass the instruction to each Config Client.

The POST request from modifying the configuration in the Git repository to initiating /bus-refresh can be automatically triggered through the Web Hook of the Git repository. All applications connected to the message bus will receive the update request. Therefore, after the configuration file is updated, the problem of updating the configuration file of the Config Client needs to be requested one by one.

9.2 Specify refresh range

If you do not want every microservice to refresh the configuration file after updating the configuration file, you can use the following configuration to solve the problem.
First, add an instance-id to each config-client:

eureka.instance.instance-id=${spring.application.name}:${server.port}

You can refresh only a certain microservice in the following way, and the instance-id corresponds to the instance-id of the microservice configuration.

http://localhost:8081/actuator/bus-refresh/{
    
    instance-id}

Reference:
1. "Spring Cloud Microservice Actual Combat"