Apollo practical experience in the infrastructure

This article from the submission Wei super students, if you have a good articles are also welcome to contact me.

Micro Service Configuration Center Apollo Guide, the following documents from the consolidation in accordance with apollo wiki, part of the instructions and code transformation best practices based on experience from the author of consolidation, any questions welcome communication.

Configuration Center

Before split into micro-service architecture, once the monomer applications only need to manage a set of configuration. After the split micro service, each system has its own configuration, and are not the same, but because of the need to service management, some configuration also need to be able to dynamically change, such as service parameters require adjustment or fusing current limiting function, configuration Center is to solve this problem.

The basic concept of configuration

Configuration is a read-only variables independent of the program
- With applications behave differently in different configurations
- Applications should not change configuration
Configuring accompany applications throughout the life cycle
- Initialization parameters and operating parameters
There are many ways you can configure load
Configuration management needs
- Access control (application level, edit, publish isolation, etc.)
- Multi-cluster configuration management environment
- Component configuration management framework classes

Configuration Center

Registration and Anti-Registration Configuration
Configuration management
Subscribe to configuration changes

Spring Environment

Environment Spring is an abstract container for application environments are two key factors (profile & properties) of.

profile

profile is a logical packet, when a bean is registered into the container, the configuration is only effective when activated.

## 配置文件使用
spring.profiles.active=xxx 

## 硬编码注解形式使用
@org.springframework.context.annotation.Profile
复制代码

properties

Properties in almost all applications have played an important role, and can come from a variety of sources: properties file, JVM system properties, system environment variables, JNDI, Servlet Context parameters, ad-hoc Properties Object, Map, etc. . Properties Relationship Environment and is convenient for the user to provide a service interface to configure the attributes of the source and parse property from them.

Extension point in Spring
- spring framework provides a convenient way to add custom data source Spring Enviroment added to the policy, as @PropertySource. docs.spring.io/spring/docs...
- spring boot provides related expansion mode, as EnviromentPostProcessor relevant. docs.spring.io/spring-boot...
- spring boot environment also provides custom extensions before you begin. docs.spring.io/spring-boot...

About Apollo

Brief introduction

Apollo (Apollo) is a research and development department Ctrip framework open source configuration management center that centralized management applications in different environments, different cluster configuration, the configuration changes can be pushed to real-time application side, and have standardized permissions, process control and other features.

Apollo supports four dimensions Management Configuration Key-Value format:

application (application)

This is well understood, the application configuration is actually used, Apollo runtime client needs to know who is the current application, so you can go get the corresponding configuration. Each application requires a unique identity, we believe that the application identity is to go along with the code, so you need to configure in code, specific information see the Java Client Guide.

environment (environment)

Corresponding to the configuration environment, Apollo runtime client needs to know what is in the current application environment, which can be configured to acquire application. We believe that the environment and independent of the code, the same code deployed in different environments should be able to get to configure different environments, so that the environment is the default configuration specified by reading (env attribute server.properties in) on the machine, but in order to facilitate the development, we also support designated by the System Property and other run-time, specific information see the Java client Guide.

cluster (cluster)

A packet applications in different instances, such as a typical data center may be divided in accordance with, the application example Shanghai room into a cluster, the application example Beijing room into another cluster. Cluster different, the same configuration can have different values, such as zookeeper address. The default cluster is designated by reading configuration (idc property server.properties in) on the machine, but also specify the run-time support through System Property, specific information see the Java Client Guide.

namespace (namespace)

A packet applications in different configurations, can be simply analogized to the namespace file, a different type of configuration is stored in different files, configuration files such as databases, the RPC configuration files, the configuration files of the application itself. Application of the configuration can be read directly namespace common component, such as DAL, RPC like. Applications may also be arranged to be adjusted by a common component inherits configuration namespace common components, such as the initial number of database connections DAL.

Meanwhile, Apollo model development based on open source, open source Address: github.com/ctripcorp/a...

The base model

Below are just Apollo base model:

User configuration has been modified in the configuration and release center
Configuration Center Apollo notify the client has a configuration update
Apollo clients pull the latest configuration from the configuration center, and notification to update the local configuration application

Apollo Architecture Description

Apollo project itself on the use of Spring Boot & Spring Cloud development.

Server

The figure of Apollo briefly describes the overall design, we can see from the bottom up:

Config Service provides configurable read, push and other functions, is the object Apollo client service.
Admin Service provides configuration changes, publishing and other functions, clients are Apollo Portal (Management Interface).
Config Service and Admin Service are multi-instance, stateless deployment, you need to register yourself and to keep the heartbeat Eureka
Eureka we stand on the floor Meta Server service package for Eureka found the interface Client Access Meta Server Config Service to obtain a list of services (IP + Port) through the domain name, and then directly through the IP + Port access services, but also in the Client side do load balance, error retry
Portal domain names Meta Server Admin Service to obtain a list of services (IP + Port), and then directly through the IP + Port access services, while at the Portal side will do load balance, error retry
To simplify deployment, we will actually Config Service, Eureka and Meta Server three logical roles deployed in the same JVM process.

Client

The client and server to maintain a persistent connection, so that it can push the first time to obtain the updated configuration.
The client will be timed from the Apollo latest configuration center server configuration pull applications.
- This is a fallback mechanism, resulting in the failure to prevent the push mechanism is not configured to update
- The client will report timing of a local version of pulling, so that under normal circumstances, the timing for the operation of the pull, the server returns 304 - Not Modified
- The default timing frequency of once every 5 minutes, pull, by the client can be specified at run time System Property: apollo.refreshInterval covered, in minutes.
After the client obtains from the Apollo configuration center server to the latest configuration of the application, it will be kept in memory
The client will get from the server to configure the cache to a local file system
- In the face of service is not available, or when the network does not make sense, still from the local recovery configuration
Applications for the latest configuration from Apollo clients, subscribe to configure update notification

Long asynchronous connection is achieved using the + polling achieve, achieve specific resolution, please see the following two articles

service notifications

client polling

Apollo High Availability deployment

In the Apollo architecture specification we mentioned client and portal are in the client load balancing based on ip + port to access the service, so the config service and admin service is stateless, scalable level, portal service based on the use of binding slb multiple servers to achieve switching, meta server empathy. | Scene | influence | downgrade | reason | |: ---- |: ---- |: ---- |: ---- | | a config service station off the assembly line | No effect | | Config service stateless The client reconnecting other config service | | all config service off the assembly line | client can not read the latest configuration, without affecting Portal | client restart can be read locally cached profile | | | an admin service station off the assembly line | no effect | | Admin service stateless, Portal reconnection other admin service | | all admin service offline | no effect on the client, portal can not update the configuration | | | | a station off the assembly line portal | no effect | | Portal domain by slb binding multiple servers, server and try again point to available | | all portal offline | no effect on the client, portal can not update the configuration | | | | a data center offline | no effect | | multi-data center deployment, data fully synchronized, Meta Server / Portal domain by slb switch automatically to the other surviving data center |

Apollo instructions

Instructions for use

Apollo Guide

Java Client Guide

Best Practices

In Spring Boot & Spring Cloud in.

Each application requires a unique identity, we believe that the application identity is to go along with the code, so you need to configure your code. On the application of identity, identification of the third-party middleware application should be uniform, and extended support apollo identity spring.application.name consistent (see specific fusion-config-apollo code), other middleware empathy.
As used during application development code configuration should be fully utilized Spring Environment Profile, logical grouping local increase in local, non-local logic development stage of the closing packet. While closing the apollo remote access configuration, -Denv = local increase in VM options in.

The following code is an extension of apollo application identification using spring.application.name , and increased monitoring configuration, monitoring is generally the infrastructure team functions, from basic hard-coded frame up, the business side did not fully perceived.

import com.ctrip.framework.apollo.ConfigService;
import com.ctrip.framework.apollo.spring.config.PropertySourcesConstants;
import com.ctrip.framework.foundation.internals.io.BOMInputStream;
import com.ctrip.framework.foundation.internals.provider.DefaultApplicationProvider;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.SpringApplicationRunListener;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.core.annotation.Order;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.EnumerablePropertySource;
import org.springframework.core.env.PropertiesPropertySource;
import org.springframework.util.StringUtils;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.nio.charset.StandardCharsets;
import java.util.Properties;
import java.util.Set;
/**
 * ApolloSpringApplicationRunListener
 * <p>
 * SpringApplicationRunListener
 * 接口说明 https://blog.csdn.net/u011179993/article/details/51555690https://blog.csdn.net/u011179993/article/details/51555690
 *
 * @author Weichao Li ([email protected])
 * @since 2019-08-15
 */
@Order(value = ApolloSpringApplicationRunListener.APOLLO_SPRING_APPLICATION_RUN_LISTENER_ORDER)
@Slf4j
public class ApolloSpringApplicationRunListener implements SpringApplicationRunListener {
    public static final int APOLLO_SPRING_APPLICATION_RUN_LISTENER_ORDER = 1;
    private static final String APOLLO_APP_ID_KEY = "app.id";
    private static final String SPRINGBOOT_APPLICATION_NAME = "spring.application.name";
    private static final String CONFIG_CENTER_INFRA_NAMESPACE = "infra.monitor";
    public ApolloSpringApplicationRunListener(SpringApplication application, String[] args) {
    }
    /**
     * 刚执行run方法时
     */
    @Override
    public void starting() {
    }
    /**
     * 环境建立好时候
     *
     * @param env 环境信息
     */
    @Override
    public void environmentPrepared(ConfigurableEnvironment env) {
        Properties props = new Properties();
        props.put(PropertySourcesConstants.APOLLO_BOOTSTRAP_ENABLED, true);
        props.put(PropertySourcesConstants.APOLLO_BOOTSTRAP_EAGER_LOAD_ENABLED, true);
        env.getPropertySources().addFirst(new PropertiesPropertySource("apolloConfig", props));
        // 初始化appId
        this.initAppId(env);
        // 初始化基础架构提供的默认配置，需在项目中关联公共 namespaces
        this.initInfraConfig(env);
    }
    /**
     * 上下文建立好的时候
     *
     * @param context 上下文
     */
    @Override
    public void contextPrepared(ConfigurableApplicationContext context) {
    }
    /**
     * 上下文载入配置时候
     *
     * @param context 上下文
     */
    @Override
    public void contextLoaded(ConfigurableApplicationContext context) {
    }
    @Override
    public void started(ConfigurableApplicationContext context) {
    }
    @Override
    public void running(ConfigurableApplicationContext context) {
    }
    @Override
    public void failed(ConfigurableApplicationContext context, Throwable exception) {
    }
    /**
     * 初始化 apollo appId
     *
     * @param env 环境信息
     */
    private void initAppId(ConfigurableEnvironment env) {
        String apolloAppId = env.getProperty(APOLLO_APP_ID_KEY);
        if (StringUtils.isEmpty(apolloAppId)) {
            //此处需要判断一下 meta-inf 下的文件中的 app id
            apolloAppId = getAppIdByAppPropertiesClasspath();
            if (StringUtils.isEmpty(apolloAppId)) {
                String applicationName = env.getProperty(SPRINGBOOT_APPLICATION_NAME);
                if (!StringUtils.isEmpty(applicationName)) {
                    System.setProperty(APOLLO_APP_ID_KEY, applicationName);
                } else {
                    throw new IllegalArgumentException(
                            "config center must config app.id in " + DefaultApplicationProvider.APP_PROPERTIES_CLASSPATH);
                }
            } else {
                System.setProperty(APOLLO_APP_ID_KEY, apolloAppId);
            }
        } else {
            System.setProperty(APOLLO_APP_ID_KEY, apolloAppId);
        }
    }
    /**
     * 初始化基础架构提供的配置
     *
     * @param env 环境信息
     */
    private void initInfraConfig(ConfigurableEnvironment env) {
        com.ctrip.framework.apollo.Config apolloConfig = ConfigService.getConfig(CONFIG_CENTER_INFRA_NAMESPACE);
        Set<String> propertyNames = apolloConfig.getPropertyNames();
        if (propertyNames != null && propertyNames.size() > 0) {
            Properties properties = new Properties();
            for (String propertyName : propertyNames) {
                properties.setProperty(propertyName, apolloConfig.getProperty(propertyName, null));
            }
            EnumerablePropertySource enumerablePropertySource =
                    new PropertiesPropertySource(CONFIG_CENTER_INFRA_NAMESPACE, properties);
            env.getPropertySources().addLast(enumerablePropertySource);
        }
    }
    /**
     * 从 apollo 默认配置文件中取 app.id 的值，调整优先级在 spring.application.name 之前
     *
     * @return apollo app id
     */
    private String getAppIdByAppPropertiesClasspath() {
        try {
            InputStream in = Thread.currentThread().getContextClassLoader()
                    .getResourceAsStream(DefaultApplicationProvider.APP_PROPERTIES_CLASSPATH);
            if (in == null) {
                in = DefaultApplicationProvider.class
                        .getResourceAsStream(DefaultApplicationProvider.APP_PROPERTIES_CLASSPATH);
            }
            Properties properties = new Properties();
            if (in != null) {
                try {
                    properties.load(new InputStreamReader(new BOMInputStream(in), StandardCharsets.UTF_8));
                } finally {
                    in.close();
                }
            }
            if (properties.containsKey(APOLLO_APP_ID_KEY)) {
                String appId = properties.getProperty(APOLLO_APP_ID_KEY);
                log.info("App ID is set to {} by app.id property from {}", appId, DefaultApplicationProvider.APP_PROPERTIES_CLASSPATH);
                return appId;
            }
        } catch (Throwable ignore) {
        }
        return null;
    }
}
复制代码

Dynamic refresh

Apollo configured to support auto-refresh types, support @Value @RefreshScope @ConfigurationProperties and logging levels dynamically refreshed. Specific code to view the link below.

@Value

@Value Apollo itself to support the dynamic refreshing to note that if @Value use SpEL expressions, dynamic refresh will fail.

// 支持动态刷新
@Value("${simple.xxx}")
private String simpleXxx;
// 不支持动态刷新
@Value("#{'${simple.xxx}'.split(',')}")
private List<String> simpleXxxs;
复制代码

@RefreshScope

RefreshScope (org.springframework.cloud.context.scope.refresh) is a special scope achieved Spring Cloud provided for realizing the configuration, examples of the thermal load.

Dynamic implementation process:

When configuration changes, call refreshScope.refreshAll () or specify bean. Extraction of standard parameters (System, jndi, Servlet) all parameter variables outside the original parameters in the Environment put under a new Spring Context container reload, close the new container after bin. Parameter extraction (exclusion criteria parameters) updated, to compare the change item, event publishing environment changes, RefreshScope regenerate Bean with the new environmental parameters. Rebuild process is very simple, clear refreshscope destroy the cache Bing Bean, the next will re-acquire a new instance from BeanFactory (This example uses the new configuration).

@ConfigurationProperties

apollo default is not supported ConfigurationProperties refresh, refresh this need with EnvironmentChangeEvent.

Log Level

apollo default log level is not supported refresh, refresh this need with EnvironmentChangeEvent.

EnvironmentChangeEvent（Spring Cloud 提供）

When observed EnvironmentChangeEvent, it will have a list of keys that have changed, the application will use the following: 1, rebind any @ConfigurationProperties bean context, code, see org.springframework.cloud.context.properties.ConfigurationPropertiesRebinder . Any property 2, for the logging.level. * The logger level code, see org.springframework.cloud.logging.LoggingRebinder. Support dynamic refresh

import com.ctrip.framework.apollo.model.ConfigChangeEvent;
import com.ctrip.framework.apollo.spring.annotation.ApolloConfigChangeListener;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.context.environment.EnvironmentChangeEvent;
import org.springframework.cloud.context.scope.refresh.RefreshScope;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.context.annotation.Configuration;
/**
 * LoggerConfiguration
 *
 * @author Weichao Li ([email protected])
 * @since 2019/11/14
 */
@Configuration
@Slf4j
public class ApolloRefreshConfiguration implements ApplicationContextAware {
    private ApplicationContext applicationContext;
    @Autowired
    private RefreshScope refreshScope;
    @ApolloConfigChangeListener
    private void onChange(ConfigChangeEvent changeEvent) {
        applicationContext.publishEvent(new EnvironmentChangeEvent(changeEvent.changedKeys()));
        refreshScope.refreshAll();
    }
    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        this.applicationContext = applicationContext;
    }
}
复制代码

Note that the original configuration if the log level needs to be initialized.

import com.ctrip.framework.apollo.Config;
import com.ctrip.framework.apollo.spring.annotation.ApolloConfig;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.logging.LogLevel;
import org.springframework.boot.logging.LoggingSystem;
import org.springframework.context.annotation.Configuration;
import javax.annotation.PostConstruct;
import java.util.Set;
/**
 * logging 初始化
 *
 * @author Weichao Li ([email protected])
 * @since 2019/11/14
 */
@Configuration
@Slf4j
public class LoggingConfiguration {
    private static final String LOGGER_TAG = "logging.level.";
    private static final String DEFAULT_LOGGING_LEVEL = "info";
    @Autowired
    private LoggingSystem loggingSystem;
    @ApolloConfig
    private Config config;
    @PostConstruct
    public void changeLoggingLevel() {
        Set<String> keyNames = config.getPropertyNames();
        for (String key : keyNames) {
            if (containsIgnoreCase(key, LOGGER_TAG)) {
                String strLevel = config.getProperty(key, DEFAULT_LOGGING_LEVEL);
                LogLevel level = LogLevel.valueOf(strLevel.toUpperCase());
                loggingSystem.setLogLevel(key.replace(LOGGER_TAG, ""), level);
            }
        }
    }
    private static boolean containsIgnoreCase(String str, String searchStr) {
        if (str == null || searchStr == null) {
            return false;
        }
        int len = searchStr.length();
        int max = str.length() - len;
        for (int i = 0; i <= max; i++) {
            if (str.regionMatches(true, i, searchStr, 0, len)) {
                return true;
            }
        }
        return false;
    }
}
复制代码

Apollo Best Practices - configuration management

Access control

Because of the configuration change the behavior of the program, even incorrect configuration can lead to disaster, so the modification must have a better system of access control. Application and configuration management has improved rights management mechanism, the management configuration is also divided into two aspects of editing and publishing, thereby reducing human error. All operations have audit logs, you can easily track down the problem

everyone should have their own accounts (the most important pre-conditions)
Each project has at least one owner (project administrator, project administrator has the following privileges)
- Permissions can be assigned project management
- You can create a cluster
- You can create Namespace
Project Administrator (owner) assign configuration permissions it is according to organizational structure
- Edit permission allows users to create on the Apollo interface, modify, delete configuration
  - After configuration changes only change in the Apollo interface, it does not affect the actual configuration of the application
- Posted permission allows users to publish on the Apollo interface, configuration rollback
  - Configuration only released after the rollback action will be applied to practical use
  - Apollo publish user operation, real-time notification to the application after the rollback operation, and the latest configuration takes effect
Project management authority administrator interface

Create a project finished, edit and publish no default permissions assigned configuration, project managers need to be authorized.

Click this namespace authorization application button

Assign permissions to modify

Assign publishing rights

Namespace

Namespace classification authority

apollo obtain permission classified into private and public.

private (proprietary)

Namespace private rights, the application can only be acquired belongs to. An application attempts to acquire other applications of private Namespace, Apollo will be reported "404" exception.

public (public)

Namespace public authority, any application can be acquired.

The classification Namespace

Namespace There are three types of private type, common type, association type (type of inheritance).

Apollo private type Namespace instructions

Private types of Namespace have private rights. Services such as the default "application" Namespace is a private type.

scenes to be used

The configuration of the service itself (such as databases, business practices and other configurations)

How to use private type Namespace

A packet applications in different configurations, can be simply analogized to the namespace file, a different type of configuration is stored in different files, such as database profile, service configuration attributes, profiles, etc.

Apollo common type Namespace instructions

Common types of Namespace have public access. Common types of Namespace equivalent configuration free from outside applications, and the de-identified by name Namespace Namespace public, so the public name of the Namespace must be globally unique.

scenes to be used

Department-level shared configuration
Sub-level shared configuration
Shared between several projects configuration
Middleware client configuration

How to use common type Namespace

Code invasive

@EnableApolloConfig({"application", "poizon-infra.jaeger"})
复制代码

Configuration form

# will inject 'application' namespace in bootstrap phase
apollo.bootstrap.enabled = true
# will inject 'application', 'poizon-infra.jaeger' namespaces in bootstrap phase
apollo.bootstrap.namespaces = application,poizon-infra.jaeger
复制代码

Apollo instructions associated type Namespace

Type of association may be called inherited type, type of association with a private authority. Namespace type associated to a common type inherits Namespace, for covering a certain configurations public Namespace.

Recommendations

Unified configuration of the base frame portion, as commonly used configurations of DAL
Configuring common components of infrastructure, such as monitoring, fuse and other public component configuration