Konfig: Use ConfigMap to implement hot update of online configuration

Preface

Using kubernetes to deploy applications is becoming more and more popular, and how can the various configurations required by the services running in kubernetes be hot updated? Is it necessary to deploy services such as zookeeper or etcd in kubernetes? The solution adopted in this article is to use ConfigMap as a persistent solution for service configuration, and use the watch capability provided by kubernetes to actively discover ConfigMap updates and update them to the service configuration in time. In this way, the operation and maintenance personnel only need to use the kubernetes console (cli or web) to modify the configuration of the online service, such as modifying the log level, downgrading, adjusting the threshold, and so on.

This article refers to the source code https://github.com/jindezgm/konfig/blob/master/konfig.go

achieve

Konfig is a configuration tree implemented using a ConfigMap of kubernetes. Although ConfigMap.Data is of map[string]string type, konfig will perform further (recursive) yaml analysis of the value in ConfigMap.Data, provided that the value is " ---\n" at the beginning. The purpose of this design is to make konfig support multi-level configuration, while only one level of ConfigMap is not friendly in some usage scenarios. Of course, multi-level configuration can also be realized by using one level, but the multi-level is reflected in the key, such as: "abc", "cd", the author thinks that it is not visually elegant.

Konfig supports obtaining the same configuration in multiple types, which can be converted to a specified type as needed, as shown in the following interface definition:

// 所有接口都会返回配置的版本号，即ConfigMap.ResourceVersion，keys是多级的key，当keys为空时表示根，即整个ConfigMap.
type Interface interface {
    // 如果keys已经被注册某种类型(参看下面的RegValue接口)，则返回指定类型的值，否则返回原生类型的值。
	Get(keys ...string) (interface{}, int64)
    // 获取布尔型
	GetBool(keys ...string) (bool, int64)
    // 获取64、32位整型、
	GetInt64(keys ...string) (int64, int64)
	GetInt(keys ...string) (int, int64)
	GetInt32(keys ...string) (int32, int64)
    // 获取64、32位浮点型
	GetFloat64(keys ...string) (float64, int64)
	GetFloat32(keys ...string) (float32, int64)
    // 获取字符串
	GetString(keys ...string) (string, int64)
    // 将指定keys下的值注册为一种类型，配合Get()接口使用可以将keys下的值转换为注册的类型返回，其中tag是成员变量的tag名称，比如json
    RegValue(ptr interface{}, tag string, keys ...string) (interface{}, int64)
    // 获取指定类型的value，konfig会将map[string]interface{}转换为value对象，其中tag是成员变量的tag名称，比如json.
    GetValue(ptr interface{}, tag string, keys ...string) int64
    // 将指定的keys下值挂载/卸载到环境变量，
    MountEnv(keys ...string)
	UnmountEnv()
    // 获取版本号
	Revision() int64
}

When konfig gets the configuration of the specified type, in addition to the primitive type, it does its best to convert the primitive type to the specified type. The following are the type conversions supported by konfig:

• int, int32, int64: support floating-point to integer, string to integer (string->float64->intxx), boolean to integer (true:1, false:0)
• float32, float64: support string to float
• bool: support integer, floating-point conversion to boolean (non-zero: true, zero: false), support for string conversion to boolean (case-insensitive "True": true, case-insensitive "False": false)
• string: Support all types to convert to string, and use fmt.Sprintf("%v") to return

Konfig guarantees the atomicity of a single interface, but if two consecutive calls to the interface may return two different versions, it indicates that the ConfigMap has changed between the two calls to the interface. If the configuration parameters obtained by calling the interface twice have high requirements for version consistency, they need to be called again until the versions of all configurations are the same. The probability of this occurrence is relatively low, and most of them can be solved by re-calling, because the update frequency of ConfigMap is extremely low. However, this method seems a bit ugly. Users can define a struct for this type of configuration, and then get all the configuration at once through GetValue(). The author's common method is to define the entire ConfigMap as a type, and then read all the configurations at once. As shown in the following code:

import "github.com/jindezgm/konfig"

// 定义自己的配置类型
type MyConfig struct {
    Bool   bool   `json:"bool"`
    Int    int    `json:"int"`
    String string `json:"string"`
}

kfg, _ := konfig.NewWithClientset(...)
var my MyStruct 
var rev int64

// 应用引用配置的功能实现
for {
    // 版本发生更新时重新获取配置
    if r := kfg.Revision(); r > rev {
        // 此处的keys为空，这是将整个ConfigMap.Data映射为MyStruct
        rev = kfg.GetValue(&my, "json")
    }
    // 使用配置
    ...
}

Because the GetValue() interface will execute a process similar to Unmarshal, it has a certain overhead and is suitable for scenarios where the call frequency is not high. If high-frequency calls are required, it is recommended to apply the cache configuration (such as the code above), and decide whether to call this interface according to the revision. If the application wants to save these troublesome operations, it calls the RegValue() interface to register the type with konfig, and konfig parses it on demand. With the Get() interface, it can meet the high-frequency call scenario. As shown in the following code:

// 将"my"下的所有值注册为MyConfig类型
kfg.RegValue(&MyConfig{}, "json", "my")
for {
    // 每次引用直接调用Get，konfig保证一致性、隔离性以及原子性
    value, _ = kfg.Get("my")
    my := value.(*MyConfig)
    ...
}

In some scenarios, a function point only needs to refer to one configuration item, and the user can directly call the interface every time it refers to it, without having to cache the configuration in their own code (when the revision becomes large, read the configuration and update the cache), because konfig's Reading performance is still guaranteed. As shown in the following code, print according to the configuration:

for {
    if p, _ := kfg.GetBool("print"); p {
        fmt.Println("Hello world")
    }
}

Of course, if you are used to reading environment variables to get the configuration, and if the container updates the environment variables, it will cause the container to restart, then you can use the MountEnv() interface to mount the configuration to the environment variables, as shown in the following code:

// keys为空，将ConfigMap.Data挂载到环境变量. 需要注意的是，MountEnv()的keys下应该只有一级配置，如果是多级，konfig会用fmt.Sprintf("%v")进行格式化
kfg.MountEnv() 
defer kfg.UnmountEnv()
for {
    if strings.ToLower(os.Getenv("print")) == "true" {
        fmt.Println("Hello world")
    }
}

Konfig supports two creation methods: 1. Using Clientset; 2. Using SharedInformerFactory. The former is suitable for scenarios where the application does not need to access kubernetes and needs to create a separate clientset for konfig. You can refer to the official example code for this part; the latter is suitable for scenarios where the application needs to access kubernetes, then konfig shares the Informer with the application. In either case, you need to authorize the pod to read the ConfigMap.

So, the question is, why not hang the ConfigMap into a file? Answer: The timeliness is not good, because the ConfigMap update to the Pod file update may take several seconds (it seems to be 10 seconds in my memory). If the online configuration needs to be updated urgently (such as downgrading), it is not very useful, but konfig does not. this problem.

insufficient

1. Currently konfig recursive parsing only supports yaml format, in fact, json is also easy to support, it feels not necessary;
2. Konfig does not support callback, that is, after the ConfigMap is updated, the changed part will be called back to the user. The current solution is to solve it through revision;
3. Although GetValue() can get multiple configurations at once, all configurations need to be under one key. If multiple configurations of different branches of the configuration tree are obtained at one time, konfig does not support it yet, and it feels like it can be implemented with flag.FlagSet;
4. MountEnv() does not compare the difference between the two configurations, and then deletes the configuration that is not in the new ConfigMap. It simply sets the value in each ConfigMap to the environment variable; and UnmountEnv() does not delete the environment variable, just No longer update the environment variables; these can be achieved, but the necessity is not yet seen;