GoFrame framework (rk-boot): Based on cloud native environment, distinguishing configuration files (Config)

introduce

Through a complete example, in the gogf/gf framework, the configuration files are distinguished according to the environment. That is, how to read different configuration files in environments such as [Test] and [Online].

We will use rk-boot to start the gogf/gf microservice .

Please visit the following address for the complete tutorial:

• https://rkdocs.netlify.app/cn

Install

go get github.com/rookie-ninja/rk-boot/gf

quick start

We will create three configuration files config/beijing.yaml, config/shanghai.yaml, config/default.yaml, and then read different files according to different environment variables.

rk-boot uses REALM, REGION, AZ, DOMAIN environment variables to distinguish different environments. This is also our recommended method of distinguishing cloud-native environments. For example, REALM="your business", REGION="Beijing", AZ="Beijing District 1", DOMAIN="test environment".

rk-boot integrates viper to handle configuration files.

1. Create a configuration file

• config/beijing.yaml

---
my-region: beijing

• config / shanghai.yaml

---
my-region: shanghai

• config/default.yaml

---
my-region: default

2. Create boot.yaml

The boot.yaml file tells rk-boot how to start the gogf/gf service.

We use config as the entry to the configuration file in boot.yaml, and can provide multiple config file paths.

Locale represents the environment of Config, we use locale to distinguish different Config.

Why does config.name use the same name?

We want to use the same code, but read different files, and we want the files to have different names. So different files are distinguished by locale. We will introduce the logic of locale in detail later.

config:
  # 默认
  - name: my-config
    locale: "*::*::*::*"
    path: config/default.yaml
  # 如果环境变量 REGION=beijing，读取此文件
  - name: my-config
    locale: "*::beijing::*::*"
    path: config/beijing.yaml
  # 如果环境变量 REGION=shanghai，读取此文件
  - name: my-config
    locale: "*::shanghai::*::*"
    path: config/shanghai.yaml
gf:
  - name: greeter
    port: 8080
    enabled: true

3. Create main.go

Set the environment variable: REGION="beijing", then read the configuration file, config/beijing.yaml will be read.

// Copyright (c) 2021 rookie-ninja
//
// Use of this source code is governed by an Apache-style
// license that can be found in the LICENSE file.
package main

import (
	"context"
	"fmt"
	"github.com/rookie-ninja/rk-boot"
	_ "github.com/rookie-ninja/rk-boot/gf"
	"os"
)

// Application entrance.
func main() {
	// Set REGION=beijing
	os.Setenv("REGION", "beijing")

	// Create a new boot instance.
	boot := rkboot.NewBoot()

	// Load config which is config/beijing.yaml
	fmt.Println(boot.GetConfigEntry("my-config").GetViper().GetString("my-region"))

	// Bootstrap
	boot.Bootstrap(context.Background())

	// Wait for shutdown sig
	boot.WaitForShutdownSig(context.Background())
}

4. Folder structure

$ tree
.
├── boot.yaml
├── config
│   ├── beijing.yaml
│   ├── default.yaml
│   └── shanghai.yaml
├── go.mod
├── go.sum
└── main.go

5. Verify

$ go run main.go

We will get the following output:

beijing

6. No matching environment variable found

If REGION="not-matched", i.e. no matching environment variable is found, the default configuration file (config/default.yaml) is read. Because the locale property of config/default.yaml is *::*::*::*

// Application entrance.
func main() {
    // Set REGION=not-matched
    os.Setenv("REGION", "not-matched")
    
    ...
    // Load config which is config/default.yaml
    fmt.Println(boot.GetConfigEntry("my-config").GetViper().GetString("my-region"))
    ...
}

We will get the following output:

$ go run main.go
default

7. The environment variable is not configured

If we did not configure the REGION environment variable, the config/default.yaml file would be read.

// Application entrance.
func main() {
    ...
    // Load config which is config/beijing.yaml
    fmt.Println(boot.GetConfigEntry("my-config").GetViper().GetString("my-region"))
    ...
}

We will get the following output:

$ go run main.go
default

concept

rk-boot uses REALM, REGION, AZ, DOMAIN four environment variables to distinguish configuration files.

These four environment variables can have arbitrary values.

Best Practices

For example, we have a [cloud album] business. This service can use different IP addresses of MySQL in different environments, so it can be configured in this way.

Architecture

Assume that our business has servers in [Beijing] and [Shanghai]. At the same time, in order to improve service availability, we have opened 2 districts in [Beijing] and [Shanghai].

At this time, we can configure the following environment variables on the machine, which can be set in batches through tools such as Ansible.

environment	Corresponding environment variable
Beijing, District 1, Test	REALM="cloud-album"，REGION="bj"，AZ="bj-1"，DOMAIN="test"
Beijing, District 1, Online	REALM="cloud-album"，REGION="bj"，AZ="bj-1"，DOMAIN="prod"
Beijing, District 2, Test	REALM="cloud-album"，REGION="bj"，AZ="bj-2"，DOMAIN="test"
Beijing, District 2, Online	REALM="cloud-album"，REGION="bj"，AZ="bj-2"，DOMAIN="prod"
Shanghai, District 1, Test	REALM="cloud-album"，REGION="sh"，AZ="sh-1"，DOMAIN="test"
Shanghai, District 1, Online	REALM="cloud-album"，REGION="sh"，AZ="sh-1"，DOMAIN="prod"
Shanghai, District 2, Test	REALM="cloud-album"，REGION="sh"，AZ="sh-2"，DOMAIN="test"
Shanghai, District 2, Online	REALM="cloud-album"，REGION="sh"，AZ="sh-2"，DOMAIN="prod"

At the same time, if we do not use services like ETCD, Consul and other services to remotely pull configuration files, we can directly add the following files to the machine. Each file has a different MySQL IP address.

folder structure

.
├── boot.yaml
├── config
│   ├── bj-1-test.yaml
│   ├── bj-1-prod.yaml
│   ├── bj-2-test.yaml
│   ├── bj-2-prod.yaml
│   ├── sh-1-test.yaml
│   ├── sh-1-prod.yaml
│   ├── sh-2-test.yaml
│   ├── sh-2-prod.yaml
│   └── default.yaml
├── go.mod
├── go.sum
└── main.go

boot.yaml

Next, we add the following config entry in boot.yaml.

config:
  # 默认入口
  - name: my-config
    locale: "*::*::*::*"
    path: config/default.yaml
  # 北京，一区，测试环境
  - name: my-config
    locale: "cloud-album::bj::bj-1::test"
    path: config/bj-1-test.yaml
  # 北京，一区，线上环境
  - name: my-config
    locale: "cloud-album::bj::bj-1::prod"
    path: config/bj-1-prod.yaml
  # 北京，二区，测试环境
  - name: my-config
    locale: "cloud-album::bj::bj-2::test"
    path: config/bj-2-test.yaml
  # 北京，二区，线上环境
  - name: my-config
    locale: "cloud-album::bj::bj-2::prod"
    path: config/bj-2-prod.yaml
  # 上海，一区，测试环境
  - name: my-config
    locale: "cloud-album::sh::sh-1::test"
    path: config/sh-1-test.yaml
  # 上海，一区，线上环境
  - name: my-config
    locale: "cloud-album::sh::sh-1::prod"
    path: config/sh-1-prod.yaml
  # 上海，二区，测试环境
  - name: my-config
    locale: "cloud-album::sh::sh-2::test"
    path: config/sh-2-test.yaml
  # 上海，二区，线上环境
  - name: my-config
    locale: "cloud-album::sh::sh-2::prod"
    path: config/sh-2-prod.yaml
gf:
  - name: greeter
    port: 8080
    enabled: true

Read the configuration file in main.go.

Because, all Configs are named my-config, when read in main.go, we can use my-config to get ConfigEntry.

package main

import (
	"context"
	"fmt"
	"github.com/rookie-ninja/rk-boot"
	"os"
)

// Application entrance.
func main() {
	// Create a new boot instance.
	boot := rkboot.NewBoot()

	// Get viper instance based on environment variable
	boot.GetConfigEntry("my-config").GetViper()

	// Bootstrap
	boot.Bootstrap(context.Background())

	// Wait for shutdown sig
	boot.WaitForShutdownSig(context.Background())
}

Override with environment variables

rk-boot integrates viper to process configuration files, so it naturally integrates all the functions that viper comes with.

This includes overriding the existing configuration [values] through [environment variables]. Let's look at an example.

1.config/default.yaml

In the config/default.yaml file, add a K/V.

---
endpoint: 8.8.8.8

2.main.go

Under normal circumstances, the following code will get [8.8.8.8], but we overwrite the value of [endpoint] through environment variables. Note that when using the environment variable override, the Key of the environment variable needs to be in uppercase English.

// Copyright (c) 2021 rookie-ninja
//
// Use of this source code is governed by an Apache-style
// license that can be found in the LICENSE file.
package main

import (
	"context"
	"fmt"
	"github.com/rookie-ninja/rk-boot"
	_ "github.com/rookie-ninja/rk-boot/gf"
	"os"
)

// Application entrance.
func main() {
	// Set ENDPOINT=localhost
	os.Setenv("ENDPOINT", "localhost")

	// Create a new boot instance.
	boot := rkboot.NewBoot()

	// Load config which is config/default.yaml
	fmt.Println(boot.GetConfigEntry("my-config").GetViper().GetString("endpoint"))

	// Bootstrap
	boot.Bootstrap(context.Background())

	// Wait for shutdown sig
	boot.WaitForShutdownSig(context.Background())
}

3. Verify

$ go run main.go
localhost

4. Use environment variable prefixes

In the actual environment, there may be a problem of conflicting environment variables. At this time, we can configure an [environment variable prefix] in Viper to mark our Config.

For example, assume that the system has already set HOSTNAME as an environment variable and initialized it to each machine. If we forcibly modify this value, we will encounter unpredictable errors. At this point, we can add a prefix.

example:

• config/default.yaml

---
hostname: my-hostname

• boot.yaml In the config options, add our ENV prefix.

config:
  - name: my-config
    locale: "*::*::*::*"
    path: config/default.yaml
    envPrefix: rk
gf:
  - name: greeter
    port: 8080
    enabled: true

• main.go At this time, we add [RK_] as a prefix when overwriting HOSTNAME through environment variables.

Refer to viper official documentation

// Copyright (c) 2021 rookie-ninja
//
// Use of this source code is governed by an Apache-style
// license that can be found in the LICENSE file.
package main

import (
	"context"
	"fmt"
	"github.com/rookie-ninja/rk-boot"
	_ "github.com/rookie-ninja/rk-boot/gf"
	"os"
)

// Application entrance.
func main() {
	// Set RK_HOSTNAME=override-hostname
	os.Setenv("RK_HOSTNAME", "override-hostname")

	// Create a new boot instance.
	boot := rkboot.NewBoot()

	// Load config which is config/default.yaml
	fmt.Println(boot.GetConfigEntry("my-config").GetViper().GetString("hostname"))

	// Bootstrap
	boot.Bootstrap(context.Background())

	// Wait for shutdown sig
	boot.WaitForShutdownSig(context.Background())
}

• verify

$ go run main.go
override-hostname

{{o.name}}

{{m.name}}