Clear architecture (Clean Architecture) of the micro-Go service: Program container (Application Container)

Clear architecture (Clean Architecture) framework is the idea of a quarantine program, the framework will not take over your application, but you decide when and where to use them. In this program, I deliberately do not use any framework when beginning, so I can better control the program structure. Only after the entire program structure layout is complete, I will consider replacing some of the components of this program by some libraries. Thus, the impact of the introduction of the framework or third-party libraries will be dependent on proper isolation of the relationship. Currently, in addition to the logger, database, gRPC and Protobuf (which is inevitable), I only used two third-party libraries OZZO-Validation ¹ and YAML ², while all other libraries are Go standard library.

You can use this program as a basis for building applications. You may ask, then this frame are we going to take over the entire application? Yes. But the fact is, whether you are self-built frame or the introduction of third-party frameworks, you need a basic framework as the basis for building applications. The foundation needs to have the correct dependencies and reliable design, then you can decide whether to introduce other libraries. Of course you can build yourself a framework, but you may end up spending a lot of time and effort to perfect it. You can also use this program as a starting point, instead of building their own projects, thus saving time and effort for you.

Procedure container is the most complex part of the project, is a critical component to the different parts of the application are bonded together. The rest of this procedure is straightforward and easy to understand, but this is not part of. The good news is, once you understand this part, then the whole program are well in hand.

Container package ( "container" package) components:

The container package consists of five parts:

"Container" ( "container") package: it is responsible for creating specific types and injecting them into other documents. Top package only one file "container.go", which defines the interface of the container.
"Servicecontainer" subpacket: implement container interface. Only one file "serviceContainer.go", which is the key "container" package. Here is the code. It begins with an "InitApp", it reads the configuration data from the file and set the logger (logger).

  type ServiceContainer struct {
      FactoryMap map[string]interface{}
      AppConfig  *config.AppConfig
  } 

  func (sc *ServiceContainer) InitApp(filename string) error {
      var err error
      config, err := loadConfig(filename)
      if err != nil {
          return errors.Wrap(err, "loadConfig")
      }
      sc.AppConfig = config
      err = loadLogger(config.Log)
      if err != nil {
          return errors.Wrap(err, "loadLogger")
      } 
      return nil
  }   
  // loads the logger
  func loadLogger(lc config.LogConfig) error {
      loggerType := lc.Code
      err := logFactory.GetLogFactoryBuilder(loggerType).Build(&lc)
      if err != nil {
          return errors.Wrap(err, "")
      }
      return nil
  }    
  // loads the application configurations
  func loadConfig(filename string) (*config.AppConfig, error) {  
      ac, err := config.ReadConfig(filename)
      if err != nil {
          return nil, errors.Wrap(err, "read container")
      }
      return ac, nil
  }

"Configs" sub-package: responsible for loading the program from the YAML configuration files, and save them to "appConfig" structure for container use.
"Logger" sub-package: it contains only one file "logger.go", which provides logging interface and a "Log" variable to access the logger. Because each file rely recording, so it requires a separate package to avoid circular dependency.
The last part is a different type of plant (factory).

Stratification and its internal application layer hierarchical match. For "usecase" and "dataservice" layer, there is "usecasefactory" and "dataservicefactory". Another plant is "datastorefactory", which is responsible for creating the underlying data link processing. Because the data provider may be gRPC or other types of services in addition to the database, it is called "datastorefactry" rather than "databasefactory". Logging component (logger) also has its own factory.

With Factories (Use Case Factory):

, But the specific type definition for each such "registration", interfaces are defined in "usecase" package used in Example "registration" subpackages "usecase" package. In addition, the package container has a corresponding factory is responsible for creating instances of specific use cases. For the "registration (Registration)" use case, it is the "registrationFactory.go". With use cases and the relationship between the plant is one to one embodiment. Specific types (concrete type) factory responsible for creating use cases for this use case and call the other members of the factory to create the required specific type (member in a struct). The lowest level is the specific type of sql.DBs and gRPC connection, they need to be passed to the persistence layer, in order to access the data in the database.

如果Go支持泛型，你可以创建一个通用工厂来构建不同类型的实例。现在，我必须为每一层创建一个工厂。另一个选择是使用反射（refection)，但它有不少问题，因此我没有采用。

“Registration” 用例工厂（Use Case Factory）:

每次调用工厂时，它都会构建一个新类型。以下是“注册（Registration）”用例创建具体类型的代码。它是工厂方法模式（factory method pattern）的典型实现。如果你想了解有关如何在Go中实现工厂方法模式的更多信息，请参阅此处³.

// Build creates concrete type for RegistrationUseCaseInterface
func (rf *RegistrationFactory) Build(c container.Container, appConfig *config.AppConfig, key string) (UseCaseInterface, error) {
    uc := appConfig.UseCase.Registration
    udi, err := buildUserData(c, &uc.UserDataConfig)
    if err != nil {
        return nil, errors.Wrap(err, "")
    }
    tdi, err := buildTxData(c, &uc.TxDataConfig)
    if err != nil {
        return nil, errors.Wrap(err, "")
    }
    ruc := registration.RegistrationUseCase{UserDataInterface: udi, TxDataInterface: tdi}

    return &ruc, nil
}

func buildUserData(c container.Container, dc *config.DataConfig) (dataservice.UserDataInterface, error) {
    dsi, err := dataservicefactory.GetDataServiceFb(dc.Code).Build(c, dc)
    if err != nil {
        return nil, errors.Wrap(err, "")
    }
    udi := dsi.(dataservice.UserDataInterface)
    return udi, nil
}

数据存储工厂（Data store factory）:

“注册（Registration）”用例需要通过数据存储工厂创建的数据库链接来访问数据库。所有代码都在“datastorefactory”子包中。我详细解释了数据存储工厂如何工作，请看这篇文章依赖注入（Dependency Injection）。

数据存储工厂的当前实现支持两个数据库和一个微服务，MySql和CouchDB，以及gRPC缓存服务; 每个实现都有自己的工厂文件。如果引入了新数据库，你只需添加一个新的工厂文件，并在以下代码中的“dsFbMap”中添加一个条目。


// To map "database code" to "database interface builder"
// Concreate builder is in corresponding factory file. For example, "sqlFactory" is in "sqlFactory".go
var dsFbMap = map[string]dsFbInterface{
    config.SQLDB:      &sqlFactory{},
    config.COUCHDB:    &couchdbFactory{},
    config.CACHE_GRPC: &cacheGrpcFactory{},
}

// DataStoreInterface serve as a marker to indicate the return type for Build method
type DataStoreInterface interface{}

// The builder interface for factory method pattern
// Every factory needs to implement Build method
type dsFbInterface interface {
    Build(container.Container, *config.DataStoreConfig) (DataStoreInterface, error)
}

//GetDataStoreFb is accessors for factoryBuilderMap
func GetDataStoreFb(key string) dsFbInterface {
    return dsFbMap[key]
}

以下是MySql数据库工厂的代码，它实现了上面的代码中定义的“dsFbInterface”。它创建了MySql数据库链接。

容器内部有一个注册表（registry），用作数据存储工厂创建的链接（如DB或gRPC连接）的缓存，它们在整个应用程序创建一次。无论何时需要它们，需首先从注册表中检索它，如果没有找到，则创建一个新的并将其放入注册表中。以下是“Build”代码。

// sqlFactory is receiver for Build method
type sqlFactory struct{}

// implement Build method for SQL database
func (sf *sqlFactory) Build(c container.Container, dsc *config.DataStoreConfig) (DataStoreInterface, error) {
    key := dsc.Code
    //if it is already in container, return
    if value, found := c.Get(key); found {
        sdb := value.(*sql.DB)
        sdt := databasehandler.SqlDBTx{DB: sdb}
        logger.Log.Debug("found db in container for key:", key)
        return &sdt, nil
    }

    db, err := sql.Open(dsc.DriverName, dsc.UrlAddress)
    if err != nil {
        return nil, errors.Wrap(err, "")
    }
    // check the connection
    err = db.Ping()
    if err != nil {
        return nil, errors.Wrap(err, "")
    }
    dt := databasehandler.SqlDBTx{DB: db}
    c.Put(key, db)
    return &dt, nil

}

Grpc Factory:

对于“listUser”用例，它需要调用gRPC微服务（缓存服务），而创建它的工厂是“cacheFactory.go”。目前，数据服务的所有链接都是由数据存储工厂创建的。以下是gRPC工厂的代码。 “Build”方法与“SqlFactory”的非常相似。


// DataStoreInterface serve as a marker to indicate the return type for Build method
type DataStoreInterface interface{}

// cacheGrpcFactory is an empty receiver for Build method
type cacheGrpcFactory struct{}

func (cgf *cacheGrpcFactory) Build(c container.Container, dsc *config.DataStoreConfig) 
     (DataStoreInterface, error) {
    key := dsc.Code
    //if it is already in container, return
    if value, found := c.Get(key); found {
        return value.(*grpc.ClientConn), nil
    }
    //not in map, need to create one
    logger.Log.Debug("doesn't find cacheGrpc key=%v need to created a new one\n", key)

    conn, err := grpc.Dial(dsc.UrlAddress, grpc.WithInsecure())
    if err != nil {
        return nil, errors.Wrap(err, "")
    }
    c.Put(key, conn)
    return conn, err
}

Logger factory:

Logger有自己的子包名为“loggerfactory”，其结构与“datastorefactory”子包非常相似。 “logFactory.go”定义了日志记录器工厂构建器接口（builder interface）和映射（map）。每个单独的日志记录器都有自己的工厂文件。以下是日志工厂的代码：

// logger mapp to map logger code to logger builder
var logfactoryBuilderMap = map[string]logFbInterface{
    config.ZAP:    &ZapFactory{},
    config.LOGRUS: &LogrusFactory{},
}

// interface for logger factory
type logFbInterface interface {
    Build(*config.LogConfig) error
}

// accessors for factoryBuilderMap
func GetLogFactoryBuilder(key string) logFbInterface {
    return logfactoryBuilderMap[key]
}

以下是ZAP工厂的代码。它类似于数据存储工厂。只有一个区别。由于记录器创建功能仅被调用一次，因此不需要注册表。

// receiver for zap factory
type ZapFactory struct{}

// build zap logger
func (mf *ZapFactory) Build(lc *config.LogConfig) error {
    err := zap.RegisterLog(*lc)
    if err != nil {
        return errors.Wrap(err, "")
    }
    return nil
}

配置文件:

配置文件使你可以全面了解程序的整体结构：

file

上图显示了文件的前半部分。第一部分是它支持的数据库配置; 第二部分是带有gRPC的微服务; 第三部分是它支持的日志记录器; 第四部分是本程序在运行时使用的日志记录器

下图显示了文件的后半部分。它列出了应用程序的所有用例以及每个用例所需的数据服务。

file

配置文件中应保存哪些数据？

不同的组件具有不同的配置项，一些组件可能具有许多配置项，例如日志记录器。我们不需要在配置文件中保存所有配置项，这可能使其太大而无法管理。通常我们只需要保存需要在运行时更改的选项或者可以在不同环境中（dev, prod, qa)值不同的选项。

设计是如何进化的?

容器包里似乎有太多东西，问题是我们是否需要所有这些？如果你不需要所有功能，我们当然可以简化它。当我开始创建它时，它非常简单，我不断添加功能，最终它才越来越复杂。

最开始时，我只是想使用工厂方法模式来创建具体类型，没有日志记录，没有配置文件，没有注册表。

我从用例和数据存储工厂开始。最初，对于每个用例，都会创建一个新的数据库链接，这并不理想。因此，我添加了一个注册表来缓存所有连接，以确保它们只创建一次。

然后我发现（我从这里获得了一些灵感⁵）将所有配置信息放在一个文件中进行集中管理是个好主意，这样我就可以在不改变代码的情况下进行更改。
我创建了一个YAML文件（appConfig [type] .yaml）和“appConfig.go”来将文件中的内容加载到应用程序配置结构（struct） - “appConfig”中并将其传递给工厂构建器（factory builder）。 “[type]”可以是“prod”，“dev”，“test”等。配置文件只加载一次。目前，它没有使用任何第三方库，但我想将来切换到Vipe⁶，因为它可以支持从配置服务器中动态重新加载程序配置。要切换到Vipe，我只需要更改一个文件“appConfig.go”。

对于日志记录，整个程序我只想要一个logger实例，这样我就可以为整个程序设置相同的日志配置。我在容器内创建了一个日志记录器包。我还尝试了不同的日志库来确定哪一个是最好的，然后我创建了一个日志工厂，以便将来更容易添加新的日志记录器。有关详细信息，请阅读日志管理⁷。

源程序:

完整的源程序链接 github: https://github.com/jfeng45/servicetmpl

索引:

[1] ozzo-validation

[2] YAML support for the Go language

[3][Golang Factory Method](https://stackoverflow.com/a/49714445)

[4][Go Microservice with Clean Architecture: Dependency Injection](https://jfeng45.github.io/posts/dependency_injection/)

[5] How I pass around shared resources (databases, configuration, etc) within Golang projects

[6][viper](https://github.com/spf13/viper)

[7][Go Microservice with Clean Architecture: Application Logging](https://jfeng45.github.io/posts/go_logging_and_error_handling/)