A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open source at any moment, without compromising any credentials.
Configuration software development is an ancient and useful concept, we need to control how code running through configuration, such as buffer time, address database, and so on.
We have long been using the configuration file to store configuration items, the software reads the configuration file at startup and configuration items loaded into memory, you can read the configuration item from memory during the software operation. If you need to change, simply modify the configuration file and re-publishing service can, in this way been used for several decades until the distributed system along with the advent of the Internet age.
For a distributed application for re-publishing service means that restart services distributed across dozens or even thousands of servers, but to re-publish the entire system is a time-consuming process and may cause fluctuations in the overall system, which obviously not elegant. To do this requires a can dynamically adjust the configuration without the need for ways to restart the application.
Dynamic Configuration
Dynamic configuration includes two main concepts:
Treated differently arranged with the code, the configuration is no longer part of the code, the same code can exhibit different functions in different configurations;
Updating the configuration code update strictly separated process configuration is no longer coupled with DI DC / configure can be updated independently.
Configuration Center implements these two concepts, provides centralized configuration management services through a separate micro basic services.
Configuration Center and Python
For JAVA applications, the distribution center and integrated development framework has been very mature, such as Spring or Spring Boot integrates open source Netflix Archaius and Spring cloud, such as Ctrip another open source Apollo distribution center can also be integrated into the Spring framework.
But for Python developers, service integration has not yet a mature open source development framework program and the distribution center, distribution center for the elegant use dynamic configuration, we integrated the Consul client development framework for Python.
Consul Configuration Center
Consul briefly introduce themselves. Consul is an open source distributed K / V system, it can be used to implement discovery and configuration service registration center. Here is the Consul architecture diagram.
And etcd, zookeeper and so on, like, Consul running several server nodes to maintain the consistency of the system, configuration items that is stored in memory server node and provide external services to read and write, another can be run on each server node to forward a request agent the server cluster, so that the service can not know the specific location Consul cluster. We are used to provide restful api agent node to read the latest configuration data. These api can access our services to the Consul cluster.
Practice, the process only api access is not enough, and we set out from the performance, reliability, operation and maintenance of three, is also designed to achieve a real-time push, timing synchronization and local backup, the client deployed three functions.
Real-time push
The traditional configuration file can be loaded into memory configuration items, configuration items directly read service in memory of the easiest and quickest, but after the traditional configuration file modifications need to restart the service, distribution center to overcome this shortcoming but not at the expense of performance loss process, we still want to read the configuration is simple and fast. At the same time, for performance reasons, the service should not directly request Consul cluster to get the latest data, that there is no way we store data to the Consul cluster can reach read speeds from the latest configuration of the memory of it?
We started a process before each service starts this service to specialized maintenance corresponding configuration data into memory and this memory for shared services use the machine by way of the IPC. This process is called watch the process.
watch进程会向Consul集群发送请求读取最新的配置项并记下这份数据的index，在下一次发送请求时会带上这个index，Consul集群收到请求后会阻塞请求30秒，在这30秒内如果数据有变化就立即返回，如果在30秒后没有变化就返回timeout断开连接，watch进程如果收到返回就更新内存中的数据，如果收到timeout就发起下一次请求。这样通过watch进程就可以实时的将最新数据维护在内存当中了，因为服务进程是通过IPC的方式从本机内存中读取配置项，所以读取速度可以与从进程内存中读取相当。并且我们将这个过程封装起来，对外暴露一个代理对象，将这个对象注入到服务进程中，所以对于服务的开发者而言，读取最新的配置项就是读取这个代理对象的属性，非常简单透明。
watch进程本身实际上就是一个事件循环，所以我们采用tornado提供的IoLoop来实现watch进程，并通过tornado的coroutine实现了异步的request，我们保证watch进程足够简单进而足够可靠。
定时同步与本地备份
如何保证在watch进程挂掉后服务还能从内存中获取新的数据？
我们在服务启动之前会再启动一个心跳进程，心跳进程会每隔15分钟获取一次全量数据，并将数据更新到内存中，这样我们的服务就有了双保险，watch与心跳机制任何一个失效都不会影响到微服务。
但是还有一种情况会影响到服务的运行，那就是当Consul集群不可用时，虽然这发生的概率很低。为此心跳进程在更新完内存之后会将数据再更新到本地的文件中，当整个Consul集群都不可用时，如果服务不重启依然可以从内存中获取最后一份数据，即使服务重启也可以从本地文件中读取备份数据到内存中。第三重机制保障了服务的可用性不会受到Consul集群可用性的影响。
客户端的部署
如何部署这个客户端呢？最直接的方式是在每台服务器上都部署一份客户端Agent。但我们并没有这样选择，原因有两点：首先这会增加运维成本，这是我们不愿意看到的，其次客户端是为本机的服务提供代理服务的所以没有必要设计成常驻的进程。
所以我们将客户端的启动嵌入到服务的启动中，一旦服务代码中声明使用了Consul配置中心，客户端就会在服务启动之前启动，并读取一份最新的配置到内存中，紧接着我们的服务就可以启动了，同样的在服务关闭之后客户端进程也会跟着关闭，这样做的原因是我们的服务器并非固定发布一种服务，所以我们自然不希望在服务发布后有其他服务的Consul客户端还在继续运行。通过这种方式我们的客户端在不增加任何运维成本的前提下提供了Consul集群的代理服务。