Inception of the company, business volume is small, a program taking care of all the business logic, this time a small number of servers, on-line simple, basic development - testing - is done on-line by the developer.
With the gradual increase in the amount of traffic, increasing function, the code amount is increased, and a single-line function need to recompile the entire program, compile time from a few seconds to minutes or even tens of minutes, on the one hand reduced efficiency, on the other hand lateral expansion bring processing performance to enhance the effect gradually weakened. Therefore, a unified split out of each sub-module, the program will be large and "micro-service" of.
The benefits of micro-services nature is self-evident, but many services are deployed, change is indeed a headache.
How to solve this problem? Baidu, a lot of service management, service standardization, process, various terms after another, but that is not a specific implementation, for me this novice really very friendly.
So combined with the specific business, to meet the online needs and achieve an automated publishing system.
background
Our development staff to complete the new features, the test is passed, then upload the package through the script, start the process and some other operations.
For business, it is not often change, so no problem really dry, but I Division has split out hundreds of micro-services, deployed on hundreds of hosts, and that such operations should be carried out dozens or even hundreds of times a day when the drawbacks began to unravel:
- Compiler package itself takes a long time, and then uploaded to the server and a long wait
- Face a bunch of fragmented script, though older drivers at ease, but these people really tired of repetitive tasks when released
- When unexpected situations requires a rollback, the operation was another fierce as a tiger
How to do it. . .
Package upload long time? Ahead of the packages in the warehouse, use the time to take a direct okay
bunch of discrete script? Naturally, it is to take over the script, provide a unified call interface ok not
difficult to roll back? Version management which version you want to use with which version
In order to achieve the release and version management, we need to record the version information services, service release a version of the host association and the services actually corresponding.
Existing host management is set up by the former chiefs, technology stack is Django 1.10.1 , very old version, in order to quickly realize the function on this basis to develop it.
Went ahead, the old lady to write code is a shuttle, ctrl + c, ctrl + v has reached a pinnacle.
Version Management Services
A service will be many versions, if a table to record all the information is bound to be a lot of redundant field, it split into service, service version two tables, make foreign key. Django corresponds to the following model:
Our code consists of gitlab management of self-built building is directly used gitlab-ci, so the service table corresponding to project the recorded address gitlab
from django.conf import settings
from django.db import models
class MicroService(models.Model):
LANGUAGE_TYPE = (
('cpp', 'cpp'),
('go', 'go'),
('python', 'python')
('other', 'other')
)
name = models.CharField(u'服务名称', max_length=64)
language = models.CharField(u'语言类型', max_length=16, choices=LANGUAGE_TYPE)
build_orig = models.CharField(u'构建来源', max_length=16, default='git')
build_url = models.CharField(u'构建地址', max_length=128) # gitlab or jenkins 的构建地址，用于触发构建任务
description = models.CharField(u'描述', max_length=256, null=True, blank=True)
created = models.DateTimeField(auto_now_add=True)
created_by = models.ForeignKey(settings.AUTH_USER_MODEL, related_name='created_by', on_delete=models.DO_NOTHING)
updated = models.DateTimeField(auto_now=True)
updated_by = models.ForeignKey(settings.AUTH_USER_MODEL, related_name='updated_by', on_delete=models.DO_NOTHING)
class Meta:
db_table = 'micro_service'
unique_together = (('name', ), )
ordering = ['-created']
When a building tasks gitlab trigger, a version of the service of the state who are prepared to build, run, success, failure, timeout, repeat construct can be used to enumerate the records state changes; not modified version is created, so only founder and creation time
from enum import Enum, unique
@unique
class BuildStatus(Enum):
pending = 0
building = 1
success = 2
timeout = 3
failed = 4
duplicate = 5
class MicroServiceVersion(models.Model):
BUILD_STATUS = tuple([(v.value, k) for k, v in BuildStatus.__members__.items()])
microservice = models.ForeignKey(MicroService, on_delete=models.DO_NOTHING)
version = models.CharField(u'版本', max_length=48) # 版本号格式: 时间戳-提交分支-提交id
description = models.CharField(u'描述', max_length=128, null=True, blank=True)
created = models.DateTimeField(auto_now_add=True)
created_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.DO_NOTHING)
status = models.PositiveSmallIntegerField(choices=BUILD_STATUS,default=1)
file_path = models.FilePathField(u'程序包存放路径', max_length=128, null=True, blank=True)
ref = models.CharField(u'构建分支', max_length=128, default='master')
class Meta:
db_table = 'micro_service_version'
unique_together=(('microservice', 'version'),)
ordering = ['-created', 'version']
Examples of service
Service will actually release a version associated hosts and services:
- Deployment, create a host in the instance table, the service version of the associated record
- Upgrade, rollback, modify the corresponding instance version
- Delete, delete data
status
When we deploy, upgrade, rollback, delete, etc., it may lead to failure due to some reasons, so use a field to record the state of the instance to facilitate the subsequent retry the operation.
locking
If user A is for instance service_a to upgrade operations, so other users can not operate service_a instance, you can lock all of these instances, the user A's operation after the completion unlocked
Corresponding model:
from asset.models import Asset
@unique
class InstanceStatus(Enum):
running = 0 # 运行中 大部分时候处于此状态
installing = 1
upgrading = 2
reverting = 3
deleting = 4
install_failed = 11
upgrade_failed = 12
revert_failed = 13
delete_failed = 14
class MicroServiceInstance(models.Model):
STATUS = tuple([(v.value, k) for k, v in InstanceStatus.__members__.items()])
microservice = models.ForeignKey(MicroService, on_delete=models.DO_NOTHING)
version = models.ForeignKey(MicroServiceVersion, on_delete=models.DO_NOTHING)
host = models.ForeignKey(Asset, on_delete=models.DO_NOTHING)
port = models.IntegerField(u'端口号', null=True, blank=True, )
tag = models.CharField(u'标签', max_length=64, blank=True, null=True)
weight = models.PositiveSmallIntegerField(u'权重', default=100)
description = models.CharField(u'描述', max_length=128, blank=True, null=True)
created = models.DateTimeField(auto_now_add=True)
created_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.DO_NOTHING)
status = models.PositiveSmallIntegerField(u'实例状态', choices=STATUS, default=0)
locked = models.BooleanField(u'操作锁定', default=False)
is_maintain = models.BooleanField(u'是否维护中', default=False) # 保留字段
class Meta:
db_table = 'micro_service_instance'
ordering = ['-created']
Clarify the relationship, version, instance, built a data model, the next step is the view.