Day943. Continuous integration pipeline - system refactoring in practice

Continuous Integration Pipeline

Hi, I am 阿昌, what I learned to record today is about 持续集成流水线content.

From the perspective of teamwork, in the process of version release, there are often problems that testing relies on development to manually generate artifacts, and version release also comes from local development. Moreover, if the project architecture evolves from a monomer to a componentized architecture, as more and more components are separated, the previous build may be able to produce products, but after componentization, multiple components need to be built first, and then the components are integrated. The complexity of collaboration will also be higher.

The final consequence is that the team's collaboration efficiency is low, and the quality of the version cannot be controlled. The daily development work of development students is often interrupted, and they become veritable "packaging engineers".

How to solve these problems? The best way is to create a reliable and repeatable software release process, and automate the entire process as much as possible, thereby improving the overall integrated release efficiency. Reduce low-value repetitive work through automation.

---

1. Continuous integration pipeline

A continuous integration pipeline is a software development practice.

As shown in the figure below, whenever the code is submitted for development, the pipeline will be triggered to execute the corresponding steps. These steps usually include scanning inspection, building, testing, deployment and other links.

If the submitted code does not meet the checks set on the pipeline, the execution of the pipeline will fail and the code will not be merged into the warehouse.

So what value can the use of continuous pipeline bring to the team?

• On the one hand, there is a significant increase in efficiency. Before using the continuous integration pipeline, the build and release of the version had to rely on the local build. If multiple versions need to be built in a day, the efficiency is very low. With the assembly line, it can help us automatically 完成这些低价值的工作. In addition, collaboration norms can also be formed within the team, and everyone is subject to the version pushed by the pipeline, which can reduce unnecessary communication.
• Another aspect is obvious 质量提升. There are many uncontrollable factors in the locally built version, such as the development of the local build process, whether the latest code is pulled, whether the signature is correct, etc., may affect the quality of the final product. With the continuous integration pipeline, there will be a unified construction environment and a standardized construction process, effectively ensuring stable product quality. In addition, corresponding quality access control can be set on the pipeline, such as static code inspection, automated test verification, architecture guard verification, etc. When the submitted code does not meet the requirements, it can not be allowed to enter the warehouse, which helps the team to find related errors as soon as possible .

---

To get the best results from the pipeline, the key is for the team to be able to pay attention to the running status of the pipeline and respond in a timely manner. Below are two assembly line disciplines that I need to follow in practice .

• The first one is that if the pipeline fails to build, the code is not allowed to be submitted, and it should be repaired as soon as possible. When the pipeline construction fails, the cause of the failure should be checked immediately and repaired as soon as possible to avoid affecting the version. If it cannot be repaired in a short time, the code should be rolled back in time, so as not to affect the code integration of other members of the team.
• The second is to ensure that the last pipeline build is successful before leaving get off work every day. This discipline is called CI red not staying overnight. Usually, it is necessary to ensure that there is at least one version available for testing every day, so as not to affect the next day's testing and relevant personnel to get the version for verification.

---

2. Sharing pipeline design

For a componentized architecture, since multiple components are separated, it is more obvious to manage the release and integration of components through automation.

If this part of the complexity cannot be transferred through automation, splitting components may be another burden for the team, requiring frequent manual management of component build releases, component integration releases, and other issues. Therefore, it is necessary to design the assembly line of the component architecture to realize the automation of component management. Sharing to see how to design the pipeline.

First of all, it is necessary to classify the pipeline and split it into component pipeline and base pipeline.

The design of the component assembly line is mainly divided into 4 operation steps, namely quality access control inspection, code inspection, component product management and release notification.

The specific steps are as follows.

1. After the component developer submits the code, the pipeline quality access control inspection is automatically triggered. The quality access control includes code scanning, compilation, and automated testing.
2. The code is merged after the architect in the team inspects it.
3. Publish the latest component version to the maven artifact repository.
4. Notify relevant stakeholders of build results and artifacts.

The design of the base pipeline is mainly divided into 5 operation steps, which are submitting the component integration list, performing quality access control inspection, code inspection, releasing the product to the version library, and finally confirming the release.

1. The product team submits the component integration version list according to the version planning.
2. Quality access control includes code inspection, compilation, integration testing, performance testing, etc.
3. The architect in the team inspects, and the product manager confirms the code and merges it in.
4. Publish the latest application version to the artifact repository.
5. After the team confirms, select the channel (internal test, gray scale, application market, full volume) for release.

In the actual process, each step of the pipeline can be flexibly adjusted according to the project situation and the perfection of internal tools.

The design of the pipeline should be the same as the code, which can be continuously iterated, not static.

For example, generally speaking, the final release stage will not be automated, and manual testing and confirmation is required before the pipeline is triggered for release and deployment.

---

3. Example of using GitHub Action to build a pipeline

There are currently many continuous integration tools, such as Jenkins, GitLab CI, GitHub Action, AzureDevOps, etc. Since the Sharing project is currently hosted on GitHub, GitHub Action is used to demonstrate how to build a pipeline. Let's first look at how to use GitHub Action 配置the pipeline .

GitHub Action yml 配置文件uses the method to describe the build status of CI, just create the corresponding yml file in the .github/workflows directory.

You can refer to the following code, and specific comments are noted on each configuration.

//配置工作流的名称
name: Sharing CI
//配置当master分支有PR或者代码提交时会触发该流水线
on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]
//配置运行的任务
jobs:
  build:
//配置运行的服务器环节，一般使用默认
    runs-on: ubuntu-latest
//配置流水线的步骤
    steps:
//配置系统的构建环境
    - uses: actions/checkout@v3
    - name: set up JDK 11
      uses: actions/setup-java@v3
      with:
        java-version: 11
        distribution: 'temurin'
        cache: gradle
//检查Gradle运行环境
    - name: Grant execute permission for gradlew
      run: chmod +x gradlew
//执行静态代码扫描，使用lint检查
    - name: Code Scan 
      run: ./gradlew app:lint
//执行代码架构检查
    - name: Arch Test 
      run: ./gradlew app:testDUT --tests com.jkb.junbin.sharing.ArchRuleTest
//执行功能自动化冒烟测试   
 - name: Smoke Test 
      run: ./gradlew app:testDUT --tests com.jkb.junbin.sharing.SmokeTesting
//构建版本
    - name: build
      run: ./gradlew app:assembleRelease  
//推送版本到artifact制品库
    - name: Upload apk to artifact
      uses: actions/upload-artifact@v3
      if: always()
      with:
        name: sharing_app
        path: ${
    
    {
    
     github.workspace }}/app/build/outputs/apk/release/*.apk

After the configuration of the pipeline is completed, the running of the pipeline will be triggered when the code is merged, and the corresponding construction status can be viewed in the Action option of the project.

In addition, in addition to automatic triggering, you can also click to enter the details to manually re-trigger the execution of the pipeline.

After the pipeline is successfully executed, you can view the built artifacts in the details.

It is especially recommended to use configuration files to define pipelines. On the one hand, changes and codes of pipelines can be included in version management. In addition, since configuration files and codes are managed together, development colleagues can pay more attention to pipelines. design.

At present, mainstream continuous integration tools all support the way of defining pipelines using configuration files, but the syntax of different tool configurations is slightly different, and the overall pipeline design is universal.

---

Four. Summary

Continuous pipeline is a software development practice that aims to create a stable and repeatable process for software release through automation.

The effect of pipelining is obvious. In terms of efficiency, it helps reduce low-value repetitive work, such as local compilation and packaging, and also reduces unnecessary communication between team members.

From the quality point of view, the unified build and release environment will make the whole environment more reliable and 减少了人工操作bring less unexpected risks. In addition, combined with the pipeline to increase the quality access control, the code quality can be checked before the version is released, and the code that does not meet the specifications and requirements can be avoided from being merged into the code warehouse.

Of course, to make the pipeline play its best role, it is necessary to rely on the team members to follow the discipline of the pipeline to ensure that the pipeline does not run overnight, and can be repaired in time when the operation fails.

For the pipeline design of the component architecture, a hierarchical and hierarchical approach can be adopted. Specifically, it manages the construction and version management of components by designing the pipeline of components, and then designs the integrated pipeline to manage the integration and release of components.

Through GitHub Action, I will show you how to build a continuous integration pipeline. GitHub Action uses yml configuration to define the pipeline. This method is more friendly to developers. You only need to describe the steps in a specific syntax format. At the same time, this Configuration files can also be versioned along with code.

In practice, it is recommended to use this configuration file to define the pipeline.

---