Enterprise-level front-end construction practice under cloud native

Language 2023-05-04 16:22:36 views: null

foreword

Containers, k8s, and cloud native are becoming more and more popular now. What changes have occurred in the construction and release of front-end projects in the era of cloud native? Compared with traditional build and deployment platforms, what are the differences of cloud native? If you are interested in enterprise-level project construction under the traditional construction and deployment platform, you can first take a look at [Enterprise-level front-end construction practice under the unified deployment platform].

Today I mainly talk about the cloud-native construction solution based on open source wheels (docker/k8s/rancher/gitlab ci)

complete process

The complete process is shown in the figure below:

1. The code is hosted in gitlab, borrowing ci cd of gitlab;

2. The ci of gitlab is mainly responsible for the build packaging of the project itself, as well as the packaging of the docker image build-docker;

3. Push the packaged docker image to the private source harbor;

4. Differentiate the environment, call the http interface of rancher, and download the docker image on the harbor to deploy the specified project of the specified cluster. Rancher also controls the k8s cluster through kubectl

    ┌──────────────┐               ┌───────────────┐
    │              │               │               │
    │    harbor    ├──────────────▲│     rancher   │
    │              │▼──────────────┤               ├──────────────┐
    │              │  docker pull  │               │              │
    └──────────────┘               └───────────────┘       kubectl│
           ▲                               ▲                      │
           │                               │                      │
           │                               │                      │
           │                               │                      │
           │build-docker                   deploy-test1           │
           │                               deploy-release         │
           │                               ......                 │
           │                               │                      │
     build │                               │                      │                         
           │                               │                      │
┌──────────┴────────────┐                  │         ┌────────────┴─────────────┐
│                       │                  │         │                          │
│                       │                  │         │                          │
│                       │                  │         │                          │
│        gitlab         ├──────────────────┘         │       k8s cluster        │
│                       │                            │                          │
│                       │                            │                          │
│                       │                            │                          │
└───────────────────────┘                            └──────────────────────────┘

Complete process of CI and CD

I will introduce the details using the gitlab ci yml file of a certain project as an example.

1. First, the required variables are defined, including rancher and harbor;

2. Execute the pre-script before_script to determine the environment variables and the name of the packaged image through the branch. The image name of the ordinary branch will not carry the hash, because basically the test environment does not need to be rolled back. The environment variable is RUNTIME_ENV, which will be brought into the container through the subsequent docker build.

3. Define the whole stage is divided into build, build-docker, deploy-test1, deploy-online. Carry out project packaging, image packaging, deployment test environment and deployment of online environment respectively.

4. Define the job executed in each stage. For example, deploy is to call the rancher interface and pass different variables.

image: docker:18.06
 
 
variables:
  # rancher 变量
  RANCHER_NAMESPACE: kef2e
  RANCHER_PROJECT: kef2e
  RANCHER_WORKLOAD: ke-cms
  # harbor 变量
  PROJECT: kef2e
  IMAGE_NAME: ke-cms
  # 部署重要变量
  RANCHER_URL: https://someurl
  RANCHER_URL_ONLINE: https://someurl
  RANCHER_TOKEN: yourtoken
  RANCHER_TOKEN_ONLINE: yourtoken
  RANCHER_PROJECT_ID: yourid
  RANCHER_PROJECT_ID_ONLINE: yourid

 
services:
  - name: docker:18.06-dind
    alias: docker
 
stages:
  - build-spa
  - build-docker
  - deploy
  - deploy_online
 

  before_script:
    - echo exec before_script
    # 现在gitlab版本没有 CI_COMMIT_SHORT_SHA 变量(v>11.7)，进行兼容
    - if [ "$CI_COMMIT_SHORT_SHA" = "" ]; then CI_COMMIT_SHORT_SHA=${CI_COMMIT_SHA:0:8} ;fi;
    # 将branch name里的/转换为tag name支持的-
    - CI_COMMIT_REF_SAFE_NAME=$(echo $CI_COMMIT_REF_NAME | sed 's/\//-/')
    # 镜像的仓库地址
    - IMAGE_URL=$DOCKER_REG/$PROJECT/$IMAGE_NAME:$CI_COMMIT_REF_SAFE_NAME
    - if [[ "$CI_COMMIT_REF_SAFE_NAME" != "release" ]]; then RUNTIME_ENV=development ; fi
    - if [[ "$CI_COMMIT_REF_SAFE_NAME" == "release" ]] || [[ "$CI_COMMIT_REF_SAFE_NAME" == "pre" ]]; then IMAGE_URL=$DOCKER_REG/$PROJECT/$IMAGE_NAME:$CI_COMMIT_REF_SAFE_NAME-$CI_COMMIT_SHORT_SHA; fi
    - echo image url:$IMAGE_URL
    - echo image RUNTIME_ENV:$RUNTIME_ENV

.deploy_tpl: &deploy_def
  script:
    - CONTAINER_URL=$RANCHER_URL/v3/projects/$RANCHER_PROJECT_ID/workloads/deployment:$RANCHER_NAMESPACE:$RANCHER_WORKLOAD
    - curl -u $RANCHER_TOKEN -X GET $CONTAINER_URL -o deployment.json -k
    - TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
    - sed -i 's#image\":[^,]*,#image\":\"'$IMAGE_URL'\",#' deployment.json
    - sed -i 's/"cattle.io\/timestamp":"[^"]*"/"cattle.io\/timestamp":"'${TIMESTAMP}'"/g' deployment.json
    - curl -u $RANCHER_TOKEN -XPUT -H "Accept:application/json" -H "Content-Type:application/json" [email protected] $CONTAINER_URL -k
    - echo RANCHER_URL:$RANCHER_URL
    - echo RANCHER_TOKEN:$RANCHER_TOKEN
    - echo RANCHER_PROJECT_ID:$RANCHER_PROJECT_ID
    - echo CONTAINER_URL:$CONTAINER_URL


# 打包spa
build-spa:
  stage: build-spa
  tags:
    - k8s
  image: harbor-registry.inner.youdao.com/ke-test/agent-base:node12
  script:
    - echo "package build directory"
    - yarn config set strict-ssl false
    - yarn config set registry http://f2enpm.inner.youdao.com/
    # https://github.com/cnpm/cnpmjs.org/issues/1246， puppeteer在CI可能install有问题
    - yarn config set puppeteer_download_host https://nexus3.corp.youdao.com/repository/npm-all/
    - yarn config set puppeteer_skip_chromium_download true
    - yarn install
    # 这里为项目线上的构建命令，根据项目自行修改
    - yarn release
  cache:
    key: "$CI_COMMIT_REF_SLUG"
    paths:
      - node_modules/
  artifacts:
    name: "$CI_COMMIT_REF_SLUG"
    paths:
      - dist
    when: always
    expire_in: 6 mos
 
# 构建镜像
build-docker:
  retry: 2
  stage: build-docker
  tags:
    - k8s
  script:
    # 登陆 dockerhub
    - echo $DOCKER_REG_PASSWD | docker login $DOCKER_REG -u $DOCKER_REG_USER --password-stdin
    # 镜像地址
    - echo $IMAGE_URL
    - docker build -t $IMAGE_URL --build-arg RUNTIME_ENV=$RUNTIME_ENV .
    - docker push $IMAGE_URL
    - docker logout $DOCKER_REG
 
 
# 部署rancher
deploy_rancher:
  stage: deploy
  image: $CURL_IMAGE
  tags:
    - k8s
  variables:
    MSG: "正式部署到 $RANCHER_CLUSTER:$RANCHER_NAMESPACE \n部署地址：$RANCHER_URL/p/$RANCHER_PROJECT_ID/workload/deployment:$RANCHER_NAMESPACE:$RANCHER_WORKLOAD-$CI_COMMIT_REF_NAME \n"
  script:
  <<: *deploy_def
  # 手动触发rancher部署
  when: manual
  
# 部署rancher online
deploy_rancher_online:
  stage: deploy_online
  image: $CURL_IMAGE
  tags:
    - k8s
  variables:
    RANCHER_URL: $RANCHER_URL_ONLINE
    RANCHER_PROJECT_ID: $RANCHER_PROJECT_ID_ONLINE
    RANCHER_TOKEN: $RANCHER_TOKEN_ONLINE
    MSG: "正式部署到 $RANCHER_CLUSTER:$RANCHER_NAMESPACE \n部署地址：$RANCHER_URL/p/$RANCHER_PROJECT_ID/workload/deployment:$RANCHER_NAMESPACE:$RANCHER_WORKLOAD-$CI_COMMIT_REF_NAME \n"
  script:
  <<: *deploy_def
  # 手动触发rancher部署
  only:
    - release
  when: manual

Through the above process, the construction and deployment of a front-end project can be completed, but there are still some problems. The meaning of enterprise level is to pursue the highest maintainability and reduce personalization as much as possible. And if we have more than 20 projects, we need such configuration yml one by one, wouldn't it be messed up? Therefore, abstract unified configuration is imperative.

CI and CD after unified configuration

Through the analysis, it can be concluded that except for the build stage, which may be inconsistent, the other stages are the same, except that the variable names such as the project id of rancher are different.

The yml file after unified configuration is as follows:

Except for the build stage, other stages and before_script have been extracted to remote management and loaded by include.

If the extreme can ensure that the things done in the build phase are consistent, you can even extract all of them and only leave the variable name configuration. Due to the large number of our projects, the cost-effectiveness of personalized transformation is not high, so the build is temporarily saved.

image: docker:18.06
 
 
variables:
  # rancher 变量
  RANCHER_NAMESPACE: kef2e
  RANCHER_PROJECT: kef2e
  RANCHER_WORKLOAD: ke-cms
  # harbor 变量
  PROJECT: kef2e
  IMAGE_NAME: ke-cms
  # 部署重要变量
  RANCHER_URL: https://someurl
  RANCHER_URL_ONLINE: https://someurl
  RANCHER_TOKEN: yourtoken
  RANCHER_TOKEN_ONLINE: yourtoken
  RANCHER_PROJECT_ID: yourid
  RANCHER_PROJECT_ID_ONLINE: yourid

 
services:
  - name: docker:18.06-dind
    alias: docker
 
stages:
  - build
  - build-docker
  - deploy-test1
  - deploy-online

include: 
    - 'https://g.hz.netease.com/api/v4/projects/49427/repository/files/before_script.yml/raw?ref=master&private_token=YOUR_TOKEN&ext=.yml'
    - 'https://g.hz.netease.com/api/v4/projects/49427/repository/files/job.yml/raw?ref=master&private_token=YOUR_TOKEN&ext=.yml'

# 打包spa
build-spa:
  stage: build
  tags:
    - k8s
  image: harbor-registry.inner.youdao.com/ke-test/agent-base:node12
  script:
    - echo "package build directory"
    - yarn config set strict-ssl false
    - yarn config set registry http://f2enpm.inner.youdao.com/
    # https://github.com/cnpm/cnpmjs.org/issues/1246， puppeteer在CI可能install有问题
    - yarn config set puppeteer_download_host https://nexus3.corp.youdao.com/repository/npm-all/
    - yarn config set puppeteer_skip_chromium_download true
    - yarn install
    # 这里为项目线上的构建命令，根据项目自行修改
    - yarn release
  cache:
    key: "$CI_COMMIT_REF_SLUG"
    paths:
      - node_modules/
  artifacts:
    name: "$CI_COMMIT_REF_SLUG"
    paths:
      - dist
    when: always
    expire_in: 6 mos

The abstracted two remote scripts are as follows:

before_script.yml is used to do some pre-operations, such as defining the image name according to conditions:

default:
  before_script:
    - echo exec before_script
    # 现在gitlab版本没有 CI_COMMIT_SHORT_SHA 变量(v>11.7)，进行兼容
    - if [ "$CI_COMMIT_SHORT_SHA" = "" ]; then CI_COMMIT_SHORT_SHA=${CI_COMMIT_SHA:0:8} ;fi;
    # 将branch name里的/转换为tag name支持的-
    - CI_COMMIT_REF_SAFE_NAME=$(echo $CI_COMMIT_REF_NAME | sed 's/\//-/')
    # 镜像的仓库地址
    - IMAGE_URL=$DOCKER_REG/$PROJECT/$IMAGE_NAME:$CI_COMMIT_REF_SAFE_NAME
    - if [[ "$CI_COMMIT_REF_SAFE_NAME" != "release" ]]; then RUNTIME_ENV=development ; fi
    - if [[ "$CI_COMMIT_REF_SAFE_NAME" == "release" ]] || [[ "$CI_COMMIT_REF_SAFE_NAME" == "pre" ]]; then IMAGE_URL=$DOCKER_REG/$PROJECT/$IMAGE_NAME:$CI_COMMIT_REF_SAFE_NAME-$CI_COMMIT_SHORT_SHA; fi
    - echo image url:$IMAGE_URL
    - echo image RUNTIME_ENV:$RUNTIME_ENV

And job.yml is used to store some public tasks:

.deploy_tpl: &deploy_def
  script:
    - CONTAINER_URL=$RANCHER_URL/v3/projects/$RANCHER_PROJECT_ID/workloads/deployment:$RANCHER_NAMESPACE:$RANCHER_WORKLOAD
    - curl -u $RANCHER_TOKEN -X GET $CONTAINER_URL -o deployment.json -k
    - TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
    - sed -i 's#image\":[^,]*,#image\":\"'$IMAGE_URL'\",#' deployment.json
    - sed -i 's/"cattle.io\/timestamp":"[^"]*"/"cattle.io\/timestamp":"'${TIMESTAMP}'"/g' deployment.json
    - curl -u $RANCHER_TOKEN -XPUT -H "Accept:application/json" -H "Content-Type:application/json" [email protected] $CONTAINER_URL -k
    - echo RANCHER_URL:$RANCHER_URL
    - echo RANCHER_TOKEN:$RANCHER_TOKEN
    - echo RANCHER_PROJECT_ID:$RANCHER_PROJECT_ID
    - echo CONTAINER_URL:$CONTAINER_URL

variables: 
  RANCHER_URL: 
  RANCHER_URL_ONLINE: 
  RANCHER_TOKEN: 
  RANCHER_TOKEN_ONLINE: 

# 构建镜像
build-docker:
  retry: 2
  stage: build-docker
  tags:
    - k8s
  script:
    # 登陆 dockerhub
    - echo $DOCKER_REG_PASSWD | docker login $DOCKER_REG -u $DOCKER_REG_USER --password-stdin
    # 镜像地址
    - echo $IMAGE_URL
    - docker build -t $IMAGE_URL --build-arg RUNTIME_ENV=$RUNTIME_ENV .
    - docker push $IMAGE_URL
    - docker logout $DOCKER_REG
 
 

# 部署rancher
deploy_rancher_test1:
  stage: deploy-test1
  image: $CURL_IMAGE
  tags:
    - k8s
  variables:
    MSG: "正式部署到 $RANCHER_CLUSTER:$RANCHER_NAMESPACE \n部署地址：$RANCHER_URL/p/$RANCHER_PROJECT_ID/workload/deployment:$RANCHER_NAMESPACE:$RANCHER_WORKLOAD-$CI_COMMIT_REF_NAME \n"
  script:
  <<: *deploy_def
  # 手动触发rancher部署
  when: manual

  # 部署rancher online
deploy_rancher_online:
  stage: deploy-online
  image: $CURL_IMAGE
  tags:
    - k8s
  variables:
    RANCHER_URL: $RANCHER_URL_ONLINE
    RANCHER_PROJECT_ID: $RANCHER_PROJECT_ID_ONLINE
    RANCHER_TOKEN: $RANCHER_TOKEN_ONLINE
    MSG: "正式部署到 $RANCHER_CLUSTER:$RANCHER_NAMESPACE \n部署地址：$RANCHER_URL/p/$RANCHER_PROJECT_ID/workload/deployment:$RANCHER_NAMESPACE:$RANCHER_WORKLOAD-$CI_COMMIT_REF_NAME \n"
  script:
  <<: *deploy_def
  # 手动触发rancher部署
  only:
    - release
  when: manual

Summarize

After completing the above unified configuration, each project only needs to configure the unique parameters of its own project and its own build phase tasks when creating the ci process to complete the front-end build and release.

When new environments and clusters are added in the future, they only need to be maintained in the script of the same configuration, and the project only needs to add a stage and corresponding variables.

Guess you like

Origin blog.csdn.net/weixin_44786530/article/details/130424880
Recommended
The number of MaxKB GitHub Stars, an open source knowledge base question and answer system based on large language models, exceeded 5,000!
Ranking
Getting the basic concepts of ROS + catkin Profile
Spring Learning (2) --- Assembling Beans in the IoC Container
js to get the src attribute of the image regularly
STM32 is based on CubeIDE and HAL library basics entry study notes: Bluetooth WIFI STM32 connects to Alibaba Cloud
Short video learning - 3, pandas simple use of pivot_table
Print directory of project configuration log, output log
Understand the difference between vi and vim in Linux in 3 minutes
1 + x certificate Web front-end development HTML5 special exercises
mangodb save and insert the difference
7-1 Family tree processing (50 points) (binary tree solution)
Daily
More
2024-05-13(8)
2024-05-12(28)
2024-05-11(32)
2024-05-10(34)
2024-05-09(32)
2024-05-08(18)
2024-05-07(34)
2024-05-06(6)
2024-05-05(0)
2024-05-04(18)

Code World

© 2018 codetd.com