[Continuous integration] GitLab CI + Docker achieve continuous integration

GitLab CI + Docker achieve continuous integration

First, the continuous integration (Continuous Integration, CI) of the basic concept

Outline

In traditional software development, integration code is usually after all the work is done at the owner in the project were coming to an end, which tend to spend a lot of time and effort. The continuous integration is a stage on the integrated approach of software development phase, in order to more regularly to build, test and integration code.

"Continuous Integration does not eliminate the Bug, but to make them very easy to find and correct."

After continuous integration can submit a new code developers immediately build, unit testing. So that we can determine the configuration of the new code or environment and other original developer of the code or environment configuration can properly integrated with the test results.

Continuous Delivery & Continuous Deployment

Continuous Delivery (Continuous Delivery): frequently the new version of the software, delivered to the quality of the team or the user, for review. If the review by the code to enter the production stage.

Continuous deployment (Continuous Deployment): The next step is the continuous delivery, referring to the later code review, automated deployment to production.

Two, GitLab continuous integration started

Starting GitLab 8.0, GitLab CI has been integrated in GitLab, we only need to add in a project .gitlab-ci.ymlfile, and then add a Runnercan perform continuous integration. And with the upgrade GitLab, GitLab has become more and more powerful.

.gitlab-ci.yml

.gtilab-ci.ymlAnd project files are stored in the root of the repository, since by definition of GitLab CI / CD in Pipeline. In fact, nothing more than a configuration file, very easy to understand, we mainly need to understand the Pipelineconcept and how to configure a.gitlab-ci.yml

Pipeline & Stages & Jobs

一个 Pipeline 大概相对于一个构建任务，里面可以包含多个流程，如安装依赖、运行测试、编译、部署测试服务器、部署生产服务器等流程，Git 提交时会触发 Pipeline。而一个 Pipeline 中又可以包含一至多个 Stage，即用来定义安装依赖、运行测试之类的流程的。然后，一个 Stage 中又包含了一至多个 Job，Jobs 表示一个 Stage 中具体的构建工作，即某个 Stage 里面执行的工作。我们可以在 Stages 里面定义这些 Job，它们之间的关系如下图所示：

注意：

• Stages 会按 .gitlab-ci.yml 中配置的顺序执行，当前面的 Stage 执行完毕后才会继续执行后面的 Stage，如果一个 Stage 失败，那么后面的 Stage 不会执行，该构建任务失败。
• Stage 中的 Jobs 会并行执行，当这个 Stage 中所有的 Job 都执行完毕，该 Stage 才算执行成功。换而言之，只要有一个 Job 执行失败，整个 Pipeline 也就失败了。

stages:
 - build
 - test
 - deploy
 
build:
  stage: build
  script: 
    - "execute-script-for-build"
    - "do something...""
  only:
    - master
  tags:
    - ruby
    - postage

test:
  stage: test
  script:
  
......

上面配置将一次 pipeline 分成了三个阶段：build、test、deploy。下面介绍配置文件中的节点：

• stages： 定义构建的阶段；
• build、test、...：定义 jobs_name，即在 stages 中定义的 stage 阶段，一般在 stage 节点注明所属的 stage；
• script：Runner 执行的脚本或命令，该节点是必须的。
• 其他节点：stage 中还有许多其他节点，例如 only、tags 等，但并不是 required ，其具体作用可在文档中了解。

GitLab Runner

当我们理解完上面的概念后，我们还没了解最重要的东西——上面的任务由谁来构建呢？答案就是 Gitlab-runner 了！

为什么不用 GitLab CI 来运行这些构建任务呢？

一般来说，构建任务任务都会占用很多的系统资源（譬如编译代码），而 GitLab CI 又是 GitLab 的一部分，如果由 GitLab CI 来运行构建任务的话，在执行构建任务的时候， GitLab 的性能会大幅下降。

GitLab CI 最大的作用是管理各个项目的构建状态，因此，运行构建任务这种浪费资源的事情就交给 GitLab Runner 来做啦！

因为 GitLab Runner 可以安装到不同的机器上，所以在构建任务运行期间并不会影响到 GitLab 的性能。

三、持续集成的实现

接下来介绍 GitLab 对 Spring Boot 程序的持续集成，当然 GitLab 不止支持 Java 应用服务，肯定也支持其他编译语言，这里主要只是像演示一下过程，过程基本上是相通的。

搭建 GitLab 服务

这不是本次的重点，所以就简要介绍下咯。
利用 Docker 和 Docker Compose 快速搭建 GitLab 服务，docker-compose.yml 文件如下：

version: '3'
services:
    web:
      image: 'twang2218/gitlab-ce-zh:10.5'
      restart: always
      hostname: '192.168.253.139'
      environment:
        TZ: 'Asia/Shanghai'
        GITLAB_OMNIBUS_CONFIG: |
          external_url 'http://192.168.253.139:8080'
          gitlab_rails['gitlab_shell_ssh_port'] = 2222
          unicorn['port'] = 8888
          nginx['listen_port'] = 8080
      ports:
        - '8080:8080'
        - '8443:443'
        - '2222:22'
      volumes:
        - /usr/local/docker/gitlab/config:/etc/gitlab
        - /usr/local/docker/gitlab/data:/var/opt/gitlab
        - /usr/local/docker/gitlab/logs:/var/log/gitlab

启动完毕后，访问 http://ip:8080，初始化安装完成后效果如下：

设置初始化密码后刷新，就可以看见登录界面了，登录！

配置 SSH 免密登录

ssh-keygen -t rsa -P 'youname'

复制用户目录下的 .ssh 文件夹下的公钥，将其复制到 GitLab 的

构建 Docker Runner

环境准备

• 创建工作目录 /usr/lcoal/docker/runner
• 创建构建目录 /usr/local/docker/runner/enviroment
• 下载 jdk-8u152-linux-x64.tar.gz 并复制到 /usr/local/docker/runner/environment

Dockerfile

在 usr/lcoal/docker/runner/environment 目录下创建 Dockerfile

FROM gitlab/gitlab-runner:v11.0.2

RUN echo 'deb http://mirrors.aliyun.com/ubuntu/ xenial main restricted universe multiverse' > /etc/apt/sources.list && \
    echo 'deb http://mirrors.aliyun.com/ubuntu/ xenial-security main restricted universe multiverse' >> /etc/apt/sources.list && \
    echo 'deb http://mirrors.aliyun.com/ubuntu/ xenial-updates main restricted universe multiverse' >> /etc/apt/sources.list && \
    echo 'deb http://mirrors.aliyun.com/ubuntu/ xenial-backports main restricted universe multiverse' >> /etc/apt/sources.list && \
    apt-get update -y && \
    apt-get clean

RUN apt-get -y install apt-transport-https ca-certificates curl software-properties-common && \
    curl -fsSL http://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | apt-key add - && \
    add-apt-repository "deb [arch=amd64] http://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable" && \
    apt-get update -y && \
    apt-get install -y docker-ce

COPY daemon.json /etc/docker/daemon.json

WORKDIR /usr/local/bin
RUN wget https://raw.githubusercontent.com/topsale/resources/master/docker/docker-compose

RUN chmod +x docker-compose

RUN mkdir -p /usr/local/java
WORKDIR /usr/local/java
COPY jdk-8u152-linux-x64.tar.gz /usr/local/java
RUN tar -zxvf jdk-8u152-linux-x64.tar.gz && \
    rm -fr jdk-8u152-linux-x64.tar.gz

RUN mkdir -p /usr/local/maven
WORKDIR /usr/local/maven
RUN wget https://raw.githubusercontent.com/topsale/resources/master/maven/apache-maven-3.5.3-bin.tar.gz

RUN tar -zxvf apache-maven-3.5.3-bin.tar.gz && \
    rm -fr apache-maven-3.5.3-bin.tar.gz


ENV JAVA_HOME /usr/local/java/jdk1.8.0_152
ENV MAVEN_HOME /usr/local/maven/apache-maven-3.5.3
ENV PATH $PATH:$JAVA_HOME/bin:$MAVEN_HOME/bin

WORKDIR /

daemon.json

在 /usr/local/docker/runner/environment 目录下创建 daemon.json，用于配置加速器和仓库地址

{
  "registry-mirrors": [
    "https://registry.docker-cn.com"
  ],
  "insecure-registries": [
    "ip:port"
  ]
}

新建 Git 项目

新建一个 Git 项目，将其克隆到本地

注册 Runner

docker exec -it gitlab-runner gitlab-runner register

# 输入 GitLab 地址
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
http://192.168.75.146:8080/

# 输入 GitLab Token
Please enter the gitlab-ci token for this runner:
1Lxq_f1NRfCfeNbE5WRh

# 输入 Runner 的说明
Please enter the gitlab-ci description for this runner:
可以为空

# 设置 Tag，可以用于指定在构建规定的 tag 时触发 ci
Please enter the gitlab-ci tags for this runner (comma separated):
可以为空

# 选择 runner 执行器，这里我们选择的是 shell
Please enter the executor: virtualbox, docker+machine, parallels, shell, ssh, docker-ssh+machine, kubernetes, docker, docker-ssh:
shell

以上交互中，GitLab 地址和 Token 令牌可以在 GitLab 的项目的设置中找到：

当上述步骤完成后，刷新当前页面，可以看见当前页面下多出来一个 runner：

完善项目

在这里我只简单地做了个小 dome，大概这么简单：

项目中用到的 .gitlab-ci.yml、Dockerfile、docker-compose.yml 如下：
.gitlab-ci.yml

stages:
  - build
  - run
  - clean

build:
  stage: build
  script:
    - /usr/local/maven/apache-maven-3.5.3/bin/mvn clean package
    - cp target/ci-test-project-1.0.0-SNAPSHOT.jar docker
    - cd docker
    - docker build -t ci-test-project .

run:
  stage: run
  script:
    - cd docker
    - docker-compose down
    - docker-compose up -d

clean:
  stage: clean
  script:
    - docker rmi $(docker images -q -f dangling=true)

Dockerfile

FROM openjdk:8-jre

RUN mkdir /app

COPY ci-test-project-1.0.0-SNAPSHOT.jar /app/app.jar

ENTRYPOINT ["java", "-Djava.security.egd=file:/dev/./urandom", "-jar", "/app/app.jar"]

EXPOSE 8080

docker-compose.yml

version: '3.1'
services:
  itoken-config:
    restart: always
    image: ci-test-project
    ports:
      - 8080:8080

提交项目

完成 demo 的之后就将项目提交到 GitLab 上，push 完成之后，点击项目的 CI/CD 可以看见一下页面

再点击进入就可以看见具体的作业了，自行体会

当项目出现所有 jobs 全部通过时，也就是持续集成初步完善了，这时我们可以打开浏览器访问 ip:8080 可以看见浏览器上显示

Hello GitLab

此时我们可以尝试修改项目中的代码再提交，比如简单地将 retuen "Hello GitLab" 改成 return "Hello World" 再提交一遍代码，等 jobs 通过后，再刷新浏览器，我们可以看见

Hello World

至此，这个小 Demo 也就完成了。

四、参考

• https://docs.gitlab.com/ee/ci/yaml/
• https://www.cnblogs.com/newP/p/5735366.html
• https://www.funtl.com/zh/spring-cloud-itoken-ci/
• https://www.mindtheproduct.com/2016/02/what-the-hell-are-ci-cd-and-devops-a-cheatsheet-for-the-rest-of-us/