GitLab and GitLab Runner installation (RPM and Docker methods), first experience with CI/CD

background

GitLabIt is a powerful version control system and collaboration platform. Record the GitLabinstallation and use records of in actual work.

When you first use GitLab, it is installed CentOS7directly as a package on , and is only used as a code hosting tool. Version: .rpm14.10.4

GitLabIn the subsequent pre - research CI/CDand assembly line, Dockerthe installation method is adopted, version: 16.2.3-jh; introduced GitLab Runner, version: 16.2.0.

System environment

[root@gitlab1 opt]# uname -a
Linux gitlab1 3.10.0-1127.el7.x86_64 #1 SMP Tue Mar 31 23:36:51 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[root@gitlab1 opt]# cat /proc/version
Linux version 3.10.0-1127.el7.x86_64 ([email protected]) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Tue Mar 31 23:36:51 UTC 2020
[root@gitlab1 opt]# cat /etc/redhat-release 
CentOS Linux release 7.8.2003 (Core)
[root@gitlab2 ~]# docker -v
Docker version 20.10.18, build b40c2f6

Install GitLab via RPM

Download address, Tsinghua mirror: https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/

[root@gitlab1 local]# wget https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/gitlab-ce-14.10.4-ce.0.el7.x86_64.rpm

错误: 无法验证 mirrors.tuna.tsinghua.edu.cn 的由 “/C=US/O=Let's Encrypt/CN=R3” 颁发的证书:
  颁发的证书已经过期。
解决：将https修改为http

[root@gitlab1 local]# rpm -ivh gitlab-ce-14.10.4-ce.0.el7.x86_64.rpm 
警告：gitlab-ce-14.10.4-ce.0.el7.x86_64.rpm: 头V4 RSA/SHA1 Signature, 密钥 ID f27eab47: NOKEY
错误：依赖检测失败：
        policycoreutils-python 被 gitlab-ce-14.10.4-ce.0.el7.x86_64 需要

解决：yum install -y curl policycoreutils-python openssh-server

[root@gitlab1 local]# rpm -ivh gitlab-ce-14.10.4-ce.0.el7.x86_64.rpm 
警告：gitlab-ce-14.10.4-ce.0.el7.x86_64.rpm: 头V4 RSA/SHA1 Signature, 密钥 ID f27eab47: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:gitlab-ce-14.10.4-ce.0.el7       ################################# [100%]
It looks like GitLab has not been configured yet; skipping the upgrade script.

       *.                  *.
      ***                 ***
     *****               *****
    .******             *******
    ********            ********
   ,,,,,,,,,***********,,,,,,,,,
  ,,,,,,,,,,,*********,,,,,,,,,,,
  .,,,,,,,,,,,*******,,,,,,,,,,,,
      ,,,,,,,,,*****,,,,,,,,,.
         ,,,,,,,****,,,,,,
            .,,,***,,,,
                ,*,.
  

     _______ __  __          __
    / ____(_) /_/ /   ____ _/ /_
   / / __/ / __/ /   / __ `/ __ \
  / /_/ / / /_/ /___/ /_/ / /_/ /
  \____/_/\__/_____/\__,_/_.___/
  

Thank you for installing GitLab!
GitLab was unable to detect a valid hostname for your instance.
Please configure a URL for your GitLab instance by setting `external_url`
configuration in /etc/gitlab/gitlab.rb file.
Then, you can start your GitLab instance by running the following command:
  sudo gitlab-ctl reconfigure

For a comprehensive list of configuration options please see the Omnibus GitLab readme
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md

Help us improve the installation experience, let us know how we did with a 1 minute survey:
https://gitlab.fra1.qualtrics.com/jfe/form/SV_6kVqZANThUQ1bZb?installation=omnibus&release=14-10

# 编辑配置，设置访问地址
[root@gitlab1 local]# vi /etc/gitlab/gitlab.rb
external_url 'http://gitlab1'

# 重新配置生效
[root@gitlab1 local]# gitlab-ctl reconfigure

Running handlers:
Running handlers complete
Chef Infra Client finished, 606/1618 resources updated in 05 minutes 34 seconds

Warnings:
Rehashing of trusted certificates present in `/etc/gitlab/trusted-certs` failed. If on a FIPS-enabled machine, ensure `c_rehash` binary is available in $PATH.

Notes:
Default admin account has been configured with following details:
Username: root
Password: You didn't opt-in to print initial root password to STDOUT.
Password stored to /etc/gitlab/initial_root_password. This file will be cleaned up in first reconfigure run after 24 hours.

NOTE: Because these credentials might be present in your log files in plain text, it is highly recommended to reset the password following https://docs.gitlab.com/ee/security/reset_user_password.html#reset-your-root-password.

gitlab Reconfigured!

Note:

1. How to check version information? To view the version in the GitLab backend: add help after the deployment domain name/IP.
2. If you modify the configuration information such as IP or port, be sure to reload the configuration after each change: gitlab-ctl reconfigure.

[root@gitlab1 local]# cat /etc/gitlab/initial_root_password | grep Password

Use a browser to access the configured address and enter your username and password.

GitLabCommon commands

1. gitlab-ctl start: Start gitlab,
2. gitlab-ctl stop: Stop gitlab.
3. gitlab-ctl status: View gitlab status
4. gitlab-ctl restart: restart the service

As a code warehouse, GitLabits use includes the following aspects:

• Create group
• Create project
• Add SSH key
• clone
• User registration
• User review
• Join group

In this way, if you want to use CI/CDthe function, you need to install it GitLab Runner(but it is not recommended to use this direct installation method. In actual production, more people use Dockerand Kubernetes). It is divided into two steps: installation and registration. Download address: https ://mirrors.tuna.tsinghua.edu.cn/gitlab-runner/yum/el7/

Note:

1. GitLab Runner registration is divided into two steps: obtain runner token and register
2. GitLab Runner is divided into shared type, group type, and project type, which are located in different locations.
3. There are two ways to register: interactive and non-interactive.

Docker installation using GitLab

Open a new virtual machine and use Dockerthe method to install and run GitLaband GitLab Runner.

[root@gitlab2 ~]# cd /opt/
[root@gitlab2 opt]# mkdir gitlab
[root@gitlab2 opt]# export GITLAB_HOME=/opt/gitlab

According to the official documentation, GitLab Runnerthe command to install and run is as follows:

sudo docker run --detach \
  --hostname gitlab.example.com \
  --publish 443:443 --publish 80:80 --publish 22:22 \
  --name gitlab \
  --restart always \
  --volume $GITLAB_HOME/config:/etc/gitlab \
  --volume $GITLAB_HOME/logs:/var/log/gitlab \
  --volume $GITLAB_HOME/data:/var/opt/gitlab \
  --shm-size 256m \
  registry.gitlab.cn/omnibus/gitlab-jh:latest

During actual execution, the parameters were simplified, the host name was set, and the mapped port number was modified.

docker run -d \
  -h gitlab2 \
  -p 80:80 -p 222:22 \
  --name gitlab \
  --restart always \
  -v $GITLAB_HOME/config:/etc/gitlab \
  -v $GITLAB_HOME/logs:/var/log/gitlab \
  -v $GITLAB_HOME/data:/var/opt/gitlab \
  --shm-size 256m \
  registry.gitlab.cn/omnibus/gitlab-jh:latest

Note: Change the host's port 22 to 222, otherwise an error will be reported:Error starting userland proxy: listen tcp4 0.0.0.0:22: bind: address already in use.

Visit JiFox GitLab URLand log in using the username rootand password from the following command:

[root@gitlab2 ~]# docker exec -it gitlab grep 'Password:' /etc/gitlab/initial_root_password
Password: zIrC8HPFfuxVmGSyx27nRbgTRwLaoiIhu+a2edEySMw=

DockerTo manage GitLabthe service through .

docker start gitlab
docker stop gitlab
docker restart gitlab
docker rm gitlab

Install and register gitlab-runner in Docker mode

Install GitLab Runner

Based on GitLabthe version of , the version of is specified GitLab Runner.

docker run -d --name gitlab-runner --restart always \
  -v /opt/gitlab-runner/config:/etc/gitlab-runner \
  -v /var/run/docker.sock:/var/run/docker.sock \
  gitlab/gitlab-runner:v16.2.0

# 查看安装的GitLab Runner的版本信息
[root@gitlab2 ~]# docker exec -it gitlab-runner gitlab-runner --version
Version:      16.2.0
Git revision: 782e15da
Git branch:   16-2-stable
GO version:   go1.20.5
Built:        2023-07-21T22:52:42+0000
OS/Arch:      linux/amd64

Register GitLab Runner

Next, create the group, create the project, clone it, and register one for the project Runner.

Create new group ——> Create new project ——> git clone http://gitlab2/iot/test.git

# 注册为`Shell`类型的执行器。
docker run --rm -v /opt/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner:v16.2.0 \
 register \
  --non-interactive \
  --executor "shell" \
  --url "http://192.168.44.149" \
  --registration-token "GR1348941NNVnhpcsLqgBbL-1JxPT" \
  --description "shell-runner" \
  --tag-list "shell,test" \
  --run-untagged="true" \
  --locked="false" \
  --access-level="not_protected"

Problems encountered during registration:

1. ERROR: Registering runner… error runner=GR1348941NNVnhpcs status=only http or https scheme supported

The reason is that I --urlwrote the value as the host name, which must be written in HTTPthe form of an address link.

2. ERROR: Registering runner… failed runner=GR1348941NNVnhpcs status=couldn’t execute POST against http://gitlab2/api/v4/runners: Post “http://gitlab2/api/v4/runners”: dial tcp: lookup gitlab2 on 192.168.44.2:53: no such host

The reason is that I --urlwrote the value as the host name, and the host name could not be successfully registered in the container. Then I changed it to IPthe address, and the registration was successful. The following is the effect of successful registration.

[root@gitlab2 ~]# docker run --rm -v /opt/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner:v16.2.0 \
>  register \
>   --non-interactive \
>   --executor "shell" \
>   --url "http://192.168.44.149" \
>   --registration-token "GR1348941NNVnhpcsLqgBbL-1JxPT" \
>   --description "shell-runner" \
>   --tag-list "shell,test" \
>   --run-untagged="true" \
>   --locked="false" \
>   --access-level="not_protected"
Runtime platform                                    arch=amd64 os=linux pid=7 revision=782e15da version=16.2.0
Running in system-mode.                            
                                                   
WARNING: Support for registration tokens and runner parameters in the 'register' command has been deprecated in GitLab Runner 15.6 and will be replaced with support for authentication tokens. For more information, see https://gitlab.com/gitlab-org/gitlab/-/issues/380872 
Registering runner... succeeded                     runner=GR1348941NNVnhpcs
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
 
Configuration (with the authentication token) was saved in "/etc/gitlab-runner/config.toml" 

CI/CD

In order to experience the following GitLab Runnerfunctions CI/CD, let's simulate the actual pipeline process with a configuration of back-end service construction and deployment, and create a new file in the root directory of the created project: (After submission, the project .gitlab-ci.ymlwill automatically start the execution of the pipeline).

stages:
  - build
  - deploy
 
build:
  stage: build
  tags:
    - shell
  only:
    - master
  script:
    - echo "mvn clean"
    - echo "mvn package"

deploy:
  stage: deploy
  tags:
    - test
  only:
    - master
  script:
    - echo "java -jar x.jar"

CI/CDProblems encountered during the process:

1. After the above configuration file was added and submitted for the first time, the pipeline was not triggered. .

GitLabIt turns out that the main branch in the new version I used masterwas changed from to main, .gitlab-ci.ymljust modify the branch name in .

2. The pipeline failed, Job log: fatal: unable to access 'http://gitlab2/iot/test.git/': Could not resolve host: gitlab2

It feels like Dockerthe container cannot obtain the host name of the host. When entering gitlab-runnerthe container where the host is located, pingthe host ipis accessible, but the host name is blocked. The simple way is to config.tomladd it in the file clone_url = "http://192.168.44.149"and IPpull the code in the way.

# 没有自带ping命令，自行安装
root@249b6b18ffa8:/# apt update
root@249b6b18ffa8:/# apt install -y iputils-ping

# 修改配置，增加clone_url配置，跟url并列
[root@gitlab2 ~]# vi /opt/gitlab-runner/config/config.toml
clone_url = "http://192.168.44.149"

# 重启gitlab-runner所在容器
[root@gitlab2 ~]# docker restart gitlab-runner
gitlab-runner

After changing to the above configuration, manually start one Pipelineand the execution is successful.

# 进入容器
[root@gitlab2 ~]# docker exec -it gitlab-runner /bin/bash

# 列出所有的runner
root@249b6b18ffa8:/# gitlab-runner list
Runtime platform                                    arch=amd64 os=linux pid=27 revision=782e15da version=16.2.0
Listing configured runners                          ConfigFile=/etc/gitlab-runner/config.toml
shell-runner                                        Executor=shell Token=XQEENnyfzyEVs8-2iDB9 URL=http://192.168.44.149

# 查看runner的状态
root@249b6b18ffa8:/# gitlab-runner verify
Runtime platform                                    arch=amd64 os=linux pid=37 revision=782e15da version=16.2.0
Running in system-mode.                            
                                                   
Verifying runner... is alive                        runner=XQEENnyf

Note:

1. CI Lint can be used to verify the syntax of .gitlab-ci.yml;
2. A pipeline contains multiple jobs, and each job contains at least one script.
3. After modifying the server's IP, the Docker container cannot be accessed from outside the server

# 服务器IP更改之后，linux数据包转发配置net.ipv4.ip_forward会变为0，即关闭状态。
[root@gitlab2 ~]# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 0

# 重启docker，docker会默认将net.ipv4.ip_forward值临时改为1（再次新增或更改IP之后会失效）
[root@gitlab2 ~]# systemctl restart docker

Summary

The above is the installation and use GitLabprocess, as well as the first experience of CI/CD. GitLabIt is a feature-rich, easy-to-use, safe and reliable collaboration platform that can help teams better manage and develop projects.

1. Integrated workflow: GitLabProvides an integrated workflow, including code hosting, issue tracking, continuous integration and deployment and other functions. This allows team members to collaborate on developing and managing projects on the same platform.
2. Code hosting: GitLabProvides powerful code hosting functions and supports Git distributed version control system. Team members can easily share and manage code for version control and collaborative development.
3. Issue Tracking: GitLabProvides issue tracking functionality to help team members record and resolve issues in the project. With issue tracking, team members can better collaborate and track project progress.
4. Continuous integration and deployment: GitLabProvides powerful continuous integration and deployment capabilities to automate building, testing and deploying code. This can greatly improve development efficiency and code quality.
5. Security: GitLabProvides powerful security features, including access control, permission management, and code review. This helps teams keep code and sensitive information secure.

Reference

• https://docs.gitlab.cn/runner/register/

---

If you have any questions or any bugs are found, please feel free to contact me.

Your comments and suggestions are welcome!