Gitlab+Rancher implements Dockerized DevOps Part II: CI builds Docker images for infrastructure/applications

project preparation

Project repository enabled CI and mirror repository
Project configuration Settings> General> Permissions> PipelinesenabledPipelines、Container registry

The project defines the CI build script
The file under the project root .gitlab-ci.ymlcan define the CI build process

• Ci Server will trigger the build and dispatch the build job to Ci Runner on every push/merge
• Builds can last for multiple stages, and each stage can handle multiple build jobs
• Different build jobs under the same stage can be built concurrently (if there are enough Runners)
• Build results can be dynamically displayed by embedding badges in the README document

The goal of this article

1. Build the nginx-phpfpm image as an infrastructure image to support the operation of the php project
2. Load the laravel project into the nginx-phpfpm image and deploy it online as an application image

CI build infrastructure mirroring

Full code:https://github.com/maifusha/docker-nginx-phpfpm

Dockerfile design

• Packages such as php:7.0-fpmimage encapsulation composer、php拓展、node前端设施、nginx、supervisorto provide a complete php project operating environment
• Custom php、nginx、supervisorconfiguration overwrites original configuration, through supervisormaintenance nginx/phpfpmservice

FROM php:7.0-fpm

LABEL maintainer="maifusha <[email protected]>"

COPY conf/sources.list /etc/apt/sources.list

RUN curl http://nginx.org/keys/nginx_signing.key > /tmp/nginx_signing.key \
    && apt-key add /tmp/nginx_signing.key && rm -f /tmp/nginx_signing.key

RUN apt-get update && apt-get install -y --no-install-recommends \
        vim \
        git \
        sudo \
        unzip \
        dnsutils \
        nginx \
        supervisor \
        openssh-client \
        build-essential \
        libpq-dev \
        python-dev \
        libmcrypt-dev \
        libpng12-dev \
        libfreetype6-dev \
        libjpeg62-turbo-dev \
    && curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash - \
    && apt-get install -y --no-install-recommends nodejs \
    && docker-php-ext-install -j$(nproc) \
        pdo_mysql \
        pdo_pgsql \
        mcrypt \
        bcmath \
        zip \
    && docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ \
    && docker-php-ext-install -j$(nproc) gd \
    && rm -rf /var/lib/apt/lists/*

RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer \
    && composer config -g repo.packagist composer https://packagist.phpcomposer.com \
    && composer global require "hirak/prestissimo:^0.3"

RUN npm install -g cnpm gulp bower yarn --registry=https://registry.npm.taobao.org \
    && yarn config set registry https://registry.npm.taobao.org

COPY conf/etc /etc
COPY conf/php-fpm /usr/local/etc

ENV PATH=$PATH:~/.composer/vendor/bin

RUN ln -sf /dev/stdout /var/log/nginx/access.log && \
    ln -sf /dev/stderr /var/log/nginx/error.log

WORKDIR /var/www/html

EXPOSE 80 443

COPY docker-entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

CMD ["supervisord", "-c", "/etc/supervisor/supervisord.conf"]

.gitlab-ci.yml build script design

• Any branch changes, trigger build_image to compile and push the branch image to the repository
• The master branch changes, triggers release_image compilation and pushes the master image to the repository

stages:
  - build
  - release

variables:
  BRANCH_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME
  RELEASE_IMAGE: $CI_REGISTRY_IMAGE:latest

before_script:
  - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY

build_image:
  stage: build
  script:
    - docker build --pull -t $BRANCH_IMAGE .
    - docker push $BRANCH_IMAGE

release_image:
  stage: release
  script:
    - docker pull $BRANCH_IMAGE
    - docker tag $BRANCH_IMAGE $RELEASE_IMAGE
    - docker push $RELEASE_IMAGE
  only:
    - master

CI builds application images

Full code:https://github.com/maifusha/laravel-ci-demo

convention

• The project contains test/staging/productionthree main environments

Dockerfile design

• nginx-phpfpmPackaging the laravel project based on the previous image
• Download composer、nodedependencies

FROM git.yoursite.com:5005/docker/nginx-phpfpm:latest

LABEL maintainer="maifusha <[email protected]>"

COPY supervisor.ini /etc/supervisor/conf.d/supervisor.ini

COPY . /var/www/html

RUN composer install --no-dev --prefer-dist --optimize-autoloader --quiet \
    && cnpm install && npm run production

RUN chown -R www-data:www-data /var/www/html

EXPOSE 80
VOLUME /var/www/html/storage

HEALTHCHECK --interval=5s --timeout=3s \
  CMD curl --fail http://localhost/ || exit 1

.gitlab-ci.yml build script design

• Arbitrary branch changes, triggering the phpcs、phpmd、phpunitcheck job in the verify phase
• Test/staging branch changes and Triggers master trigger respective image build pushes and environment deployments
• deploy_production builds need to be triggered manually by Triggers master
  • Create Triggers token: Settings> CI/CD> TriggersAdd trigger
  • TriggersApi triggers:curl -X POST "git.yoursite.com:5005/projects/项目id/trigger/pipeline?token=Triggers令牌&ref=master_or_tag"
  • Each trigger will create a mirror history of the git commit version number
• Application deployment is temporarily designed to be swarm deployment instead of k8s. It needs to be prepared in advance to get through Job Runner's ssh access to the test/staging/production environment server
  • Generate ssh key pair: ssh-keygen -t rsa -C "mailbox" -b 4096
  • Write the local public key to the authorized_keys file of the remote server: ssh-copy-id -i ~/.ssh/id_rsa.pub root@server ip
  • Add the private key to Project->CI/CD->Secret Variables as the id_rsa security variable (when building, the security variable will be encrypted and transmitted to the JobRunner)

stages:
  - verify
  - build
  - deploy


phpcs:
  image: git.yoursite.com:5005/docker/php-fpm:latest
  stage: verify
  except:
    - triggers
  script:
    - composer install --dev --quiet --prefer-dist --optimize-autoloader
    - ./vendor/bin/phpcs --standard=phpcs.xml

phpmd:
  image: git.yoursite.com:5005/docker/php-fpm:latest
  stage: verify
  except:
    - triggers
  script:
    - composer install --dev --quiet --prefer-dist --optimize-autoloader
    - ./vendor/bin/phpmd app,tests text resources/rulesets/cleancode.xml,resources/rulesets/codesize.xml,resources/rulesets/controversial.xml,resources/rulesets/design.xml,resources/rulesets/naming.xml,resources/rulesets/unusedcode.xml

phpunit:
  image: git.yoursite.com:5005/docker/php-fpm:latest
  stage: verify
  except:
    - triggers
  script:
    - composer install --dev --quiet --prefer-dist --optimize-autoloader
    - ./vendor/bin/phpunit --configuration phpunit.xml --coverage-text --colors=never


build_test:
  image: git.yoursite.com:5005/docker/docker:latest
  stage: build
  only:
    - test
  script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
    - docker build --pull -t $CI_REGISTRY_IMAGE:test .
    - docker push $CI_REGISTRY_IMAGE:test

build_staging:
  image: git.yoursite.com:5005/docker/docker:latest
  stage: build
  only:
    - staging
  script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
    - docker build --pull -t $CI_REGISTRY_IMAGE:staging .
    - docker push $CI_REGISTRY_IMAGE:staging

build_production:
  image: git.yoursite.com:5005/docker/docker:latest
  stage: build
  only:
    - triggers
  script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
    - docker build --pull -t $CI_REGISTRY_IMAGE:$CI_BUILD_REF .
    - docker push $CI_REGISTRY_IMAGE:$CI_BUILD_REF
    - docker tag $CI_REGISTRY_IMAGE:$CI_BUILD_REF $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:latest


deploy_test:
  image: git.yoursite.com:5005/docker/docker:latest
  stage: deploy
  environment:
    name: test
    url: http://test.yoursite.com
  only:
    - test
  when: manual
  before_script:
    - eval $(ssh-agent -s) && echo -n "$id_rsa"|ssh-add -
  script:
    - ssh -o "StrictHostKeyChecking no" test机账号@test机ip
      "
      docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY};
      docker service update \
            --update-parallelism 2 \
            --update-delay 1ms \
            --with-registry-auth \
            --image $CI_REGISTRY_IMAGE:test \
            服务名称 \
      "

deploy_staging:
  image: git.yoursite.com:5005/docker/docker:latest
  stage: deploy
  environment:
    name: staging
    url: http://staging.yoursite.com
  only:
    - staging
  when: manual
  before_script:
    - eval $(ssh-agent -s) && echo -n "$id_rsa"|ssh-add -
  script:
    - ssh -o "StrictHostKeyChecking no" staging机账号@staging机ip
      "
      docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY};
      docker service update \
            --update-parallelism 2 \
            --update-delay 1ms \
            --with-registry-auth \
            --image $CI_REGISTRY_IMAGE:staging \
            服务名称 \
      "

deploy_production:
  image: git.yoursite.com:5005/docker/docker:latest
  stage: deploy
  environment:
    name: production
    url: http://www.yoursite.com
  only:
    - triggers
  before_script:
    - eval $(ssh-agent -s) && echo -n "$id_rsa"|ssh-add -
  script:
    - ssh -o "StrictHostKeyChecking no" production机账号@production机ip
      "
      docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY};
      docker service update \
            --update-parallelism 2 \
            --update-delay 1ms \
            --with-registry-auth \
            --image $CI_REGISTRY_IMAGE:latest \
            服务名称 \
      "

{{o.name}}

{{m.name}}