"Internet Architecture" Software Architecture-Jenkins Construction and Maven gitlab Automated Deployment Process (Part I) (5)

When I haven’t used jenkins a few years ago, I need to use eclipse to make a war package every time, and then upload it to the server carefully, change the name of the server’s original war package, mv to the bak directory, stop the service, and delete the original For some webapps projects, put the newly uploaded war package into the directory mentioned by the tomcat webapp and start the project. Every time you change the name of an html tag, you need to upload it again, which is such a tedious operation every time. In fact, small companies can still tolerate it. If it is a relatively large project, the operation and maintenance personnel who continue to stay on this script will be exhausted, because it is possible to deploy dozens of projects at a time. Source code: https://github.com/limingios/netFuture/tree/master/jenkins/ysource/idig8

[External link image transfer failed. The source site may have an anti-leech link mechanism. It is recommended to save the image and upload it directly (img-OdZy1Wau-1596675955350)(https://upload-images.jianshu.io/upload_images/11223715-8f67051b451e50bf.png ?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)]

jenkins

• history

Hudson was developed by Sun in the summer of 2004,
and the first version was released as open source in February 2005.
CruiseControl was the big brother in the CI industry when Hudson was released, but soon, Hudson surpassed CruiseControl in about 2007. At the JavaOne conference in May 2008, Hudson won the Duke's Choice award in the category of development solutions. Since then, the younger brother turned to become the elder brother, and Hudson became synonymous with CI.
In September 2010, Tortoise Shell Company secretly turned Hudson®™ into a registered trademark. In November 2010, the core developers of the Hudson community found out and were angry. The two sides had a not very friendly meeting, and the talk fell apart unexpectedly. After Christmas
, the first snow in 2011 came later than before. Several bald uncles made a difficult decision in McDonald's luxury private room:
mv -f hudson jenkins
Hudson and Jenkins both have codes;
Hudson has the support of Oracle and Sonatype's corporate and Hudson's registered trademark
Jenkins owns most of the core developers, communities, and subsequent commits.

[External link image transfer failed. The source site may have an anti-leech link mechanism. It is recommended to save the image and upload it directly (img-qR6s4Hsg-1596675955362)(https://upload-images.jianshu.io/upload_images/11223715-94ce4a5767771e01.png ?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)]

Jenkins git, maven gitlab, tomcat build continuous integration environment process

1> The developer pushes the new version to the git server (Gitlab).
2> Gitlab then triggers the jenkins master node for a build. (Through web hook or timing detection)
3> The jenkins master node assigns this build task to one of several registered slave nodes, and this slave node builds according to a pre-set script. This script can do many things, such as compiling, testing, generating test reports and so on. These tasks that need to be completed manually can be handed over to jenkins.
4> We need to compile in the build, here we use the distributed compiler distcc to speed up the compilation.
The working principle of jenkins is to first copy the source code from gitlab to the local, and then build according to the set script. We can see that the key to the entire system is the build script, which tells jenkins what tasks need to be performed in an integration.

This time, I will combine a relatively simple Maven aggregation project, combined with the previous maven private server, git private server and the jenkins to be built to complete the automated construction.

• gitlab, nexus private server, jenkins, tomcat construction

Generate 4 virtual machines from the source code and prepare for work. Vagrant has installed the corresponding docker.
Installing nexus with docker is to avoid complicated operations such as environment variables and user authorization. For how to install vagrant for different systems, please refer to
mac install vgarant: https://idig8.com/2018/07/29/docker-zhongji-07/
window install vgarant https://idig8.com/2018/07/ 29/docker-zhongji-08/

System type	IP address	Node role	CPU	Memory	Hostname
Centos7	192.168.66.100	gitlab	2	2G	gitlab
Centos7	192.168.66.101	jenkins	2	2G	jenkins
Centos7	192.168.66.102	nexus	2	2G	nexus
Centos7	192.168.66.103	tomcat	2	2G	tomcat

(1). Virtual machine vagrant tells the installation steps

vagrant up

(2). The machine window/mac enables remote login under the root user

su -
# 密码
vagrant
#设置 PasswordAuthentication yes
vi /etc/ssh/sshd_config
sudo systemctl restart sshd

Four machines execute commands in common

su -
#密码
vagrant
service docker restart
yum install -y lrzsz

Note: The following machines except for jenkins do not use docker, all others use docker fully installed

66.100 gitlab installation

Administrator: root
password: 123456789qwe

mkdir gitlab
cd gitlab
vi start.sh
mkdir postgresql redis gitlab 
chown -R 200 postgresql
chown -R 200 redis
chown -R 200 gitlab

• Scripting

cur_dir=`pwd`
docker stop gitlab-postgresql
docker rm gitlab-postgresql
docker stop gitlab-redis
docker rm gitlab-redis
docker stop gitlab
docker rm gitlab
docker run --name gitlab-postgresql -d \
    --env 'DB_NAME=gitlabhq_production' \
    --env 'DB_USER=gitlab' --env 'DB_PASS=password' \
    --env 'DB_EXTENSION=pg_trgm' \
    --volume ${cur_dir}/postgresql:/var/lib/postgresql \
    sameersbn/postgresql:10
docker run --name gitlab-redis -d \
    --volume ${cur_dir}/redis:/var/lib/redis \
    sameersbn/redis:4.0.9-1
docker run --name gitlab -d \
    --link gitlab-postgresql:postgresql --link gitlab-redis:redisio \
    --publish 10022:22 --publish 10080:80 \
    --env 'GITLAB_PORT=10080' --env 'GITLAB_SSH_PORT=10022' \
    --env 'GITLAB_ROOT_PASSWORD=123456789qwe' \
    --env 'GITLAB_SECRETS_DB_KEY_BASE=long-and-random-alpha-numeric-string' \
    --env 'GITLAB_SECRETS_SECRET_KEY_BASE=long-and-random-alpha-numeric-string' \
    --env 'GITLAB_SECRETS_OTP_KEY_BASE=long-and-random-alpha-numeric-string' \
    --volume ${cur_dir}/gitlab:/home/git/data \
    sameersbn/gitlab[root@gitlab ~]# 

• Start gitlab

The point is that don't use rz to upload the start.sh script I provided. Sh start.sh will report an error. sh start.sh needs to be executed twice, don’t ask me why, I don’t know, the first time the container is executed, it hangs by itself, just start it again.

sh start.sh
sh start.sh

66.101 jenkins installation

This does not use a container, relatively speaking, the installation is more complicated. This machine not only needs to install jenkins, but also needs to install maven, git, jdk1.8

• In order to let you old iron install jdk1.8 git and maven beauty, I wrote a script

# @Author: liming
# @Date:   2018-11-26 23:14:59
# @Last Modified by:   liming
# @Last Modified time: 23:15:05
# @urlblog idig8.com
# 个人公众号  编程坑太多

#!/bin/bash
SOFT_PATH=/opt/soft

if [ ! -d $SOFT_PATH ];then
mkdir $SOFT_PATH
else
echo "文件夹已经存在"
fi

yum install -y wget 
#install jdk1.8
cd $SOFT_PATH
wget wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u141-b15/336fa29ff2bb4ef291e347e091f7f4a7/jdk-8u141-linux-x64.tar.gz"
tar -zxvf jdk* -C $SOFT_PATH
cd jdk*
JAVA_HOME=`pwd` 

#install maven3.2.3
cd $SOFT_PATH
wget https://archive.apache.org/dist/maven/maven-3/3.2.3/binaries/apache-maven-3.2.3-bin.tar.gz
tar -zxvf apache-maven-3.2.3-bin.tar.gz -C $SOFT_PATH
mv apache-maven-3.2.3 maven-3.2.3
cd maven*
MAVEN_HOME=`pwd`

#install git 2.8.0
cd $SOFT_PATH
yum -y install zlib-devel openssl-devel cpio expat-devel gettext-devel curl-devel perl-ExtUtils-CBuilder perl-ExtUtils- MakeMaker
wget https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.8.0.tar.gz
tar -zxvf git-2.8.0.tar.gz -C $SOFT_PATH
cd git*
./configure
make install
ln -s /usr/local/bin/git /usr/bin/git

#追加环境变量
echo "export JAVA_HOME=${JAVA_HOME}" >> /etc/profile
echo "export PATH=$""JAVA_HOME/bin:$""PATH" >> /etc/profile
echo "export MAVEN_HOME=${MAVEN_HOME}" >> /etc/profile
echo "export PATH=$""MAVEN_HOME/bin:$""PATH" >> /etc/profile
source /etc/profile
#输出信息
echo "-----source update-----"
echo "java version"
java -version
echo "maven version"
mvn -v
echo "-----path-----"
echo "JAVA_HOME:"$JAVA_HOME
echo "MAVEN_HOME:"$MAVEN_HOME
source /etc/profile

• View environment variables

If the environment variable does not take effect, please execute: source /etc/profile

source /etc/profile
git --version
java -version
mvn -v   

• Download jenkins

Official website: https://jenkins.io/

Download jenkins

wget https://mirrors.tuna.tsinghua.edu.cn/jenkins/war-stable/2.138.3/jenkins.war
# 先这样启动目的是直接可以看到秘钥因为第一次安装需要秘钥
java -jar jenkins.war --ajp13Port=-1 --httpPort=8888
#这是后台启动
nohup java -jar jenkins.war --ajp13Port=-1 --httpPort=8888 &

• All pipline plugins are installed

• I also used docker to install the jenkins plug-in. The old installation failed with various errors. This time I installed all successfully without the docker plugin

66.102 nexus installation

Administrator: admin
Password: admin123

mkdir nexus
cd nexus
vi start.sh
mkdir data && chown -R 200 data

• start.sh

#!/bin/bash
cur_dir=`pwd`
docker stop nexus
docker rm nexus
docker run -d -p 8081:8081 --name nexus -v  ${cur_dir}/data:/sonatype-work sonatype/nexus

• Startup script

sh start.sh

66.103 tomcat installation

What else does tomcat talk about? Don't understand? It's done, brother!

mkdir tomcat
cd tomcat
vi start.sh

• script

#!/bin/bash
cur_dir=`pwd`
docker stop tomcat
docker rm tomcat
docker run --name tomcat -p 8080:8080 -v ${cur_dir}/tomcat-persistence:/bitnami bitnami/tomcat:latest

• Startup script

sh start.sh

• put the war package here

All the above programs have been installed, and the automated deployment process has begun

application	IP address	service	port	Install the app	Installation method
gitlab	192.168.66.100	gitlab	10080	gitlab	docker
jenkins	192.168.66.101	jenkins	8888	jdk8 maven3.2 git2.8	shell
nexus	192.168.66.102	nexus	8081	nexus	docker
tomcat	192.168.66.103	tomcat	8080	tomcat	docker

PS: The next step is to associate these 4 servers, submit the local code to gitlab, and then perform automated deployment in the form of jenkins operation commands.