1. Background
In actual development, we often have to test while developing. Of course, the test mentioned here is not the unit test of the programmer's own code, but the tester's test after the code is submitted by the same group of programmers;
Or after the front and back ends are separated, the interface is often modified and then redeployed;
These situations will involve frequent package deployment;
General steps for manual packaging:
1. Submit the code
2. Ask the same group if there is any code to submit
3. Pull the code and package it (war package, or jar package)
4. Upload to Linux server
5. Check whether the current program is running
6. Close the current program
7. Start the new jar package
8. Observe the log to see if the startup is successful
9. If a colleague says that he still has code that has not been submitted... Repeat steps 1 to 8 again! ! ! ! ! (one morning gone)
Then, there is a tool that can automatically package and deploy the code after submitting it to git. The answer is yes: Jenkins
Of course, in addition to Jenkins, there are other tools that can automate deployment, such as Hudson, etc.
It's just that Jenkins is relatively more widely used
1. jenkins
1.1 what is jenkins
Jenkins is the software for continuous development and deployment
1.2. Implementation principle of Jenkins automated deployment
yum install -y curl policycoreutils-python openssh-server perl postfix
systemctl enable --now sshd postfixrpm -ivh gitlab-ce-12.3.0-ce.0.el7.x86_64.rpm
vim /etc/gitlab/gitlab.rb
external_url 'http://192.168.50.25:82'
nginx['listen_port'] = 82gitlab-ctl reconfigure
gitlab-ctl restart
Browser access: http://192.168.50.25:82, if 502 is displayed, wait for a while and then refresh
1. Set the new password of the administrator root user, pay attention to the format requirements, set it here as root@123
2. Use the administrator root user to log in (root/root@123)
3. After logging in, click the user option drop-down at the top right of the page to select [Settings], click [Preferences] on the left menu, select [Simplified Chinese] in [Language] of [Localization], click the [Save changes] button, and then refresh the page to use Chinese display.
//Gitlab add group, create user, create project
#Create Group
Click the [GitLab] icon on the upper left of the page, click [New Group]
[Group Name], [Group URL], [Group Description] are all set Become devops_group
[Visibility level] set to private, and
keep the default value for others, click [Create group]
#Create a project
Click [New Project]
[Project URL] Group option drop-down select devops_group
[Project Name] set to web_demo
[Project Description] set to web_demo
[Visible level] set to private
Click [Create Project]
#create user
Click the [wrench] icon at the top of the page to enter the "management area"
click [new user]
[name] set to zhangsan
[user name] set to zhangsan
[e-mail] set to [email protected]
here password is not set yet , and keep the default values for others, first click [Create User]
Note: Access level: Regular (represents ordinary users, who can only access his groups and projects); Admin (represents administrators, can access all groups and project)After creating a user, click [Edit] in the upper right corner, and you can set a password here, such as zhangsan@123
#Add users to the group
Click the [Group] drop-down at the top of the page, select devops_group,
click the left menu [Members]
[Add members to devops_group] select zhangsan, select Owner for role permissions,
and click [Add to group]
#Using the account created by yourself to manage
Exit the current account and log in with the account zhangsan created by yourself (zhangsan/zhangsan@123)
For the first login, you will be forced to change the password (zhangsan/zhangsan@123), which can be the same as the old password, log in again
//Jenkins installation
#Install JDK
tar zxvf jdk-11.0.5_linux-x64_bin.tar.gz -C /usr/local/vim /etc/profile
export JAVA_HOME=/usr/local/jdk-11.0.5
export CLASSPATH=.:${JAVA_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATHsource /etc/profile
java -version
#Install jenkins
wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo --no-check-certificate
rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.keyyum install epel-release -y
yum install jenkins -y
Or directly use the downloaded installation package
yum localinstall -y jenkins-2.401.2-1.1.noarch.rpm
#Modify the running user and listening port of jenkins
vim /etc/sysconfig/jenkins
JENKINS_USER="root"
JENKINS_PORT="8888"
##The new version of jenkins also needs to modify the username and port in the /usr/lib/systemd/system/jenkins.service file
vim /usr/lib/systemd/system/jenkins.service
User=root
Group=root
Environment="JAVA_HOME=/usr/local/jdk-11.0.5" comment remove
Environment="JENKINS_PORT=8888"
#Modify the startup script of jenkins to add the java path
vim /etc/init.d/jenkins
candidates="
......
/usr/local/jdk-11.0.5/bin/java
/usr/bin/java
systemctl daemon-reload
systemctl start jenkins
systemctl enable jenkins
vim /var/lib/jenkins/hudson.model.UpdateCenter.xml
<url>https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json</url>sed -i 's#https://www.google.com#https://www.baidu.com#' /var/lib/jenkins/updates/default.json
http://192.168.50.25:8888
systemctl restart jenkins
Browser access: http://192.168.50.25:8888, obtain and enter the admin account password
To install the git plugin, pull the code from the gitlab repository
Click the user name drop-down and select Credentials -> Jenkins Global -> Add Credentials
Type select "Username with password", enter the Gitlab username and password (zhangsan/zhangsan@123), enter gitlab-auth-password for the description and click "OK"
.
Create a FreeStyle project: New Item -> Task Name (test01) FreeStyle Project -> Determine
Source Code Management: Select Git, Repository URL: http://192.168.80.10:82/devops_group/web_demo.git (can be cloned from Gitlab using HTTP ),
select gitlab-auth-password from the Credentials drop-down, and click Save.
Click Build Now to start building. Check the console output to see that the build is successful. The built package will be generated in the /var/lib/jenkins/workspace/test01 directory of the server.
//Jenkins builds the Maven project
Maven is an automated build tool primarily for Java projects. Maven can also be used to build and manage projects developed in C#, Ruby, Scala, and other languages.
#Install Maven on the Jenkins server to compile and package the project
cd /opt
tar -zxvf apache-maven-3.6.2-bin.tar.gz
mv apache-maven-3.6.2 /usr/local/mavenvim /etc/profile
......
export MAVEN_HOME=/usr/local/maven
export PATH=$MAVEN_HOME/bin:${JAVA_HOME}/bin:$PATHsource /etc/profile
mvn -v
#Jenkins Linking JDK and Maven
tar xf jdk-8u171-linux-x64.tar.gz -C /usr/local/
Manage Jenkins -> Global Tool Configuration
-> JDK
Click Add JDK, enter jdk11 for the alias, uncheck Install automatically, enter /usr/local/jdk-11.0.5 for JAVA_HOME,
click Add JDK, enter jdk1.8 for the alias, uncheck Select Install automatically, JAVA_HOME input /usr/local/jdk1.8.0_171
-> Maven
click Add Maven, Name input maven3, uncheck Install automatically, MAVEN_HOME enter /usr/local/maven
click "Apply" and "Save" .
#Add Jenkins global variables to enable Jenkins to recognize the JDK and Maven environment commands
Manage Jenkins -> Configure System -> Global Properties, check the Environment variables
key-value pair list to add: key JAVA_HOME, value /usr/local/jdk-11.0. 5
key M2_HOME, value /usr/local/maven
key PATH+EXTRA, value $M2_HOME/bin
#Modify Maven's settings.xml
mkdir /root/repo #Create a local warehouse directory
vim /usr/local/maven/conf/settings.xml
54 --> #Local warehouse changed to /root/repo/
<localRepository>/root/repo</localRepository>
153 --> #添加阿里云私服地址
<mirror>
<id>alimaven</id>
<mirrorOf>central</mirrorOf>
<name>aliyun maven</name>
<url>http://maven.aliyun.com/nexus/content/groups/public/</url>
</mirror>
</mirrors>
//Tomcat installation and configuration
#Install Tomcat8.5
tar zxvf jdk-8u171-linux-x64.tar.gz -C /usr/local/
vim /etc/profile
export JAVA_HOME=/usr/local/jdk1.8.0_171
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:${JRE_HOME}/bin:$PATH
source /etc/profile
tar zxvf apache-tomcat-9.0.16.tar.gz
mv /opt/apache-tomcat-9.0.16/ /usr/local/tomcat
/usr/local/tomcat/bin/startup.sh
//Configure Tomcat user role permissions
By default, Tomcat does not configure user role permissions.
Verification method: You can access http://192.168.80.12:8080 with a browser, click the manager webapp under Managing Tomcat, and find that it returns a 403 page.
However, the subsequent Jenkins deployment project to the Tomcat server needs to use Tomcat users for remote deployment, so modify the following configuration of tomcat, add users and permissions
vim /usr/local/tomcat/conf/tomcat-users.xml
43 --> #add Tomcat user role permissions, specify the user and password are tomcat, and grant permissions
<role rolename="tomcat"/>
<role rolename="role1"/>
<role rolename="manager-script"/>
<role rolename=" manager-gui"/>
<role rolename="manager-status"/>
<role rolename="admin-gui"/>
<role rolename="admin-script"/>
<user username="tomcat" password="tomcat " roles="manager-gui,manager-script,tomcat,admin-gui,admin-script"/>
</tomcat-users>
vim /usr/local/tomcat/webapps/manager/META-INF/context.xml
<!-- # Comment out Valve configuration
<Valve className="org.apache.catalina.valves.RemoteAddrValve"
allow="127\.\ d+\.\d+\.\d+|::1|0:0:0:0:0:0:0:1" /> --
>/usr/local/tomcat/bin/shutdown.sh
/usr/local/tomcat/bin/startup.shVisit again: http://192.168.80.12:8080/manager/html, enter account/password tomcat/tomcat, you can log in successfully
login again
//Jenkins builds the Maven project
// Freestyle project build
Project integration process: pull code -> compile -> package -> deploy
#Pull the code
New item -> task name (web_demo_freestyle) FreeStyle Project -> OK
Source code management: select Git, Repository URL: [email protected]:devops_group/web_demo.git
Credentials pull down select gitlab-auth-ssh, click Save
and click Build Now, try build project
# Compile and package
Configuration—>Build->Add Build Step->Execute Shell, enter the following command
echo "Start compiling and packaging"
mvn clean package
echo "End of compiling and packaging"
Click Build Now to try to build the project
#Deploy the project to the remote Tomcat
1) Install the Deploy to container plug-in.
Jenkins itself cannot implement the function of remote deployment to Tomcat. You need to install the Deploy to container plug-in to realize it:
Manage Jenkins -> Manage Plug-ins-Optional Plug-ins -> Search for the Deploy to container plug-in and select it to install directly
2) Add post-build operations
Configuration—> Post-build operations-> Add post-build operations steps-> Deploy war/ear to a container
WAR/EAR files Input target/*.war
Containers —> Add Container —> Tomcat 9.x Remote -> Credentials, click Add -> Jenkins
user name input tomcat, password input tomcat, description input tomcat-auth, click Add
Credentials drop-down select tomcat-auth
Tomcat URL input http://192.168.80.12:8080
Click Save
Click Build Now to try to build the project
3) After the deployment is successful, refresh the http://192.168.80.12:8080/manager/html page, you can find that there is an extra project in the application, click on the new project to access it
//Pipeline pipeline project construction
Pipeline script is implemented by Groovy language, but we don't need to learn Groovy
Pipeline alone supports two grammars: Declarative (declarative) and Scripted Pipeline (scripted) grammar
#Install the Pipeline plugin (recommended plugins are installed by default)
Manage Jenkins -> Manage Plugins -> Optional Plugins –> Install Pipeline and Pipeline Stage View
#Pipeline Grammar Quick Start
1) Scripted Scripted Pipeline
New Item -> Task Name (web_demo_pipeline) Pipeline -> OK
流水线 -> 定义 选择 Pipeline script -> 选择 Scripted Pipeline
生成内容如下：
node {
def mvnHome
stage('Preparation') { // for display purposes
// Get some code from a GitHub repository
git 'https://github.com/jglick/simple-maven-project-with-tests.git'
// Get the Maven tool.
// ** NOTE: This 'M3' Maven tool must be configured
// ** in the global configuration.
mvnHome = tool 'M3'
}
stage('Build') {
// Run the maven build
withEnv(["MVN_HOME=$mvnHome"]) {
if (isUnix()) {
sh '"$MVN_HOME/bin/mvn" -Dmaven.test.failure.ignore clean package'
} else {
bat(/"%MVN_HOME%\bin\mvn" -Dmaven.test.failure.ignore clean package/)
}
}
}
stage('Results') {
junit '**/target/surefire-reports/TEST-*.xml'
archiveArtifacts 'target/*.jar'
}
}
●Node: Node, a Node represents a Jenkins node, Master or Agent node, is the specific operating environment for executing the Step, and will be used later when talking about the Master-Slave architecture of Jenkins.
●Stage: Stage, a Pipeline can be divided into several Stages, each Stage represents a set of operations, such as: Build, Test, Deploy, Stage is a concept of logical grouping.
●Step: Step, Step is the most basic operation unit, it can be to print a sentence, or to build a Docker image, which is provided by various Jenkins plug-ins, such as the command: sh 'make', which is equivalent to our usual shell terminal Execution of the make command is the same.
Write a simple scripted Pipeline:
node { def mvnHome stage('pull code') { echo 'pull code' } stage('compile and build') { echo 'compile and build' } stage('project deployment') { echo 'Project Deployment' } }
2) Declarative declarative
Pipeline -> Select the HelloWorld template
to generate the content as follows:
pipeline { agent any stages { #stages: represents all execution stages of the entire pipeline. Usually there is only one stage, which contains multiple stages stage('Hello') { #stage: represents a certain stage in the pipeline, and there may be n. It is generally divided into stages such as pulling code, compiling and building, and deploying. steps { #steps: Represents the logic that needs to be executed in a stage. In the steps are shell scripts, git pull code, ssh remote release and other arbitrary content echo 'Hello World' } } } }
Write a simple declarative Pipeline:
pipeline { agent any stages { stage('pull code') { steps { echo 'pull code' } } stage('compile and build') { steps { echo 'compile and build' } } stage ('Project Deployment') { steps { echo 'Project Deployment' } } } } Click Save Click Build Now to view Console Output to see the entire build process
#Use code generator to generate pipeline script:
pipeline -> pipeline syntax -> fragment generator
#Generate a pull stage
select checkout: Check out from version control
SCM select Git, Repository URL: [email protected]:devops_group/web_demo.git
Credentials drop-down select gitlab-auth-ssh
specified branch */master
click generate pipeline script
to generate After copying the script to the pipeline, click Save and try to build;
#Generate a build stage
Select sh: Shell Script
Shell Script Enter mvn clean package
and click Generate Pipeline Script
After copying the generated script into the pipeline, click Save and try to build;
#Generate a deployment stage
Select deploy: Deploy war/ear to a container
WAR files Enter targer/*.war
Containers —> Add Container —> Tomcat 8.x Remote -> Credentials drop-down select tomcat-auth
Tomcat URL Enter http:// 192.168.80.12:8080
Click Generate Pipeline Script
After copying the generated script into the pipeline, click Save to try to build and deploy.