Chapter 21 Docker Jenkins Continuous Integration Multi-Project Build

Jenkins continuous integration multi-project build

job scheduling

Build after other projects are built: Triggered when other projects are triggered. There are three situations in it, that is, when other projects build successfully, fail, or are unstable;

Poll SCM: Regularly check source code changes (according to the version number of the SCM software), checkout the latest code if there is an update, and then execute the build action.
H/5 * * * * (check source changes every 5 minutes)

Build periodically: Periodically build the project (it does not care whether the source code changes), as shown in the following configuration:
H 2 * * * (the source code must be built once at 2:00 every day)

Fill in 0 * * * * in Schedule.

The first parameter represents the minute minute, the value is 0~59;

The second parameter represents the hour hour, with a value of 0~23;

The third parameter represents the day, with a value of 1~31;

The fourth parameter represents the month, with a value of 1~12;

The last parameter represents the week, with a value of 0~7, 0 and 7 both represent Sunday.

So 0 * * * * means that the build is executed at the 0th minute of each hour.

Build example:

Because the code of the project is generally stored in SVN, and a SVN often has multiple project teams submitting code, and each project team is composed of multiple people, and each of them is constantly updating their own piece of code. Therefore, for a company, SVN submission records are often very frequent. Because of this, Jenkins often executes automated builds on a daily basis. The following example is in a day Commonly used timed build examples.

Build every 5 minutes: H/5 * * * *

Build every two hours: HH/2 * * *

Scheduled build every day before get off work at noon: 0 12 * * *

Build regularly every afternoon before leaving get off work: 0 18 * * *

project rollback

Although the automatic deployment of the project is realized, sometimes when the deployment fails, we need to roll back to the specified version of the build, so that we can build and deploy the project more flexibly. We can choose "parameterized build process" to pass different parameters to choose whether to make a new build or rollback
Steps:

1. Archive after construction After
   construction, archive the completed files for later use when rolling back

2. Parameters and build process

Use a parameterized build process so that subsequent scripts can perform different actions based on different variables. Add the "Choice" parameter to configure different options, let you choose whether to release or rollback, and add the "String Parameter" parameter to pass the version number to be rolled back.

3. execult build (not maven)

Choose the "Execute Shell" method for the build, and customize the build script according to the variables. At this time, if the build process of publishing and installing maven is used for a new build, if it is a rollback, know the files after the historical build and copy them to the current build Results directory.

4. Click Build, choose release or rollback according to different parameters, and fill in the historical version number to roll back to when rolling back

jenkins plugin

Both the Multijob plug-in and Jenkins Pipeline can implement serial execution of multiple jobs. When most processes need to execute public jobs, it is really convenient to use Multijob. You only need to add custom jobs. Pipeline uses the Groovy language to convert multiple job executions into scripting language execution. The entire process is completed by defining Node and Stage. The scalability is very good, and of course the learning cost will increase. There are pros and cons to each, choose according to your needs.

Summary of commonly used plugins in jenkins

Summary of commonly used jenkins plugins:
Build-timeout Plugin: Task build timeout plugin
Naginator Plugin: Task retry plugin
Build User Vars Plugin: User variable acquisition plugin
Build Pipeline Plugin View: Pipeline pipeline flow chart display plugin

Build Flow Plugin: workflow plug-in, support DSL script definition workflow
Build Graph View Plugin: build Flow plug-in view (need to be re-installed to take effect)
Multijob Plugin: multi-task plug-in
Build-timeout Plugin: job build time-out plug-in
Build Timestamp Plugin: The task log timestamp plugin, which adds the current time in front of each output of the job log.
Parameterized Trigger Plugin: This is an extended plug-in that enables each job to pass some job-related information when connecting. Join
Plugin: This is also a The highlight of the plug-in that triggers a job is that the condition for triggering a job is to wait for all downstream jobs of the current job to complete.
Files Found Trigger: Detect the specified directory, and start the build if a file of the specified pattern is found.
BuildResultTrigger Plugin: Start this build based on the success or failure of other jobs.
Publish Over SSH Plugin: Publish files via ssh
Rebuild Plugin: Re-execute the plugin
ws-cleanup Plugin: Workspace cleanup plugin
Cron Column Plugin: Routinely run some jobs through scheduled tasks

Job Configuration History Plugin: Usage experience: Make the job have the ability of version management, and the diff and rollback functions are very good
HTTP Request Plugin: Usage experience: You can use this plugin to call various api interface implementations and internal systems in the form of http before and after construction Linkage
Periodic Backup: Experience: Backup is something that must be guaranteed in the operation and maintenance of a system. The recovery function of this plugin may not be available and needs to be done manually. The advantage is that you can back up Job Import Plugin regularly: Experience: You can quickly import other
jenkins For existing jobs in the cluster, credentials must be provided to import the jenkins system that needs authentication.
Status Monitor Plugin: build status plug-in
Build Monitor View: Experience: Based on this plug-in, the dashboard function can be realized
Build Environment Plugin: build environment plug-in, which can build the environment Compare.
FTP plugin:
Monitoring: Monitoring of Jenkins

Basic dependency plugin:
jQuery Plugin: jQuery plugin

How to use findbugs:
Purpose: To automate code walkthroughs, to prompt garbage codes or provide code optimization suggestions
1. First download findbugs
http://findbugs.sourceforge.net/downloads.html 2. Next configure findbugs http
in ant
://www.51testing.com/?uid-265524-action-viewspace-itemid-200942
3. Download the findbugs plug-in in jenkins again
without configuration. If it is called by ant, the findbugs.xml file will be automatically generated in the project and in jenkins When drawing the findbugs curve

How to use emma:
Purpose: To check the test code coverage.
If you use maven, you only need to specify the target target emma:emma.
If you use ant, please continue to read
1. First download emma
http://emma.sourceforge.net/downloads. html
2. Secondly, configure emma in ant,
a practical example of ant http://ouzhong.blog.hexun.com/32369273_d.html
ibm ant-emma configuration: http://www.ibm.com/developerworks/cn/java/ j-lo-emma/
3. Download the emma-jenkins plugin
https://wiki.jenkins-ci.org/display/JENKINS/Emma+Plugin

Other brief introductions can be used when necessary:
​​1.ant plugin
2.disk usage plugin This plugin can monitor the physical space spent by each build project
3.thinbackup This plugin can back up your job configuration
4.jenkins workspace cleanup plugin This The plugin can clear the workspace before each build
5.build-name-setter This plugin can set the name displayed for each build
6.git plugin
7.PMD plugin this plugin
8.python plugin This plugin supports you to write python like a shell Code
9.email-ext plugin This plugin provides you to send emails in HTML format
10.hudson next build number plugin If you want to put the version number of each build of hudson into your product version number, you can use this plugin
11.HTML publisher plugin supports reports in HTML format
12.android lint plugin supports android project inspection
13.Copy Artifact Plugin supports copying the results generated after build to other projects
14.promoted supports manual testing and verification, or releases products manually after the build Execute
15. Jenkins description setter plugin This plugin can support the modification of the job description, such as QR code
16.Hudson Port Allocator Plug-in Open and close the port, or choose a free port to build
17.Android Emulator Plugin
18.JobConfigHistory Plugin This plugin is used to view the jobconfig history
19.Jenkins Sounds plugin This plugin is used after the build A sound prompt will be given
20. JIRA Plugin This plugin is used to work with Jira

Maven builds individual modules in multi-module project separately

1. Root POM points to parent pom.xml

2. Goals and options specify the parameters of the building module: mvn -pl jsoft-web -am clean package, build the jsoft-web project and other projects it depends on separately

illustrate:

1. There may be scenarios where multi-module projects do not refer to each other. At this time, a single project can be built separately, and the compilation can be completed by specifying the pom.xml file of the sub-module.

2. If each of the multi-module projects is referenced, then compiling the pom.xml file of the sub-module separately will directly report an error. The solution is to compile the parent project pom.xml.

3. If the parent project is compiled, the compilation time may be very slow, and some projects do not need to be compiled. The solution is as follows:

Solution:

Maven options:

-pl, --projects
        Build specified reactor projects instead of all projects
-am, --also-make
        If project list is specified, also build projects required by the list
-amd, --also-make-dependents
        If project list is specified, also build projects that depend on projects on the list

First switch to the root directory of the project

Build the module ringyin-commonb separately, and build other modules that the ringyin-common module depends on

mvn install -pl ringyin-common -am

Build the module ringyin-common separately, and build other modules that depend on the module ringyin-common at the same time

mvn install -pl ringyin-common -am -amd

clean install -Pdev -Dmaven.test.skip=true

Goals and options specify the parameters of the building module: mvn -pl jsoft-web -am clean package, build the jsoft-web project and other projects it depends on separately.

Multiple SCMs is a plug-in of Jenkins, which mainly solves the need to pull multiple warehouse codes for one release. For example, if you want to pull project A and pull the code of project B from the subfolder asset/src/h5 of project A, put it in folder asset/src/h5. Just use this plugin.
The configuration of the Additional Behaviours item is to solve the problem of pulling two projects at the same time, and the B project must be placed in the asset/src/h5 of the A project.

git checkout [-q] [-f] [-m] [<branch>]
git checkout [-q] [-f] [-m] --detach [<branch>]
git checkout [-q] [-f] [-m] [--detach] <commit>
git checkout [-q] [-f] [-m] [[-b|-B|--orphan] <new_branch>] [<start_point>]
git checkout [-f|--ours|--theirs|-m|--conflict=<style>] [<tree-ish>] [--] <paths>…
git checkout [<tree-ish>] [--] <pathspec>…
git checkout (-p|--patch) [<tree-ish>] [--] [<paths>…]

Use of Jenkins plugin for continuous integration (1) - serial and parallel connection between multiple jobs

Reprinted from: http://qa.blog.163.com/blog/static/190147002201391661510655/

In addition to being open source and free, one of the most attractive features of Jenkins is the support for plug-ins.

Jenkins not only has a wealth of third-party plug-ins, but also can write plug-ins by itself, and deeply integrate with other tools, so as to achieve a customization that meets the company's product management needs.

Next, we will share a series of practical experience in the use and development of Jenkins plug-ins, and introduce some classic and practical third-party plug-ins and plug-ins developed by the company itself. We hope that you can flexibly use them in your work according to the needs of the project.

The first five plug-ins introduced this time are all related to Jenkins jobs. Through these plug-ins, various jobs can be connected as needed to make them work together.

Before we start the introduction, let's take a look at why these plug-ins are needed to connect each job. Some people may say, why can't all the continuous integration steps of the product be built on one job?

- First of all, Jenkins has a limitation, that is, all tasks of a job's one-time build can only run on one node. Need to be under Windows) you must create a different job

- Again, if there is only one job, the construction time will inevitably be longer, which means that the quality feedback of the product will be slower (you must wait until all tasks are completed to get feedback)

- Also, building jobs separately is also for the purpose of "weak coupling" between each other, so that the configuration changes of corresponding jobs can affect other jobs as little as possible

- Finally, this can make the continuous integration process of the product clearer and discover the corresponding shortcomings of the product in time

1. Copy Artifact Plugin

Function introduction : This is a relatively simple plug-in, which simply copies the construction of a job to the workspace of the current job.

Simple configuration:

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-CFHsBhU7-1599615307566)(assets/2294865485222059354.png)]

Practical application :

- After the product is compiled/packaged, it can be used when it needs to be deployed in multiple environments for testing/joint debugging/drilling. A job is responsible for compiling and packaging the code, and archiving the builds (usually WAR, JAR, TAR, etc.), and then multiple jobs can obtain the corresponding builds for product deployment, ensuring the integrity of the deployment environment. consistency.

Tips：

+ Make sure that the upstream job's builds are archived

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-czKztyZh-1599615307569)(assets/1403152759002810735.png)]

+ By default, the copied builds will be stored in the same directory structure, if you just want to keep the files, you can choose "Flatten directories", so that all builds will be copied directly to the "Target directory" without saving Directory Structure

2. Parameterized Trigger Plugin

Function introduction : This is an extended plug-in, so that each job can pass some job-related information when connecting

​ - the parameters of the current job

​ - custom parameters

​ - SCM related information

​ - Running Node information

Simple configuration :

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-XQyPlJv3-1599615307572)(assets/888335026599348390.png)]

Practical application :

- Transfer SVN Revision: In the code checkout stage, the corresponding SVN Revision information will be obtained, and this information will be passed to the downstream job, and the corresponding version of the code will be directly checked out in each downstream job to maintain the version consistency of each build. Prevent inconsistencies in the running versions of each job due to frequent code submissions

- Keep each job running under the same node: If there are multiple sets of test environments, you can check this option to maintain the consistency of the build environment

Tips：

+ Passing SVN Revision or GIT Revision can be done through the following settings

[External link image transfer failed, the source site may have an anti-theft link mechanism, it is recommended to save the image and upload it directly (img-zkRbZke1-1599615307577)(assets/1628332740371262405.png)] [External link image transfer failed, the source site may have Anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-Q45NtTGv-1599615307579)(assets/6597962570100087784.png)]

If the downstream job needs to be used in SCM management, it can be configured through the following settings

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-y03XmYut-1599615307581)(assets/6597835026751262986.png)]

+ If you don't need to pass any parameters, be sure to check "Trigger build without parameters"

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-XdDXhZQv-1599615307583)(assets/2858659863573689837.png)]

3. Build Pipeline Plugin

Function introduction : This is a plug-in for generating a specific view, which can visualize the relationship between jobs and visualize the process of the product.

Simple configuration :

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-mIqV5WAx-1599615307586)(assets/753227037777807296.png)]

Practical application :

- When configuring the continuous integration of products, there are often multiple jobs working together, such as compiling/packaging, static code checking, unit testing, interface testing, UI testing, performance/stress testing, and each product has certain dependencies on each other . By setting the initial job in this plug-in, the relationship between jobs can be sorted out intuitively, and the overall situation of each build of the product can also be seen. In the later stage, you can also select the appropriate version from the build information to increase the release link.

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-zVEjSAnb-1599615307588)(assets/717761190712256131.png)]

Tips：

+ By setting the option "Show pipeline parameters in revision box", you can visually display the build number of the job, combined with the VersionNumberPlugin , you can display the SVN revision

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-FyGUfRuR-1599615307589)(assets/1548675321880058404.png)]

+ By setting the option "Show pipeline parameters in project headers", you can visually display some parameters of the job, such as the test environment

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-u2ID4L9i-1599615307591)(assets/1138284805918161457.png)]

4. Multijob Plugin

Function introduction : This is a plug-in that integrates ParameterizedTrigger and BuildPipeline, but it forms a new job, not a view.

And it does not require dependencies between jobs themselves. In this way, relative independence can be maintained when creating a job, and the continuous integration environment required by the product can be assembled through this plug-in.

Simple configuration :

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-DGJypcnl-1599615307593)(assets/1998472334745775061.png)]

Practical application :

- If you need to create multiple pipelines, and some jobs in each pipeline can be shared, you can consider using this plugin. For example, the product has two branches, develop and release, and you need to check out the code (A1, A2), compile (B), and unit test (C) respectively. For the develop branch, only a simple regression test (D) is required, and for the release branch Not just regression testing (D), but also performance testing (E). In this case, there will be two pipelines A1-BCD and A2-BCDE. For D, the downstream job is uncertain. At this time, it can be realized by establishing two Multijobs (of course, it can also be passed through ParameterizedTrigger. Set and read variables to achieve, but the configuration is more complicated)

Tips：

+ Be sure to select "MultiJob Project" when creating a job, this plugin cannot be selected by default under "free-style project"

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-m1fOoGy4-1599615307595)(assets/2233785415275847580.png)]

+ For the results of some jobs do not want to affect the execution of downstream jobs, such as static code inspection, the following settings can be used to achieve the goal

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-ymxPyXk3-1599615307597)(assets/6598209960216216822.png)]

5. Join Plugin

Function introduction : This is also a plug-in that triggers a job. The highlight is that the condition for triggering a job is to wait for all downstream jobs of the current job to complete.

Of course, this only makes sense if the current job has two or more downstream jobs.

To give a simple example, A triggers two jobs B1 and B2 at the same time, and then configures this plug-in to trigger C. At this time, C will not be executed until both B1 and B2 are completed.

Simple configuration :

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-kES9EzWM-1599615307599)(assets/2240822289693614677.png)]

Practical application :

- The deployment of some products will depend on some third-party products, and you can deploy your own products and third-party services at the same time. At this time, the next test work needs to wait for all these deployments to be completed. Naturally, this plug-in will be useful

- In order to speed up the construction, it is usually possible to set the unit test and the smoke test to be performed at the same time, and the subsequent functional test is expected to be performed only when the unit test and the smoke test pass. In this case, the Join plugin is necessary it's

Tips：

+ By default, the job triggered by the Join plugin cannot pass parameters. If necessary, you can check "Trigger parameterized build on other project", so that the Join plugin and the ParameterizedTrigger plugin are actually integrated.

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-nYLMuY6A-1599615307601)(assets/711850216201331475.png)]

Summary: The plug-ins described above are not used in general, and some functions can be realized by more than one plug-in, so you can flexibly apply them according to the needs of products and projects, and build a reliable, maintainable, and strong visualization. Integrated environment

script execution plugin

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-iToAosnC-1599615307603)(assets/1544772728525.png)]

reference

Jenkins continuous integration comprehensive practice

https://blog.csdn.net/kefengwang/article/details/54233584

Pipeline plug-in introduction and use

https://blog.csdn.net/diantun00/article/details/81075007

getting started guide

http://www.cnblogs.com/zz0412/tag/jenkins/

http://www.cnblogs.com/itech/archive/2011/11/04/2236230.html

Jenkins actual project operation

https://www.cnblogs.com/kangaoxiaoshi/p/6022406.html

Continuous integration environment with strong maintenance and visualization**

script execution plugin

[External link image transfer...(img-iToAosnC-1599615307603)]

reference

Jenkins continuous integration comprehensive practice

https://blog.csdn.net/kefengwang/article/details/54233584

Pipeline plug-in introduction and use

https://blog.csdn.net/diantun00/article/details/81075007

getting started guide

http://www.cnblogs.com/zz0412/tag/jenkins/

http://www.cnblogs.com/itech/archive/2011/11/04/2236230.html

Jenkins actual project operation

https://www.cnblogs.com/kangaoxiaoshi/p/6022406.html

https://cloud.tencent.com/developer/article/1342915