Docker continuous deployment, automatic deployment with Git and Jenkins, puppet

CI: continuous integration Continuous integration
Git and Jenkins cooperate to automatically build and deploy Puppet, saltstack can
also complete automatic deployment


http://www.infoq.com/cn/articles/effective-ops-part-06

Foreword There are overwhelming articles

about Docker, but High-quality articles tend to be mostly translated. It is said that Docker is naturally suitable for continuous integration/continuous deployment, but again, articles that can be implemented and practical are rare.

Based on these circumstances, although our column is positioned as an operation and maintenance management text, this article is a special case, a practical case explanation - how JAVA projects can achieve continuous deployment through Docker (just four simple steps), namely:

relevant manufacturer content

Why does Qiniu hold a conference with the proposition of "data"? InfoQ China's operation team strives to create - EGO super geek state CNUTCon wonderful content foresight: Container application case sharing of first-tier Internet companies When did BAT start using container or Docker technology? What is the application scenario? Container cluster management behind Docker - from Borg to Kubernetes (1)
Related sponsors


CNUTCon Container Technology Conference, August 28-29, 2015 at Crowne Plaza Beijing Xinyunnan. Sign up now.

Developers upload code through git push, and with the cooperation of Git and Jenkins, program deployment and release are automatically completed, without the need for operation and maintenance personnel to participate in the whole process.

This is a true container-level implementation, and the benefits of this are not only improved efficiency, but also a change:

for the first time, developers are truly responsible for their own code - and can finally skip operations and maintenance. The testing department, which independently maintains the operating environment (the first is the test/development environment).

Those who are difficult will not, those who will meet are not difficult. With 4 simple configurations, continuous deployment can be achieved gracefully. This article is placed in the table of contents as usual, please enjoy.

Technical ideas of continuous deployment

Effect display

Configure Git and Jenkins linkage

Configure Jenkins to automatically update code

Detailed explanation of effect graphic and text

FAQ

Well, we officially start.

1. The technical idea of ​​continuous deployment

In this example, assume that the name of our JAVA project is hello. A brief technical idea is as follows.



In this case, it is assumed that the code is hosted on git.oschina.com, and Jenkins and Docker Registry (similar to the yum source) each run in a Docker container. The JAVA project itself also runs separately in a container called hello.

The continuous deployment solution adopted in this article is to pull code from the private Docker Registry. Some workarounds are to put the code on the host and let the container read it through the volume group map. The reason this approach is not recommended is that splitting the code out of the container goes against Docker's container principles:

it also leads to increased loading and unloading complexity. From the perspective of freight workers, the whole is the most economical. Only in this way can real container-level migration be realized.

In other words, in the container era, it is the right way to abandon the idea of ​​file distribution in the past. More on this in the Q&A at the end of this article.

A container is a process. The reason and significance of using the above solution for Docker continuous deployment is also here. The life cycle of a container should be much shorter than that of a virtual machine. If there is a problem with a container, it should be killed immediately instead of trying to recover.

2. Effect display How amazing is the effect finally achieved in

this article ? And see the demo below.

2.1 The effect before the program code update

We use timestamps to concisely and explicitly represent program updates.



2.2 Submitting program code update

In this example, we changed the timestamp of the home page from 201506181750 to 201506191410 (see below).



2.3 Upload the new code to Git

Perform the following operations in sequence, and enter the correct git account password.



and then?

Then do nothing. Grab a cup of tea (if you don't like coffee), wait quietly for the auto-deployment to happen, and watch a series of processes that are automatically triggered and run like a robot (please describe it later).

Why does it take 3~5 minutes? Just because of the JAVA project in this case, the Maven package needs to be downloaded from abroad for Jenkins to call and compile JAVA. In the formal application environment, the Maven source can be placed in the domestic or computer room. If you only need to do continuous deployment of PHP projects, it will be faster.

2.4 Check the effect of the code update

After waiting quietly for a few minutes, the new code has indeed been automatically deployed.



So how did all this happen? Is it complicated? otherwise. Just follow the steps below and you can do it quickly.

3. Configure

the . It is mainly divided into the following three steps.

3.1 Jenkins configuration Git source

Create a new project java-app in Jenkins, and configure to pull program code from Git. The details are as follows:



3.2 Jenkins configuration remote build

The token is configured in Jenkins for use when git is called remotely.



3.3 Git open hook

How to make Git pass messages and tasks to Jenkins after receiving the user's updated code? This is also very simple to configure with the help of Git's hook function, as follows.



4. Configure Jenkins to automatically update code

The main job of Jenkins is to configure "remote builds". After receiving the message from Git, trigger this remote build (to the target server), perform a series of jobs, rebuild the container, etc. according to the predefined task list. Details are as follows:



We excerpted the most critical Shell script content. These Docker-related operations have already been mentioned in Part 1 "Technical Ideas" and will not be repeated here.



5. Detailed explanation of the effect picture and text

In this chapter 2.3, our operation at that time is as follows, the purpose is to submit the updated code to Git.



At that time, there was no detailed description of what happened next. Since the principle has been clearly explained above, we can talk about what actually happened next.

5.1 Uploading code to Git

It seems that the whole process has been completed and exited smoothly. In fact, the background work has only just begun.



At this time, the Git server will be triggered to send an operation request to the corresponding Jenkins server. This work is too fast, and there is nothing to say. We will see what Jenkins does next.

5.2 The wonderful interaction of Jenkins The

following automatic process gives us a little sense of accomplishment, and it is worth a cup of coffee (if you don't like tea) and watch it quietly.

1) Jenkins will automatically "pop up" a build task.



2) Let's click in to see the specific operation log. Yes, accepting tasks from Git.



3) Download Maven related packages (that is, this process is slow).



4) After the download is complete, start using the new hello project package of maven BUILD.



5) Then rebuild the Maven container, build a new Image and push it to the Docker private library.



6) Finally, pull up the Docker container again. Thus, it was born again. Ha ha



6. FAQ

Question 1: Is it because the project is based on JAVA that such a relatively complicated method (rather than putting the update code on the host and then the volume group mapping); whether the PHP project can use the update code on the host and then volume Group mapping this way?

Answer 1: Splitting the code out of the container violates the container principle. This leads to increased loading and unloading complexity. From the perspective of freight workers, the whole is the most economical. Everything is versioned. Ditch the file distribution of the past. This is the right way. As for the file size, the large war package is only 50M or 100M, which is not a problem under the existing network, and the performance problem is best optimized. It is also recommended to pay attention to docker 2 docker, p2p transmission.

Question 2: If the overall code exceeds 500m or 1g, is the overall container not very good? If the container is separated from the code, the image is about 100m (2 layers, base+service), and then the code is placed in the shared storage, each code is updated, such as the svn code, you can directly perform svn update in the shared storage. Can control the version

Answer 2: If your code is 500M, it only means that it is time for business development to hit the board.

Question 3: If the test environment uses the complete container service provided by you, it is fine, but in the production environment, running docker in the cluster as an application, if each container has complete code, is it a bit bloated? Run the basic service image, bind the code in the shared storage through the volume group function, and add Crontab, Python and Shell scripts, so that each code update only needs to be done once.

Answer 3: Environmental consistency has never been solved in the past. When we did paas 10 years ago, it was similar. It's not that it's bad, times have changed, and it's hard to have a good system after all. You can't just consider the current convenience. If you compare container technology and VM, I think it will make me very tangled when making a decision.

Supplement 3: The script is generally the thinking of a typical operation and maintenance engineer, quick & dirty. It is generally difficult to make a product or system. Overall considerations and scalability considerations are relatively small. The difficulty of doing docker now is how to look at it. Should it be used as the basic unit of scheduling, or should it be considered as the basic unit of deployment? Let's talk about the plan.