An article takes you through building a DevOps platform using Jenkins and Kubernetes

Introduction to DevOps

First, let’s take a look at a picture (the picture comes from the Internet, if there is any infringement, please contact me to delete it):

DevOps is actually a system that can provide a complete process from coding to construction to testing to release to deployment. This article will take you to experience a complete process of automated deployment using Jenkins and Kubernetes.
Video tutorial address: https://www.bilibili.com/video/BV1z8411S7Z8/

Jenkins environment preparation

This section explains the installation of jenkins, including preparations for JDK, Maven, and Docker

Prepare JDK

Download jdk

The JDK11 we want to install this time (be careful not to use jdk8, I will explain why later), you can go to the download interface:https://www.oracle.com /java/technologies/downloads/#java11, what we want to install this time is the Linux version, so click:

Then you need to agree to the license and click to download jdk11:

Finally, log in to your oracle account to download. If you don't have an oracle account, you can register.

Install jdk

Next we need to upload the JDK to the jenkins server, create an installation directory and install it, using the command (note that /opt/environment is the root directory of the installation and can be modified by yourself): a>

mkdir -p /opt/environment
tar -zxvf jdk-11.0.19_linux-x64_bin.tar.gz -C /opt/environment
cd /opt/environment
ll

You can view the results of the installation:

Configure jdk environment variables

We need to configure the following lines of code at the end of /etc/profile (note that /opt/environment/jdk-11.0.19 is the root directory of java, pay attention to modifications):

# Set java environment
JAVA_HOME=/opt/environment/jdk-11.0.19
PATH=$PATH:$JAVA_HOME/bin
export JAVA_HOME PATH

After configuration, it looks like this:

Finally, execute the following command to make the configuration take effect:

# 使配置生效
source /etc/profile
# 检查java是否安装成功
java -version

Seeing this means that the java installation is complete:

Prepare maven

Download maven

We can directly execute this command to download the maven compressed package:

wget https://archive.apache.org/dist/maven/maven-3/3.8.8/binaries/apache-maven-3.8.8-bin.tar.gz

Unzip maven

You can use this line of command to decompress (note/opt/environment is the installation directory):

tar -zxvf apache-maven-3.8.8-bin.tar.gz -C /opt/environment

Configure maven

There is a file in the conf folder in the root directory of maven (/opt/environment/apache-maven-3.8.8). We need to modify this file. First we It is necessary to modify the storage directory after downloading the dependencies. There is a detailed description on line 53 of the file:settings.xml

You can see that this directory is in the .m2 directory of the user's home directory by default. I plan to change this directory to/opt/environment/apache-maven-3.8.8/repository, so add a line of configuration directly below: a>

<localRepository>/opt/environment/apache-maven-3.8.8/repository</localRepository>

As shown in the picture:

Because Maven's default dependency download address is the official Maven address abroad, the download may be too slow or even fail when downloading dependencies, so we need to change the download address to the domestic Alibaba Cloud address and find mirrors tag, then add a mirror configuration item below and add the code:

<mirror>
  <id>alimaven</id>
  <name>aliyun maven</name>
  <url>https://maven.aliyun.com/repository/public</url>
  <mirrorOf>central</mirrorOf>
</mirror>

After configuration, it looks like this:

Configure maven environment variables

We need to continue to modify the /etc/profile file and add the following lines of configuration below (note that /opt/environment/apache-maven-3.8.8 is the root directory of maven):

# Set maven environment
MAVEN_HOME=/opt/environment/apache-maven-3.8.8
PATH=$PATH:$MAVEN_HOME/bin
export MAVEN_HOME PATH

After modification, it looks like this:

Next we use these lines of commands to make the configuration take effect:

# 使配置生效
source /etc/profile
# 检查maven是否安装成功
mvn -v

The execution result is as shown in the figure:

Install Docker

I have already written an article about installing docker. The document address is:https://blog.csdn.net/m0_51510236/article/details/115054073< a i=2>, we can execute the following command to quickly install docker:

sudo yum remove docker \
  docker-client \
  docker-client-latest \
  docker-common \
  docker-latest \
  docker-latest-logrotate \
  docker-logrotate \
  docker-engine
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
sudo yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
sudo sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
sudo yum install -y docker-ce docker-ce-cli containerd.io
sudo systemctl daemon-reload
sudo systemctl enable --now docker

The only difference from my previous article is that the default yum source of docker is changed to the yum source of Alibaba Cloud. The result after installation is as shown in the figure (the docker version may be different depending on the installation time):

Install git

Just use the following command to install it

yum install -y git

Install Jenkins

This time I prepared the download address of the latest stable version of Jenkins. According to the official documentation, the minimum jdk version supported by the latest stable version is jdk11, so this is also the reason why jdk11 was installed before:

Install jenkins dependencies:

yum install -y fontconfig

You can directly use this line of command to download the latest stable version of the war package of jenkins:

wget https://get.jenkins.io/war-stable/2.401.2/jenkins.war

I plan to store this war package under /opt/server/jenkins, so we use the following command to move it there:

mkdir -p /opt/server/jenkins
mv jenkins.war /opt/server/jenkins

Next we can run jenkins directly,

cd /opt/server/jenkins
nohup java -jar jenkins.war --httpPort=8080 >> /dev/null &

Initialize jenkins

After running jenkins with the above steps, we can directly access http://<jenkins服务器地址>:8080 to enter the jenkins console:

Follow the prompts to view the initialized Jenkins password:

cat /root/.jenkins/secrets/initialAdminPassword

Check the password as shown below:

Next, we directly use this password to log in to the Jenkins console, and then we click to select the plug-in to install:

You can add or cancel the installation of some plug-ins according to your needs. After selecting, click the install button directly:

Next, you will enter the plug-in installation interface. If the installation fails, it is okay. After entering the Jenkins console, we can still install some plug-ins according to our needs:

After the installation is completed, we need to create the first administrator user. After filling in the content according to the prompts, we click directly保存并完成:

Configure the address of jenkins. If there is no problem, click directly保持并完成:

True point开始使用Jenkins：

Then we came to the main interface of jenkins:

ClickManage Jenkins then clickPlugins, we need to install some more plug-ins:

Point 击Available plugins：

SearchGit Parameter and select in the previous selection box:

Then searchMaven Integration and select in the previous selection box, and finally clickInstall without restart:

The installation is completed as shown in the figure:

Then we continue to under Manage Jenkins: Tools

Click to find the location of JDK 新增JDK:

Fill in the content according to the prompts:

Then we continue to search below, find the Maven configuration and click 新增Maven:

After filling in the content as prompted, click Apply and then Save:

Now Jenkins has been initialized.

Prepare code warehouse and docker image warehouse

For the installation tutorial of the privatized code warehouse, please refer to the article I wrote:https://blog.csdn.net/m0_51510236/article/details/120440229

For the sake of simplicity and speed, this article directly usesgithub as the code repository

For privatized docker warehouse, you can refer to the article I wrote about Harbor:https://blog.csdn.net/m0_51510236/article/details/125030239

Of course, this time I will use Alibaba Cloud’s free personal version of the container image service instead, address:https://www.aliyun.com/product/acr

Prepare for Kubernetes

I have written an article before about building a kubernetes cluster:https://blog.csdn.net/m0_51510236/article/details/130842122< a i=2>, I will not explain the installation steps in detail here, and I have already set up a kubernetes cluster:

This article uses this kubernetes cluster for testing.

Prepare java project

We come tospring initializrCreate a SpringBoot project, the creation options are as shown in the figure:

Then import the project into the development tools:

Add a new controller in DevOpsController.java with the following code:

package com.greateme.devops.controller;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

/**
 * <p>
 * 测试DevOps的控制器
 * </p>
 *
 * @author XiaoHH
 * @version 1.0.0
 * @date 2023-07-17 星期一 12:18:47
 * @file DevOpsController.java
 */
@RestController
public class DevOpsController {
    
    

    /**
     * 用于测试的请求mapping
     */
    @GetMapping("/devOps")
    public String devOps() {
    
    
        return "Hello DevOps --- v1.0.0";
    }
}

The execution result is shown in the figure:

Build DevOps

We need to push the code to GitHub, then let Jenkins pull the code, make it into a Docker image and push the image to the image warehouse, and finally deploy it to Kubernetes. Next, follow the steps of this article.

Create code repository

We need to create a code repository and push the code to GitHub (it can also be other code repositories):

Create Dockerfile

I have written a related article before:https://blog.csdn.net/m0_51510236/article/details/122700574 , of course, this article can also help you use Dockerfile to package Docker projects. First, we go to DockerHub to determine the image we need to use. This time we will use openjdk:11-jre to package the project as the basic image:

Create a docker folder in the project root directory, and then create a Dockerfile under the docker folder with the following content:

# 使用openjdk11做基础镜像
FROM openjdk:11-jre

# 作者
MAINTAINER XiaoHH

# 将可执行的jar包放到容器当中去
ADD java/devops-example.jar app.jar

# 暴露8080端口
EXPOSE 8080

# JVM 调优参数
ENV JAVA_OPTION="-Dfile.encoding=UTF-8 -XX:+UseParallelGC -XX:+PrintGCDetails -Xloggc:/var/log/devops-example.gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC"

# JVM 内存默认值
ENV XMX=128m
ENV MXS=128m
ENV XMN=64m

# 运行程序
ENTRYPOINT ["sh", "-c",  "java -Djava.security.egd=file:/dev/./urandom -jar -Xmx${XMX} -Xms${MXS} -Xmn${XMN} $JAVA_OPTION /app.jar"]

Note devops-example.jar This file name is used because I configured the file name in pom.xml:

<build>
  <finalName>${artifactId}</finalName>
  <plugins>
    <plugin>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-maven-plugin</artifactId>
    </plugin>
  </plugins>
</build>

The configuration is as shown in the figure:

CreateJenkinsfile

Create a Jenkinsfile file in the root directory of the project, file content:

pipeline {
    agent any

    stages {
        stage('Hello') {
            steps {
                echo 'Hello World'
            }
        }
    }
}

The above is a very simple HelloWorld content, we will modify it later

Submit code

First we submit the code once, which I believe is very simple:

Then we come to the command line, create a tag named v1.0.0 and push it:

git tag -a v1.0.0 -m 'New Version'
git push origin v1.0.0

The execution result is as shown in the figure:

You can see that we already have a tag on GitHub:

View tag list:

Configure the Jenkins server to pull the code without password

First we need to generate an SSH Key on the Jenkins server and execute these lines of commands on the server (note that some of the configurations are modified):

# 配置个人信息（注意替换邮箱地址）
git config --global user.name "XiaoHH"
git config --global user.email "[email protected]"
# 生成 SSH key
ssh-keygen -t rsa -C "JenkinsServer"
# （之后连续几个回车即可生成SSH key）

After several consecutive carriage returns, a .ssh folder will be generated in the home directory, which contains a id_rsa.pub file. This is your ssh public key. File:

We need to upload this public key file to GitHub to pull the code without a password. After clicking the avatar on GitHub, click Settings:

Next point SSH and GPG Keys Sum New SSH key :

Fill in the content according to the prompts:

Test pull code:

git clone 你的ssh仓库地址

As shown in the figure, the code was successfully pulled:

Create pipeline

Go to the Jenkins homepage and click 新增Item:

Create a pipeline type task:

Add a build parameter, the parameter is the git tag (tag), and fill in the corresponding content as prompted (if Jenkins has been restarted, it will be displayed参数化构建过程):

Pull down to the pipeline definition:

There is also a script file name behind it (finally click Apply and Save):

Then try to hit build:

Here you can see the tags we defined and submitted before, click on the tag you want to build, and then click to start building:

You can see that the only hello step we defined successfully outputs HelloWorld:

Next we need to let this pipeline complete the following steps:

• Switch to the version we specified
• Build the project through maven
• Convert the built maven project into a Docker image
• Push the Docker image to the mirror warehouse
• Deploy to Kubernetes through remote connection

Jenkinsfile switches the code repository to the specified version

Click Configure:

Scroll to the bottom and there is a pipeline syntax:

After clicking in, follow the prompts:

Then we don’t need to fill in our own branch for the following branches. We need to fill in ${tag} to reference the tag variable we defined before. Finally, click Generate Script to see the generated script below: < /span>

Then copy the generated script and go to our projectJenkinsfile Delete the Hello stage we defined before and add a new one Checkout From Git stage (the name can be modified by yourself), and copy the generated script into it:

Build maven project

We can continue to use the script generator to generate scripts, select and fill in the prompts:

In Shell Script we wrote a script, I will explain it separately now:

# 使用maven打包项目
mvn clean package -Dmaven.test.skip
# 创建docker下存放jar包的目录
mkdir docker/java
# 移动构建好的jar包到docker的打包目录
mv target/devops-example.jar docker/java
# 操作完后清理工作空间
mvn clean

Similarly, we add a new stageBuild By Maven in the project's Jenkinsfile and put the generated code inside as shown:

Next, we submit the code and try to build it again:

You can see that we built successfully, but because the first time we build Maven, we need to download dependencies, so the build is slow.

Build Docker image

Prepare warehouse

First we need to prepare the Docker warehouse. When we go to Alibaba Cloud, we add a new warehouse and select a personal instance. If this is your first time using it, you may need to create a personal instance:

First we create a namespace:

Choose to create a mirror warehouse:

Select the local repository for the code repository:

After the creation is completed, we also need to use Docker to log in to the Alibaba Cloud image warehouse:

Login results on the Jenkins server:

Pay attention to enter the password. The warehouse is ready. Next we start building the image.

Scripting

Copy this public network address. This will be the image name used to build the image:

We continue to use the script generator to generate the script:

Script description:

# 进入到打包目录
cd docker
# 打成Docker包，后面的tag引用打包时选择的tag
docker build -t registry.cn-hangzhou.aliyuncs.com/xiaohh-test/devops-example:${tag} .
# 打包完成删除这次打包的java目录
rm -rf java

Add a new stage in Jenkinsfile Build By Docker Copy the generated script into it, as shown in the figure:

Push the image to the Docker repository

After packaging is completed, push the image directly to Alibaba Cloud, still using the script generator:

Add a new Push Docker Image To Repository stage and put the generated script into it:

After submitting the code, try again and the build is successful:

Successfully pushed the image to the mirror warehouse:

Deploy to Kubernetes

Manual deployment

First, we use yaml to deploy to kubernetes, and first create a secret to save the Docker login information:

# 注意修改docker服务器地址、登陆用户名、密码和邮箱
kubectl create secret docker-registry devops-secret --docker-server='registry.cn-hangzhou.aliyuncs.com' --docker-username='devops@1005882688039568' --docker-password='***' --docker-email='***@qq.com' --dry-run -o yaml >> deploy-devops.yaml

After execution, view the contents of deploy-devops.yaml:

Then we generate a Deployment to deploy this java program. Before generating, we need to add --- after deploy-devops.yaml because we need to separate two yaml sections, as shown in the figure: a>

Command to generate yaml:

kubectl create deployment devops-example --image=registry.cn-hangzhou.aliyuncs.com/xiaohh-test/devops-example:v1.0.0 --replicas=1 --port=8080 --dry-run -o yaml >> deploy-devops.yaml

Because we need to add a password when pulling the image, we also need to modify this file and add a under spec.template.spec To reference the secret we created before, as shown in the figure:imagePullSecrets

Complete yaml configuration content:

apiVersion: v1
data:
  # 这个里面存储了docker的登陆名和密码
  .dockerconfigjson: ***
kind: Secret
metadata:
  creationTimestamp: null
  name: devops-secret
type: kubernetes.io/dockerconfigjson

---

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: devops-example
  name: devops-example
spec:
  replicas: 1
  selector:
    matchLabels:
      app: devops-example
  strategy: {
    
    }
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: devops-example
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/xiaohh-test/devops-example:v1.0.0
        name: devops-example
        ports:
        - containerPort: 8080
        resources: {
    
    }
      imagePullSecrets:
      # 这里引用上面创建的Secret
      - name: devops-secret
status: {
    
    }

Execute the following command to create this deployment:

kubectl apply -f deploy-devops.yaml

View the creation results:

You can see that a secret and a deployment have been created. Check whether the deployment is completed:

kubectl get deploy,pod -o wide

You can see that the deployment is completed:

When we visit this pod, we can see that the return is successful:

Next we expose the port to external access, also generate a yaml file, and now add --- at the end:

Then generate yaml and splice it at the end of the file:

kubectl expose deployment devops-example --name=devops-example-svc --port=8080 --target-port=8080 --type=NodePort -o yaml --dry-run >> deploy-devops.yaml

As shown in the figure after splicing:

We continue to execute this yaml file:

kubectl apply -f deploy-devops.yaml

You can see the execution results. The secret and deployment have not changed, but a new service has been created:

Take a look at the accessed ports:

kubectl get pod,svc -o wide

You can see that the port is 31337:

We can directly bring the server address of any node and this port to access the application:

Manual deployment is complete

Jenkins automatically completes Kubernetes deployment

Update deployment only needs to execute the following line of commands. The last --record parameter is to record this update so that the version can be rolled back in the future:

kubectl set image deployment devops-example devops-example=registry.cn-hangzhou.aliyuncs.com/xiaohh-test/devops-example:{
    
    新的版本} --record

And the above{新的版本} is the tag (tag) in our git. Next, we can start to generate the script, but before generating the script, we have to do another step for Jenkins. Install a Publish Over SSH plug-in. This plug-in allows us to execute commands remotely. Search and install directly:

Then we restart Jenkins:

# 先停止jenkins这个应用
kill $(jps -l | grep jenkins.war | awk '{print $1}')
# 再等它停止完成之后启动应用
cd /opt/server/jenkins/
nohup java -jar jenkins.war --httpPort=8080 >> /dev/null &

Then we come to 系统管理 -> 系统配置 here:

Scroll down to find Publish over SSH Here, click SSH Servers Add below, we want to add a remote server:

Then click Apply and Save, and then we continue to the pipeline syntax interface to generate pipeline commands:

Select the host and enter the command. Note that the following tags are the tags selected by the user during the referenced build:

Finally generate the script:

Finally we have to add a new Exec On Kubernetes to the Jenkinsfile and copy the generated script into it:

We have completed building the pipeline. Next, let's modify the file and try deploying it. First, we modify the Controller. I will change the returned content:

Then we submit the code:

Then make a new tag:

git tag -a v1.1.0 -m 'Version v1.1.0'
git push origin v1.1.0

View the results and the new tag is created:

We can also see this tag on github:

Then we come to jenkins, select a new tag and start building from scratch:

You can see that the build was successful:

Refresh the access interface and you can see the updated results:

What happens if we click on the v1.0.0 build again? Try this:

You can see that the version will be retired again:

Okay, the automated DevOps platform is set up, get out of class is over!