ジェンキンスさんはK8Sアプリケーションをリリース

開発が完了し、コンソールがまだないので、私たちのK8Sアプリケーションは、現在だけジェンキンスに直接実装されて公開します。

1.基本イメージ

階層的な生産拠点の画像

CentOSの - > CentOSの-sshdの - > centos74-JRE [678] - > centos74-JRE [678] - [tomcat6 | tomcat7 | tomcat8 | springboot]

あなたがアプリケーションを公開すると、JRE \ミドルウェアパラメータに応じて、基本イメージを選択

2.リソースアプリケーションパラメータ説明

必須の

事業	プロジェクトの名前、純粋なデジタル「P」を追加する最初の
アプリ	アプリケーション名、接頭辞としてプロジェクト名は、K8Sクラスタ内のアプリは、アプリケーションの識別子として、ユニークです
ドメイン	ドメイン名、塗りなし「NULL」
sessionSticky	nginxのip_hashに似て、セッションを保持し、「true」または「false」
複製	あるコピーの数、ノード数
javaVersion	JREのバージョン、サポートJRE6 \ jre7 \ jre8
ミドルウェア	ミドルウェア、サポートtomcat6 \ tomcat7 \ tomcat8 \ springboot
rwFileList	Tomcatのプロジェクトファイルの書き込み権限のために、複数のカンマ、塗りなし「NULL」で区切られました
コードレポ	gitのアドレス
warDir	戦争は、このような「ターゲット」として、相対パスをパッケージ化
mvnCommand	MVN packageコマンド
sbStartCommand	Springbootプロジェクト開始文書コマンド指定されたプロファイル
dingToken	dingdingグループは、「NULL」せずに、トークンを与えるの通知を送信する必要があります

オプション（O＆Mは、デフォルトの構成を有しています）

cpuReq	核の数のCPU初期割当て
cpuLmt	CPUは、核の最大数を割り当てられました
MEMREQ	メモリの初期割り当てサイズ（M）
memLmt	メモリ割り当ての制限サイズ（M）、および一般的に同じMEMREQ
jvmSize	JVMのサイズ、デフォルトの「ヌル」、式自動的に計算MEMREQ
directMemSize	外部メモリの最大サイズは、デフォルト「NULL」、式自動的に計算MEMREQ
mvnVersion	MVNバージョン
mvnJavaVersion	Mvn环境的jdk版本
imageVersion	基础镜像版本（研发不需了解）

3. jenkins pipeline

所有变量抽出至environment下，便于配置，也便于后续自研平台调用；

整个过程分为checkout、mvn、build、deploy、post action几个部分；

checkout：拉取代码

mvn：调用mvn命令打包

build：执行make_dockerfile.sh，生成dockerfile，调用docker命令build镜像、push镜像

deploy：执行make_deploymentfile.sh，make_ingressfile.sh，调用kubectl命令创建deployment、ingress（含service）

post action：发送钉钉通知完整jenkinsfile如下

pipeline {
    environment {
        project = 'myproj'
        app = 'myproc-myapp'
        domain = 'mydomain.ali.com'
        replicate = '4'
        sessionSticky = 'false'
        javaVersion = 'jre8'
        middleware = 'springboot'
        rwFileList = 'null'
        codeRepo = 'http://git.mydomain.com/usergroup/myapp.git'
        warDir = 'target'
        mvnCommand = 'clean install -DskipTests=true -Ptest'
        sbStartCommand = '-Dspring.profiles.active=test'
        dingToken = '8826e85c02c796aa5aea0a39d292e1111111111111111111111'
        
        cpuReq = '0.2'
        cpuLmt = '2'
        memReq = '1024'
        memLmt = '1024'
        jvmSize = 'null'
        directMemSize = 'null'
        mvnVersion = 'maven-3.6.0'
        mvnJavaVersion = 'jdk1.8.0_221'
        imageVersion = 'v1'

    }
    
    agent any
    
    options {
        timeout(time: 10, unit: 'MINUTES') 
    }
    
    stages {
        
        stage('checkout') {
            steps {
                script {
                    def codeRepoCred
                    def reg1 = /.+git\.mydomain\.com.+/
                    if ("${codeRepo}".matches(reg1)) {
                        codeRepoCred = 'mycred'
                    }
                    checkout([$class: 'GitSCM', branches: [[name: '$tag']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: "${codeRepoCred}", url: "${codeRepo}"]]])
                }
            }
        }
        stage('mvn') {
            environment {
                PATH = "/usr/local/java/${mvnJavaVersion}/bin:$PATH"
            }
            steps {
                sh "/usr/local/maven/${mvnVersion}/bin/mvn -f pom.xml ${mvnCommand}"
            }
        }
        stage('build') {
            steps {
                sh "/data1/template/make_dockerfile.sh ${WORKSPACE} ${javaVersion} ${middleware} ${cpuLmt} ${memLmt} ${rwFileList} ${imageVersion} ${sbStartCommand} ${jvmSize} ${directMemSize}"
                script {
                    tagnew = "${tag}".replaceAll("/","_");
                    sh "/usr/bin/docker build -f ${WORKSPACE}/deploy/dockerfile -t harbor.mydomain.com/$project/${app}:${tagnew} ${WORKSPACE}/${warDir}"
                    sh "/usr/bin/docker push harbor.mydomain.com/$project/${app}:${tagnew}"
                    sh "/usr/bin/docker rmi harbor.mydomain.com/$project/${app}:${tagnew}"
                }
            }
        }
        stage('deploy') {
            steps {
                script {
                    tagnew = "${tag}".replaceAll("/","_");
                    sh "/data1/template/make_deploymentfile.sh ${WORKSPACE} ${app} ${project} ${replicate} ${BUILD_NUMBER} ${cpuReq} ${cpuLmt} ${memReq} ${memLmt} ${tagnew}"
                    sh "/usr/bin/kubectl apply -f ${WORKSPACE}/deploy/deployment.yaml"
                    sh "/usr/bin/kubectl delete ingress ${app} > /dev/null 2>&1 || true"
                    sh "/usr/bin/kubectl delete service ${app} > /dev/null 2>&1 || true"
                    if ("${domain}" != "null") {
                        sh "/data1/template/make_ingressfile.sh ${WORKSPACE} ${app} ${domain} ${sessionSticky}"
                        sh "/usr/bin/kubectl apply -f ${WORKSPACE}/deploy/ingress.yaml"
                    }
                    sh "/usr/bin/kubectl rollout status deployment/${app}"
                }

            }
        }
    }
    post {
		success {
			dingTalk accessToken:"https://oapi.dingtalk.com/robot/send?access_token=${dingToken}", 
			imageUrl:'', 
			jenkinsUrl:'http://10.40.16.212/', 
			message:' deploy success', 
			notifyPeople:''
		}
		failure {
			dingTalk accessToken:'https://oapi.dingtalk.com/robot/send?access_token=${dingToken}', 
			imageUrl:'', 
			jenkinsUrl:'http://10.40.16.212/', 
			message:' deploy failed', 
			notifyPeople:''
		}
    }
}

4. shell与模板文件

jenkins服务器上的模板相关文件：*.template

make_*.sh传递pipeline中的变量，根据template文件生成各个项目特定的dockerfile、deployment.yaml、ingress.yaml

dockerfile.template

将应用启动需要的参数，输出至用户.bash_profile中，生成环境变量；后续脚本直接调用环境变量

FROM harbor.mydomain.com/baseimage/imageTag
LABEL type="appimage"
COPY --chown=appuser:appuser *.war /apps/deploy/
RUN echo "export CPU_LIMIT=cpuLmtTag" >> /home/appuser/.bash_profile \
    && echo "export MEM_LIMIT=memLmtTag" >> /home/appuser/.bash_profile \
    && echo "export JVM_SIZE=jvmSizeTag" >> /home/appuser/.bash_profile \
    && echo "export DIRECT_MEM_SIZE=directMemSizeTag" >> /home/appuser/.bash_profile \
    && echo "export PERM_NAME=permNameTag" >> /home/appuser/.bash_profile \
    && echo "export FILE_LIST=rwFileListTag" >> /home/appuser/.bash_profile \
    && echo "export SB_START_COMMAND=sbStartCommandTag" >> /home/appuser/.bash_profile \
    && echo -e "LIBSYSCONFCPUS=\${CPU_LIMIT}" >> /home/appuser/.bash_profile \                
    && echo -e "if [ \${LIBSYSCONFCPUS} -lt 2 ]; then LIBSYSCONFCPUS=2; fi" >> /home/appuser/.bash_profile \
    && echo "export LIBSYSCONFCPUS" >> /home/appuser/.bash_profile \      
    && echo -e "export LD_PRELOAD=/usr/local/lib/libsysconfcpus.so:\${LD_PRELOAD}" >> /home/appuser/.bash_profile
CMD ["/apps/script/pod_start.sh"]

deployment.template

apiVersion: apps/v1
kind: Deployment
metadata:
  name: appTag
  labels:
    app: appTag
spec:
  replicas: replicateTag
  selector:
    matchLabels:
      app: appTag
  template:
    metadata:
      labels:
        app: appTag
        project: projectTag
        buildnum: "buildnumTag"
    spec:
      dnsPolicy: Default
      containers:
      - name: appTag
        image: harbor.mydomain.com/projectTag/appTag:tagTag
        imagePullPolicy: Always
        ports:
        - containerPort: 9999
        securityContext:
          privileged: false
        resources:
          requests:
            cpu: cpuReqTag
            memory: memReqTagMi
            ephemeral-storage: 2Gi
          limits:
            cpu: cpuLmtTag
            memory: memLmtTagMi
            ephemeral-storage: 5Gi
        readinessProbe:
          #httpGet:
          #  path: /healthProbe.jsp
          tcpSocket:
            port: 9114
          initialDelaySeconds: 30
          periodSeconds: 3
        livenessProbe:
          #httpGet:
          #  path: /healthProbe.jsp
          tcpSocket:
            port: 9114
          initialDelaySeconds: 120
          periodSeconds: 3
        lifecycle:
          preStop:
            exec:
              command: ["/apps/script/pod_pre_stop.sh"]
        volumeMounts:
        - mountPath: /apps/heapdump
          name: dump-vol
      volumes:
      - name: dump-vol
        hostPath:
          path: /var/lib/docker/dumpvol
          type: Directory

ingress.template

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: appTag
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: domainTag
    http:
      paths:
      - path: /
        backend:
          serviceName: appTag
          servicePort: http
---
kind: Service
apiVersion: v1
metadata:
  name: appTag
  annotations:
    traefik.ingress.kubernetes.io/affinity: "sessionStickyTag"
    traefik.ingress.kubernetes.io/session-cookie-name: "lbcookie"
spec:
  clusterIP: None
  selector:
    app: appTag
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 9999

make_dockerfile.sh

#!/bin/bash

workspace=$1
jre=$2
middleware=$3
cpulmt=$4
memlmt=$5
rwfilelist=$6
imageVersion=$7
sbstartCommand=$8
jvmSize=$9
directMemSize=${10}

if [ $# -ne 10 ]; then
    echo "Usage: $0 [workspace] [jre] [middlewre] [cpulmt] [memlmt] [rwfilelist] [imageVersion] [sbstartCommand] [jvmSize] [directMemSize]"
    exit 1
fi

if [ $9 == "null" ]; then
    if [ $memlmt -le 3072 ]; then
        jvmSize=`expr $memlmt / 2`
    else
        jvmSize=`expr $memlmt - 1500`
    fi
fi

if [ ${10} == "null" ]; then
    if [ $memlmt -le 2048 ]; then
        directMemSize=256
    else
        directMemSize=512
    fi
fi

if [ $jre == "jre6" || $jre == "jre7" ]; then
    permName="PermSize"
else
    permName="MetaspaceSize"
fi

image="centos74-"${jre}"-"${middleware}":"${imageVersion}

mydockerfile=$workspace/deploy/dockerfile
mkdir -p $workspace/deploy
cp /data1/template/dockerfile.template $mydockerfile

/usr/bin/sed -i "s/imageTag/${image}/g" $mydockerfile
/usr/bin/sed -i "s/cpuLmtTag/${cpulmt}/g" $mydockerfile
/usr/bin/sed -i "s/memLmtTag/${memlmt}/g" $mydockerfile
/usr/bin/sed -i "s/rwFileListTag/${rwfilelist}/g" $mydockerfile
/usr/bin/sed -i "s/sbStartCommandTag/${sbstartCommand}/g" $mydockerfile
/usr/bin/sed -i "s/jvmSizeTag/${jvmSize}/g" $mydockerfile
/usr/bin/sed -i "s/directMemSizeTag/${directMemSize}/g" $mydockerfile
/usr/bin/sed -i "s/permNameTag/${permName}/g" $mydockerfile

make_deploymentfile.sh

#!/bin/bash

workspace=$1
app=$2
project=$3
replicate=$4
buildnum=$5
cpuReq=$6
cpuLmt=$7
memReq=$8
memLmt=$9
tag=${10}

if [ $# -ne 10 ]; then
    echo "Usage: $0 [workspace] [app] [project] [replicate] [buildnum] [cpuReq] [cpuLmt] [memReq] [memLmt] [tag]"
    exit 1
fi

mydeploymentfile=$workspace/deploy/deployment.yaml
cp /data1/template/deployment.template $mydeploymentfile

/usr/bin/sed -i "s/appTag/${app}/g" $mydeploymentfile
/usr/bin/sed -i "s/projectTag/${project}/g" $mydeploymentfile
/usr/bin/sed -i "s/replicateTag/${replicate}/g" $mydeploymentfile
/usr/bin/sed -i "s/buildnumTag/${buildnum}/g" $mydeploymentfile
/usr/bin/sed -i "s/cpuReqTag/${cpuReq}/g" $mydeploymentfile
/usr/bin/sed -i "s/cpuLmtTag/${cpuLmt}/g" $mydeploymentfile
/usr/bin/sed -i "s/memReqTag/${memReq}/g" $mydeploymentfile
/usr/bin/sed -i "s/memLmtTag/${memLmt}/g" $mydeploymentfile
/usr/bin/sed -i "s/tagTag/${tag}/g" $mydeploymentfile

make_ingressfile.sh

#!/bin/bash

workspace=$1
app=$2
domain=$3
sessionSticky=$4

if [ $# -ne 4 ]; then
    echo "Usage: $0 [workspace] [app] [domain] [sessionSticky]"
    exit 1
fi

myingressfile=$workspace/deploy/ingress.yaml
cp /data1/template/ingress.template $myingressfile

/usr/bin/sed -i "s/appTag/${app}/g" $myingressfile
/usr/bin/sed -i "s/domainTag/${domain}/g" $myingressfile
/usr/bin/sed -i "s/sessionStickyTag/${sessionSticky}/g" $myingressfile

sdmei

发布了24 篇原创文章 · 获赞 25 · 访问量 2万+

私信关注