kubernetes 1.17.2结合ceph13.2.8 实现jenkins部署并用traefik2.1代理

其他 2020-03-15 18:34:09 阅读次数: 0

注：关于ceph、kubernetes集群的部署在此不声明，相信搜到本篇博文，你一定对ceph、kubernetes的部署环节手刃有余。

注：本篇博文牵扯到的技术点有：ceph、kubernetes、harbor、jenkins、traefik

ceph服务器操作

#ceph -s      //查看ceph集群状态
#ceph osd pool create jenkins 128      //创建pool  建议每个pool存放的是通类应用
#ceph auth get-or-create client.jenkins mon 'allow r' osd 'allow class-read, allow rwx pool=jenkins' -o ceph.client.jenkins.keyring    //创建普通用户管理对应pool

注意：ceph集群的状态要先调试成ok.

image-20200315164133224

image-20200315164335421

kubernetes拉取harbor镜像

# cat ~/.docker/config.json |base64 -w 0        //node节点访问私有仓库的认证
ewoJImF1dGhzIjogewoJCSJoYXJib3IubGludXguY29tIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NmVtbHpaV1psYVhwb2RRPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE5LjAzLjUgKGxpbnV4KSIKCX0KfQ==
# cat secret_harbor.yaml 
##########################################################################
#Author:                     zisefeizhu
#QQ:                         2********0
#Date:                       2020-02-19
#FileName:                   secret_harbor.yaml
#URL:                        https://www.cnblogs.com/zisefeizhu/
#Description:                The test script
#Copyright (C):              2020 All rights reserved
###########################################################################
apiVersion: v1
kind: Secret
metadata:
  name: k8s-harbor-login
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: ewoJImF1dGhzIjogewoJCSJoYXJib3IubGludXguY29tIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NmVtbHpaV1psYVhwb2RRPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE5LjAzLjUgKGxpbnV4KSIKCX0KfQ==
# kubectl create -f secret_harbor.yaml 
secret/login created
# kubectl get secret
NAME                          TYPE                                  DATA   AGE
ceph-admin-secret             kubernetes.io/rbd                     1      3d16h
ceph-kube-secret              kubernetes.io/rbd                     1      3d16h

注意：前提是node节点可以访问到harbor，关于这部分可以参考我的这篇博文：https://www.cnblogs.com/zisefeizhu/p/12329864.html

image-20200315162612795

部署动态存储

# pwd
/data/k8s/jenkins      //单个服务单个目录
 
# cat namespace.yaml    //单个服务单个名称空间，便于管理
##########################################################################
#Author:                     zisefeizhu
#QQ:                         2********0
#Date:                       2020-03-09
#FileName:                   namespace.yaml
#URL:                        https://www.cnblogs.com/zisefeizhu/
#Description:                The test script
#Copyright (C):              2020 All rights reserved
###########################################################################
apiVersion: v1
kind: Namespace
metadata:
  name: jenkins
  labels:
    name: jenkins

#kubernetes结合ceph需要使用第三方插件
# cat external-storage-rbd-provisioner.yaml   
##########################################################################
#Author:                     zisefeizhu
#QQ:                         2********0
#Date:                       2020-03-09
#FileName:                   external-storage-rbd-provisioner.yaml
#URL:                        https://www.cnblogs.com/zisefeizhu/
#Description:                The test script
#Copyright (C):              2020 All rights reserved
###########################################################################
apiVersion: v1
kind: ServiceAccount
metadata:
  name: rbd-provisioner
  namespace: jenkins
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: rbd-provisioner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
  - apiGroups: [""]
    resources: ["services"]
    resourceNames: ["kube-dns"]
    verbs: ["list", "get"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: rbd-provisioner
subjects:
  - kind: ServiceAccount
    name: rbd-provisioner
    namespace: jenkins
roleRef:
  kind: ClusterRole
  name: rbd-provisioner
  apiGroup: rbac.authorization.k8s.io

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: rbd-provisioner
  namespace: jenkins
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: rbd-provisioner
  namespace: jenkins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: rbd-provisioner
subjects:
- kind: ServiceAccount
  name: rbd-provisioner
  namespace: jenkins

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rbd-provisioner
  namespace: jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      app: rbd-provisioner
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: rbd-provisioner
    spec:
      containers:
      - name: rbd-provisioner
        image: "harbor.linux.com/rbd/rbd-provisioner:latest"
        imagePullPolicy: IfNotPresent
        env:
        - name: PROVISIONER_NAME
          value: ceph.com/rbd
      imagePullSecrets: 
        - name: k8s-harbor-login
      serviceAccount: rbd-provisioner

#敏感数据创建secret，这没什么可说的
# cat ceph-jenkins-secret.yaml 
##########################################################################
#Author:                     zisefeizhu
#QQ:                         2********0
#Date:                       2020-03-09
#FileName:                   ceph-wordpress-secret.yaml
#URL:                        https://www.cnblogs.com/zisefeizhu/
#Description:                The test script
#Copyright (C):              2020 All rights reserved
###########################################################################
apiVersion: v1
kind: Secret
metadata:
  name: ceph-admin-secret
  namespace: jenkins
data:
  key: QVFBZ2pXVmVGOVJISkJBQTBTUDRoOTVZYVdHNEN6TzNaUWtIdVE9PQ==
type: kubernetes.io/rbd
---
apiVersion: v1
kind: Secret
metadata:
  name: ceph-jenkins-secret
  namespace: jenkins
data:
  key: QVFEUjRHWmVNUFJpRnhBQUQ1Zlg1UG9JRUNkMG85Qk5kVzN5SUE9PQ==
type: kubernetes.io/rbd

# cat ceph-jenkins-storageclass.yaml      //
##########################################################################
#Author:                     zisefeizhu
#QQ:                         2********0
#Date:                       2020-03-09
#FileName:                   ceph-wordpress-storageclass.yaml
#URL:                        https://www.cnblogs.com/zisefeizhu/
#Description:                The test script
#Copyright (C):              2020 All rights reserved
###########################################################################
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ceph-jenkins
  namespace: jenkins
  annotations:
    storageclass.kubernetes.io/is-default-class: "false"
provisioner: ceph.com/rbd
reclaimPolicy: Retain
parameters:
  monitors: 20.0.0.207:6789,20.0.0.208:6789,20.0.0.210:6789
  adminId: admin
  adminSecretName: ceph-admin-secret
  adminSecretNamespace: jenkins
  pool: jenkins
  fsType: xfs
  userId: jenkins
  userSecretName: ceph-jenkins-secret
  imageFormat: "2"
  imageFeatures: "layering"
  
# cat jenkins-pvc.yaml 
##########################################################################
#Author:                     zisefeizhu
#QQ:                         2********0
#Date:                       2020-03-10
#FileName:                   jenkins-pvc.yaml
#URL:                        https://www.cnblogs.com/zisefeizhu/
#Description:                The test script
#Copyright (C):              2020 All rights reserved
###########################################################################
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-pvc
  namespace: jenkins
 # labels:
 #   app: gitlab
spec:
  storageClassName: ceph-jenkins
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi  
# kubectl get pvc -n jenkins
NAME          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
jenkins-pvc   Bound    pvc-d386c125-5302-468a-8a94-a2570f0a4ca0   20Gi       RWO            ceph-jenkins   2d8h

部署jenkins应用

# pwd
/data/k8s/jenkins
 
# cat jenkins.yaml     //核心资源清单
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: jenkins
  namespace: jenkins
  labels:
    name: jenkins
spec:
  selector:
    matchLabels:
      name: jenkins
  serviceName: jenkins
  replicas: 1
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      name: jenkins
      labels:
        name: jenkins
    spec:
      terminationGracePeriodSeconds: 10
      serviceAccountName: jenkins
      #登陆私有仓库harbor认证
      imagePullSecrets: 
        - name: k8s-harbor-login
      containers:
        - name: jenkins
          image: harbor.linux.com/dev/jenkins:lts
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
            - containerPort: 50000
          resources:
            limits:
              cpu: 1
              memory: 1Gi
            requests:
              cpu: 0.5
              memory: 500Mi
          env:
            - name: LIMITS_MEMORY
              valueFrom:
                resourceFieldRef:
                  resource: limits.memory
                  divisor: 1Mi
            - name: JAVA_OPTS
              # value: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
              value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/jenkins_home
          livenessProbe:
            httpGet:
              path: /login
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 5
            failureThreshold: 12 # ~2 minutes
          readinessProbe:
            httpGet:
              path: /login
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 5
            failureThreshold: 12 # ~2 minutes
      securityContext:
        fsGroup: 1000
      volumes:
      - name: jenkins-home
        persistentVolumeClaim:
          claimName: jenkins-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: jenkins
  namespace: jenkins
spec:
  # type: LoadBalancer
  selector:
    name: jenkins
  # ensure the client ip is propagated to avoid the invalid crumb issue when using LoadBalancer (k8s >=1.7)
  #externalTrafficPolicy: Local
  ports:
    -
      name: http
      port: 80     
      targetPort: 8080
      protocol: TCP
    -
      name: agent
      port: 50000
      protocol: TCP

# cat jenkins-serviceaccount.yaml 
##########################################################################
#Author:                     zisefeizhu
#QQ:                         2********0
#Date:                       2020-03-10
#FileName:                   jenkins-serviceaccount.yaml
#URL:                        https://www.cnblogs.com/zisefeizhu/
#Description:                The test script
#Copyright (C):              2020 All rights reserved
###########################################################################
# In GKE need to get RBAC permissions first with
# kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin [--user=<user-name>|--group=<group-name>]

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
  namespace: jenkins
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins
  namespace: jenkins
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkins
  namespace: jenkins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: jenkins
# kubectl get pods -n jenkins
NAME                               READY   STATUS        RESTARTS   AGE
jenkins-0                          1/1     Running       14         30h
rbd-provisioner-5c97b9d5ff-95qwj   1/1     Running       13         2d8h

部署代理

# cat jenkins-ingressroute.yaml 
##########################################################################
#Author:                     zisefeizhu
#QQ:                         2********0
#Date:                       2020-03-10
#FileName:                   jenkins-ingressroute.yaml
#URL:                        https://www.cnblogs.com/zisefeizhu/
#Description:                The test script
#Copyright (C):              2020 All rights reserved
###########################################################################
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: jenkins
  namespace: jenkins
spec:
  entryPoints:
    - web
  routes:
  #登陆域名   需要自己在主机添加hosts解析  或者自建dns也行
  - match: Host(`jenkins.linux.com`)
    kind: Rule
    services:
    - name: jenkins    #和jenkins核心资源清单一致，  name是jenkins service的name
      port: 80         #jenkins pod 暴漏端口

image-20200315180713459

登陆密码

# kubectl get pods -n jenkins
NAME                              READY   STATUS    RESTARTS   AGE
jenkins-0                         1/1     Running   1          18m
rbd-provisioner-dbc4c8b59-grfg2   1/1     Running   1          125m
# kubectl logs jenkins-0  -n jenkins
VM settings:
    Max. Heap Size: 1.00G
    Ergonomics Machine Class: server
    Using VM: OpenJDK 64-Bit Server VM

#注意 jenkins首次登陆密码
ac4fe3940ec145fe9104eda3ca390d0a    

This may also be found at: /var/jenkins_home/secrets/initialAdminPassword

*************************************************************
*************************************************************
*************************************************************

2020-03-10 02:44:33.446+0000 [id=39]    INFO    hudson.model.UpdateSite#updateData: Obtained the latest update center data file for UpdateSource default
2020-03-10 02:44:36.699+0000 [id=25]    INFO    hudson.model.UpdateSite#updateData: Obtained the latest update center data file for UpdateSource default
2020-03-10 02:44:42.206+0000 [id=25]    INFO    jenkins.InitReactorRunner$1#onAttained: Completed initialization
2020-03-10 02:44:43.044+0000 [id=39]    INFO    h.m.DownloadService$Downloadable#load: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller
2020-03-10 02:44:43.044+0000 [id=39]    INFO    hudson.util.Retrier#start: Performed the action check updates server successfully at the attempt #1
2020-03-10 02:44:43.055+0000 [id=39]    INFO    hudson.model.AsyncPeriodicWork#lambda$doRun$0: Finished Download metadata. 81,778 ms
2020-03-10 02:44:43.414+0000 [id=19]    INFO    hudson.WebAppMain$3#run: Jenkins is fully up and running

登陆密码为：

ac4fe3940ec145fe9104eda3ca390d0a

image-20200315180746691

注：本篇博文完全原创，后续将发布大量有技术的原创博文，请持续关注。
注：关于ceph集群的部署、kubernetes1.17.2高可用集群的部署可以看我的历史博文。
注：关于jenkins的使用不在本篇讲解范围内(主要是截图太多了，有空再发表吧)

猜你喜欢

转载自www.cnblogs.com/zisefeizhu/p/12499084.html
今日推荐
基于大语言模型的开源知识库问答系统 MaxKB GitHub Star 数量突破 5,000 个！
美国拟限制 AI 大模型出口中国和俄罗斯
苹果将与 OpenAI 达成协议，将 ChatGPT 应用于 iPhone
openKylin 社区生态委员会第六次会议圆满召开
阿里云正式发布通义千问 2.5
Python 3.13 发布首个 Beta：实验性自由线程模式和 JIT、改进交互式解释器
Stack Overflow 拿我的代码去训练 AI 大模型，还封了我的账号
Pop!_OS 的 COSMIC 桌面完成 App Store 上架工作
《2024 年一季度互联网投融资运行情况》研究报告
报告：Django 仍然是 74% 开发者的首选
15 年前上了“FFmpeg 耻辱柱”，今天他还得谢谢咱——腾讯QQPlayer一雪前耻？
TIOBE 5 月榜单：Fortran “复活”进入 Top 10
周排行
记一下去大梅沙的准备（2018-05-26）
Spring 注解 事务
基于HTTP协议的客户端缓存
阿里云rds 备份和还原
[PHP] 几个拖慢 PHP 程序/API 运行速度的点
python 代码风格------------PEP8规则
js控制json生成菜单——自制菜单（一）
将字符串: 'k:1|k1:2|k2:3|k3:4 ' ,处理成 python 字典: {'k':1, 'k1':2, ...}
微信小程序转支付宝小程序
Qt551.窗口滚动条
每日归档
更多
2024-05-13(18)
2024-05-12(0)
2024-05-11(38)
2024-05-10(38)
2024-05-09(35)
2024-05-08(42)
2024-05-07(14)
2024-05-06(40)
2024-05-05(0)
2024-05-04(7)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022