阿里云K8S服务支持CSI存储卷

编程语言 2018-08-08 10:47:05 阅读次数: 0

CSI插件简介

CSI是Container Storage Interface的简称，为容器编排系统和存储系统之间建立一套标准的存储调用接口。

Kubernetes支持CSI接口标准，并在1.10版本进入beta阶段。K8S除了支持CSI方式提供存储接口外，还有In-Tree方式、Flexvolume方式提供的各种存储接口。

In-Tree方式是嵌入在K8S源码内部的存储挂载实现，存在以下问题：

存储插件需要一同随K8S发布。
存储插件的问题有可能会影响K8S部件正常运行。
存储插件享有K8S部件同等的特权存在安全隐患。
存储插件开发者必须遵循K8S社区的规则开发代码。

FlexVolume机制通过调用一个可执行文件方式去实现挂载，它能够做到让存储提供方进行独立开发维护。

部署可执行文件时，需要host的root权限，依然存在安全隐患。
存储插件在执行mount、attach操作时，往往需要到host去安装第三方工具或加载依赖库，这样会使部署变得复杂。

CSI标准的出现解决了上述问题，后续K8S与存储系统的集成标准将全面支持CSI。

阿里云存储支持CSI Plugin

目前阿里云CSI存储插件支持云盘、OSS、NAS等存储类型，并支持通过StorageClass创建动态存储卷；

开源地址：https://github.com/AliyunContainerService/csi-plugin

插件的部署拓扑如下：
csi-disk.png

使用演示

云盘CSI Plugin部署

环境准备

  1. 通过阿里云容器服务创建K8S集群（1.10版本及以上），或者自己通过阿里云ECS服务器搭建K8S集群。
  2. 配置Kubelet启动参数：--enable-controller-attach-detach=true；

部署 CSI Attacher

# kubectl create -f attacher.yaml

CSI attacher使用0.2.0版本，attacher.yaml内容如下：

# This YAML file contains RBAC API objects,
# which are necessary to run external csi attacher for cinder.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: csi-attacher

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: external-attacher-runner
rules:
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["volumeattachments"]
    verbs: ["get", "list", "watch", "update"]

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: csi-attacher-role
subjects:
  - kind: ServiceAccount
    name: csi-attacher
    namespace: default
roleRef:
  kind: ClusterRole
  name: external-attacher-runner
  apiGroup: rbac.authorization.k8s.io
---

kind: Service
apiVersion: v1
metadata:
  name: csi-attacher
  labels:
    app: csi-attacher
spec:
  selector:
    app: csi-attacher
  ports:
    - name: dummy
      port: 12345

---
kind: StatefulSet
apiVersion: apps/v1beta1
metadata:
  name: csi-attacher
spec:
  serviceName: "csi-attacher"
  replicas: 1
  template:
    metadata:
      labels:
        app: csi-attacher
    spec:
      tolerations:
      - effect: NoSchedule
        operator: Exists
        key: node-role.kubernetes.io/master
      - effect: NoSchedule
        operator: Exists
        key: node.cloudprovider.kubernetes.io/uninitialized
      nodeSelector:
         node-role.kubernetes.io/master: ""
      serviceAccount: csi-attacher
      containers:
        - name: csi-attacher
          image: registry.cn-hangzhou.aliyuncs.com/plugins/csi-attacher:v0.2.0
          args:
            - "--v=5"
            - "--csi-address=$(ADDRESS)"
          env:
            - name: ADDRESS
              value: /var/lib/kubelet/plugins/csi-diskplugin/csi.sock
          imagePullPolicy: "IfNotPresent"
          volumeMounts:
            - name: socket-dir
              mountPath: /var/lib/kubelet/plugins/csi-diskplugin
      volumes:
        - name: socket-dir
          hostPath:
            path: /var/lib/kubelet/plugins/csi-diskplugin
            type: DirectoryOrCreate

部署CSI Provsioner

# kubectl create -f provisioner.yaml

CSI provisioner使用0.2.0版本，内容如下：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: csi-provisioner

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: external-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "list"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["list", "watch", "create", "update", "patch"]

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: csi-provisioner-role
subjects:
  - kind: ServiceAccount
    name: csi-provisioner
    namespace: default
roleRef:
  kind: ClusterRole
  name: external-provisioner-runner
  apiGroup: rbac.authorization.k8s.io

---
kind: Service
apiVersion: v1
metadata:
  name: csi-provisioner
  labels:
    app: csi-provisioner
spec:
  selector:
    app: csi-provisioner
  ports:
    - name: dummy
      port: 12345

---
kind: StatefulSet
apiVersion: apps/v1beta1
metadata:
  name: csi-provisioner
spec:
  serviceName: "csi-provisioner"
  replicas: 1
  template:
    metadata:
      labels:
        app: csi-provisioner
    spec:
      tolerations:
      - effect: NoSchedule
        operator: Exists
        key: node-role.kubernetes.io/master
      - effect: NoSchedule
        operator: Exists
        key: node.cloudprovider.kubernetes.io/uninitialized
      nodeSelector:
         node-role.kubernetes.io/master: ""
      serviceAccount: csi-provisioner
      containers:
        - name: csi-provisioner
          image: registry.cn-hangzhou.aliyuncs.com/plugins/csi-provisioner:v0.2.0
          args:
            - "--provisioner=csi-diskplugin"
            - "--csi-address=$(ADDRESS)"
            - "--v=5"
          env:
            - name: ADDRESS
              value: /var/lib/kubelet/plugins/csi-diskplugin/csi.sock
          imagePullPolicy: "IfNotPresent"
          volumeMounts:
            - name: socket-dir
              mountPath: /var/lib/kubelet/plugins/csi-diskplugin
      volumes:
        - name: socket-dir
          hostPath:
            path: /var/lib/kubelet/plugins/csi-diskplugin
            type: DirectoryOrCreate

部署Disk Plugin

Disk Plugin部署为DaemonSet应用，命令：

# kubectl create -f plugin.yaml

CSI driver-registrar使用0.2.0版本；

Socket地址设置为：/var/lib/kubelet/plugins/csi-diskplugin/csi.sock；

apiVersion: v1
kind: ServiceAccount
metadata:
  name: csi-diskplugin

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: csi-diskplugin
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "update"]
  - apiGroups: [""]
    resources: ["namespaces"]
    verbs: ["get", "list"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["volumeattachments"]
    verbs: ["get", "list", "watch", "update"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: csi-diskplugin
subjects:
  - kind: ServiceAccount
    name: csi-diskplugin
    namespace: default
roleRef:
  kind: ClusterRole
  name: csi-diskplugin
  apiGroup: rbac.authorization.k8s.io

---
# This YAML file contains driver-registrar & csi driver nodeplugin API objects,
# which are necessary to run csi nodeplugin for disk.

kind: DaemonSet
apiVersion: apps/v1beta2
metadata:
  name: csi-diskplugin
spec:
  selector:
    matchLabels:
      app: csi-diskplugin
  template:
    metadata:
      labels:
        app: csi-diskplugin
    spec:
      tolerations:
        - key: node-role.kubernetes.io/master
          operator: Exists
          effect: NoSchedule
      serviceAccount: csi-diskplugin
      hostNetwork: true
      hostPID: true
      containers:
        - name: driver-registrar
          image: registry.cn-hangzhou.aliyuncs.com/plugins/driver-registrar:v0.2.0
          args:
            - "--v=5"
            - "--csi-address=$(ADDRESS)"
          env:
            - name: ADDRESS
              value: /var/lib/kubelet/plugins/csi-diskplugin/csi.sock
            - name: KUBE_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          volumeMounts:
            - name: socket-dir
              mountPath: /var/lib/kubelet/plugins/csi-diskplugin
        - name: csi-diskplugin
          securityContext:
            privileged: true
            capabilities:
              add: ["SYS_ADMIN"]
            allowPrivilegeEscalation: true
          image: registry.cn-hangzhou.aliyuncs.com/plugins/csi-diskplugin:v1.10-7424d08
          args :
            - "--nodeid=$(NODE_ID)"
            - "--endpoint=$(CSI_ENDPOINT)"
            - "--v=5"
          env:
            - name: NODE_ID
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: CSI_ENDPOINT
              value: unix://var/lib/kubelet/plugins/csi-diskplugin/csi.sock
          imagePullPolicy: "IfNotPresent"
          volumeMounts:
            - name: plugin-dir
              mountPath: /var/lib/kubelet/plugins/csi-diskplugin
            - name: pods-mount-dir
              mountPath: /var/lib/kubelet/pods
              mountPropagation: "Bidirectional"
            - mountPath: /dev
              name: host-dev
            - mountPath: /sys
              name: host-sys
            - mountPath: /lib/modules
              name: lib-modules
              readOnly: true
            - mountPath: /run/dbus
              name: host-run
              readOnly: true
            - mountPath: /var/log/alicloud
              name: host-log
      volumes:
        - name: plugin-dir
          hostPath:
            path: /var/lib/kubelet/plugins/csi-diskplugin
            type: DirectoryOrCreate
        - name: pods-mount-dir
          hostPath:
            path: /var/lib/kubelet/pods
            type: Directory
        - name: socket-dir
          hostPath:
            path: /var/lib/kubelet/plugins/csi-diskplugin
            type: DirectoryOrCreate
        - name: host-dev
          hostPath:
            path: /dev
        - name: host-run
          hostPath:
            path: /run/dbus
        - name: host-sys
          hostPath:
            path: /sys
        - name: lib-modules
          hostPath:
            path: /lib/modules
        - name: host-log
          hostPath:
            path: /var/log/alicloud/

部署检查

# kubectl get pod

NAME                                 READY     STATUS    RESTARTS   AGE
csi-attacher-0                       1/1       Running   0          1d
csi-diskplugin-5w7wz                 2/2       Running   0          1d
csi-diskplugin-j99h9                 2/2       Running   0          1d
csi-diskplugin-ljg9s                 2/2       Running   0          1d
csi-diskplugin-nhbfp                 2/2       Running   0          1d
csi-provisioner-0                    1/1       Running   0          1d

使用CSI插件挂载云盘

下面以创建动态云盘数据卷为例，介绍如何使用CSI插件挂载；

创建云盘StorageClass

定义一个SSD、ReadWrite、北京B区的StorageClass；

# kubectl create -f storageclass.yaml

storageclass.yaml内容如下：

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
   name: csi-disk
provisioner: csi-diskplugin
parameters:
    zoneId: cn-beijing-b
    regionId: cn-beijing
    fsType: ext4
    type: cloud_ssd
    readOnly: "false"
reclaimPolicy: Delete

创建应用

创建Nginx服务并通过PVC挂载云盘，在PVC配置csi-disk storageClassName；

# kubectl create -f nginx.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: disk-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 25Gi
  storageClassName: csi-disk
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment1
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
        volumeMounts:
          - name: disk-pvc
            mountPath: "/data"
      volumes:
        - name: disk-pvc
          persistentVolumeClaim:
            claimName: disk-pvc

云盘挂载验证

查看nginx容器：

# kubectl get pod | grep nginx
# nginx-deployment1-5879d9db88-wr9j9   1/1       Running   0          1d

查看容器内挂载信息：

# kubectl exec -ti nginx-deployment1-5879d9db88-wr9j9 df | grep data
/dev/vdd        25671908   45084  24299720   1% /data
# kubectl exec -ti nginx-deployment1-5879d9db88-wr9j9 touch /data/aliyun
# kubectl exec -ti nginx-deployment1-5879d9db88-wr9j9 ls /data
aliyun    lost+found

删除容器，验证云盘高可用：

# kubectl delete pod nginx-deployment1-5879d9db88-wr9j9
# kubectl get pod
NAME                                 READY     STATUS    RESTARTS   AGE
nginx-deployment1-5879d9db88-mjb5p   1/1       Running   0          14s
# kubectl exec -ti nginx-deployment1-5879d9db88-mjb5p ls /data
aliyun    lost+found

上述操作通过CSI插件成功挂载阿里云云盘，并验证了应用数据的高可用性；

猜你喜欢

转载自yq.aliyun.com/articles/623259
今日推荐
面壁智能发布 Eurux-8x22B 开源大模型 —— 堪称「理科状元」
开源日报 | 谷歌扶持鸿蒙上位；开源Rabbit R1；Docker加持的安卓手机；微软的焦虑和野心；海尔电器把开放平台关了
中国码农的“35岁魔咒”
蘭雅 CorelDRAW 插件 2024.5.1 国际劳动节版，免费下载
Arc Browser for Windows 1.0 正式 GA
90后程序员开发视频搬运软件、不到一年获利超 700 万，结局很刑！
周排行
【转】spring中对控制反转和依赖注入的理解
tms webcore 安装和使用
java程序员进阶相关书籍
SpringMVC接受请求参数、
如何保存训练好的机器学习模型
MyEclipse、Eclipse设置项目JDK的三个地方
商超行业微信小程序开发定制一般多少钱 （行业技术人员解读）
Markdown编辑器语言——30分钟入门到到精通
Linux系统下MongoDB的简单安装与基本操作
Power Strings
每日归档
更多
2024-05-07(14)
2024-05-06(40)
2024-05-05(0)
2024-05-04(7)
2024-05-03(19)
2024-05-02(0)
2024-05-01(4)
2024-04-30(1)
2024-04-29(40)
2024-04-28(0)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022