官方文档传送门 -> 持久卷 https://kubernetes.io/zh/docs/concepts/storage/persistent-volumes/
- PV和PVC
- 访问模式
- 回收策略
- PVC绑定PV
@PV和PVC
持久卷 PersistentVolume, PV 是集群中的资源
可以由管理员事先供应(静态供应)，或者使用存储类(Storage Class)动态供应
持久卷申领 PersistentVolumeClaim, PVC 是对这些资源的请求
概念上与 Pod 类似，Pod 会耗用节点资源，而 PVC申领会耗用 PV资源。Pod 可以请求特定数量的资源(CPU和内存); PVC申领也可以请求特定的大小和访问模式。
@访问模式
ReadWriteOnce -- 卷可以被一个节点以读写方式挂载；
ReadOnlyMany -- 卷可以被多个节点以只读方式挂载；
ReadWriteMany -- 卷可以被多个节点以读写方式挂载。
在命令行接口（CLI）中，访问模式也使用以下缩写形式：
RWO - ReadWriteOnce
ROX - ReadOnlyMany
RWX - ReadWriteMany
注意：每个卷只能同一时刻只能以一种访问模式挂载，即使该卷能够支持多种访问模式。
@回收策略
保留 Retain -- 删除PV对象。与之相关的、位于外部基础设施中的存储资产仍然存在。
删除 Delete -- 删除PV对象。删除与之相关的、位于外部基础设施中的存储资产。
回收 Recycle -- 基本擦除 (rm -rf /thevolume/*)。目前仅NFS和HostPath支持回收(Recycle)。已被废弃，取而代之的建议方案是使用动态供应。
@PVC绑定PV
如下yaml文件定义了Deployment,PVC和SVC
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-pv
spec:
selector:
matchLabels:
app: test-pv
replicas: 3
template:
metadata:
labels:
app: test-pv
spec:
containers:
- name: nginx
image: nginx
volumeMounts:
- name: pv1
mountPath: /usr/share/nginx/html
volumes:
- name: pv1
persistentVolumeClaim:
claimName: test-pvc
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test-pvc
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5Gi
---
apiVersion: v1
kind: Service
metadata:
name: web-pv
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: test-pv这个时候还没有PV，所以Pod一直处于Pending状态
[root@k8s-master ~]# kubectl apply -f test-deploy-pvc.yaml
deployment.apps/web-pv created
persistentvolumeclaim/test-pvc unchanged
service/web-pv unchanged
[root@k8s-master ~]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
test-pvc Pending 7m28s
[root@k8s-master ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
web-pv-65d8dd9df8-h68nt 0/1 Pending 0 38s
web-pv-65d8dd9df8-qrkcd 0/1 Pending 0 38s
web-pv-65d8dd9df8-zx5x2 0/1 Pending 0 38s
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl describe pod web-pv-65d8dd9df8-h68nt
……
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 56s (x2 over 56s) default-scheduler 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims.
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl get pv
No resources found
[root@k8s-master ~]#创建PV，如下yaml定义了pv0001
[root@k8s-master ~]# cat test-pv0001.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv0001
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /nfstest
server: 192.168.231.123
[root@k8s-master ~]#创建后能获取到PV, 容器开始创建
[root@k8s-master ~]# kubectl apply -f test-pv0001.yaml
persistentvolume/pv0001 created
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv0001 5Gi RWX Recycle Bound default/test-pvc 10s
[root@k8s-master ~]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
test-pvc Bound pv0001 5Gi RWX 13m
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
web-pv-65d8dd9df8-h68nt 0/1 ContainerCreating 0 4m37s
web-pv-65d8dd9df8-qrkcd 0/1 ContainerCreating 0 4m37s
web-pv-65d8dd9df8-zx5x2 0/1 ContainerCreating 0 4m37s
[root@k8s-master ~]#容器创建好后进入容器，在挂载的持久卷中创建一个文件
[root@k8s-master ~]# kubectl exec -it web-pv-65d8dd9df8-h68nt -- bash
root@web-pv-65d8dd9df8-h68nt:/# df -h
Filesystem Size Used Avail Use% Mounted on
overlay 18G 7.6G 11G 43% /
tmpfs 64M 0 64M 0% /dev
tmpfs 910M 0 910M 0% /sys/fs/cgroup
/dev/sda3 18G 7.6G 11G 43% /etc/hosts
shm 64M 0 64M 0% /dev/shm
192.168.231.123:/nfstest 18G 7.6G 11G 43% /usr/share/nginx/html
tmpfs 910M 12K 910M 1% /run/secrets/kubernetes.io/serviceaccount
tmpfs 910M 0 910M 0% /proc/acpi
tmpfs 910M 0 910M 0% /proc/scsi
tmpfs 910M 0 910M 0% /sys/firmware
root@web-pv-65d8dd9df8-h68nt:/# cd /usr/share/nginx/html
root@web-pv-65d8dd9df8-h68nt:/usr/share/nginx/html# touch pvhaha
root@web-pv-65d8dd9df8-h68nt:/usr/share/nginx/html#在NFS服务器能看到这个文件
[root@k8s-node2 ~]# cd /nfstest
[root@k8s-node2 nfstest]# ls
pvhaha
[root@k8s-node2 nfstest]#(关于回收策略~~)
这个PV的回收策略是Recycle，现在来试试删除PVC的效果
[root@k8s-master ~]# kubectl delete -f test-deploy-pvc.yaml
deployment.apps "web-pv" deleted
persistentvolumeclaim "test-pvc" deleted
service "web-pv" deleted
[root@k8s-master ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv0001 5Gi RWX Recycle Released default/test-pvc 17m
[root@k8s-master ~]#过了一会儿，PV状态由Released变为Available
[root@k8s-master ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv0001 5Gi RWX Recycle Available 36m
[root@k8s-master ~]#并且在NFS服务器这个文件被删除了
[root@k8s-node2 nfstest]# pwd
/nfstest
[root@k8s-node2 nfstest]# ls
[root@k8s-node2 nfstest]#在NFS服务器这个文件没有被删除，需要手动处理
更新PV，将回收策略改为Retain
[root@k8s-master ~]# kubectl apply -f test-pv0001.yaml
persistentvolume/pv0001 configured
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl apply -f test-deploy-pvc.yaml
deployment.apps/web-pv created
persistentvolumeclaim/test-pvc created
service/web-pv created
[root@k8s-master ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
web-pv-65d8dd9df8-7fsgd 1/1 Running 0 109s
web-pv-65d8dd9df8-cdgpr 1/1 Running 0 109s
web-pv-65d8dd9df8-dmkbf 1/1 Running 0 109s
[root@k8s-master ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv0001 5Gi RWX Retain Bound default/test-pvc 41m
[root@k8s-master ~]# kubectl exec -it web-pv-65d8dd9df8-7fsgd -- bash
root@web-pv-65d8dd9df8-7fsgd:/# cd /usr/share/nginx/html
root@web-pv-65d8dd9df8-7fsgd:/usr/share/nginx/html# ls
root@web-pv-65d8dd9df8-7fsgd:/usr/share/nginx/html# touch pv-retain
root@web-pv-65d8dd9df8-7fsgd:/usr/share/nginx/html#同理在持久卷中创建一个文件，然后删除绑定的PVC，创建的文件保留着，需要手动处理
[root@k8s-node2 nfstest]# ls
pv-retain
[root@k8s-node2 nfstest]#将PV的回收策略改为Retain
[root@k8s-master ~]# kubectl apply -f test-deploy-pvc.yaml
deployment.apps/web-pv created
persistentvolumeclaim/test-pvc created
service/web-pv created
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl delete -f test-pv0001.yaml
persistentvolume "pv0001" deleted
[root@k8s-master ~]# kubectl apply -f test-pv0001.yaml
persistentvolume/pv0001 created
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv0001 5Gi RWX Delete Failed default/test-pvc 84m
[root@k8s-master ~]#可以观察到PV的状态是Failed
[root@k8s-master ~]# kubectl describe pv pv0001
Name: pv0001
...
Message: Error getting deleter volume plugin for volume "pv0001": no deletable volume plugin matched
Source:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: 192.168.231.123
Path: /nfstest
ReadOnly: false
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning VolumeFailedDelete 59s persistentvolume-controller Error getting deleter volume plugin for volume "pv0001": no deletable volume plugin matched
[root@k8s-master ~]#
PV卷的类型是NFS,不支持Delete吧(?)
来自官网 回收策略 -> 目前，仅 NFS 和 HostPath 支持回收（Recycle）。 AWS EBS、GCE PD、Azure Disk 和 Cinder 卷都支持删除（Delete）。
(关于请求的储存大小~~)
现在创建4个PV, 看看PVC会如何按需绑定, test-pv.yaml内容如下
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv0001
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /nfstest/pv0001
server: 192.168.231.123
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv0002
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /nfstest/pv0002
server: 192.168.231.123
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv0003
spec:
capacity:
storage: 20Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /nfstest/pv0003
server: 192.168.231.123
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv0004
spec:
capacity:
storage: 50Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /nfstest/pv0004
server: 192.168.231.123
[root@k8s-master ~]#同时在NFS服务器测创建对应的路径
[root@k8s-node2 ~]# cd /nfstest
[root@k8s-node2 nfstest]# mkdir pv0001
[root@k8s-node2 nfstest]# mkdir pv0002
[root@k8s-node2 nfstest]# mkdir pv0004
[root@k8s-node2 nfstest]# mkdir pv0003
[root@k8s-node2 nfstest]# ls
pv0001 pv0002 pv0003 pv0004
[root@k8s-node2 nfstest]#
创建这4个PV --> 5G, 10G, 20G, 50G, 当前状态都是Available
[root@k8s-master ~]# kubectl apply -f test-pv.yaml
persistentvolume/pv0001 created
persistentvolume/pv0002 created
persistentvolume/pv0003 created
persistentvolume/pv0004 created
[root@k8s-master ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv0001 5Gi RWX Recycle Available 9s
pv0002 10Gi RWX Recycle Available 9s
pv0003 20Gi RWX Recycle Available 9s
pv0004 50Gi RWX Recycle Available 9s
[root@k8s-master ~]#然后创建Deployment,SVC和PVC
(1) Deploymnet:web-pv1,PVC:test-pvc1(storage 4Gi)
[root@k8s-master ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv0001 5Gi RWX Recycle Bound default/test-pvc1 3m57s
pv0002 10Gi RWX Recycle Available 3m57s
pv0003 20Gi RWX Recycle Available 3m57s
pv0004 50Gi RWX Recycle Available 3m57s
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
test-pvc1 Bound pv0001 5Gi RWX 26s
[root@k8s-master ~]#需求4G, test-pvc1绑定的是5G容量的pv0001
(2) Deploymnet:web-pv2,PVC:test-pvc2(storage 4Gi)
[root@k8s-master ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv0001 5Gi RWX Recycle Bound default/test-pvc1 46m
pv0002 10Gi RWX Recycle Bound default/test-pvc2 46m
pv0003 20Gi RWX Recycle Available 46m
pv0004 50Gi RWX Recycle Available 46m
[root@k8s-master ~]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
test-pvc1 Bound pv0001 5Gi RWX 43m
test-pvc2 Bound pv0002 10Gi RWX 30s
[root@k8s-master ~]#需求也是4G, test-pvc2绑定的是10G容量的pv0002 (5G容量的pv0001已被绑定)
一旦绑定关系建立，则 PVC绑定就是排它性的。 PVC 申领与 PV 卷之间的绑定是一种一对一的映射，实现上使用 ClaimRef 来记述 PV 卷 与 PVC 申领间的双向绑定关系。
(3) Deploymnet:web-pv3,PVC:test-pvc3(storage 21Gi)
[root@k8s-master ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv0001 5Gi RWX Recycle Bound default/test-pvc1 50m
pv0002 10Gi RWX Recycle Bound default/test-pvc2 50m
pv0003 20Gi RWX Recycle Available 50m
pv0004 50Gi RWX Recycle Bound default/test-pvc3 50m
[root@k8s-master ~]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
test-pvc1 Bound pv0001 5Gi RWX 46m
test-pvc2 Bound pv0002 10Gi RWX 4m12s
test-pvc3 Bound pv0004 50Gi RWX 24s
[root@k8s-master ~]#需求21G, test-pvc3绑定的是50G容量的pv0004 (20G容量的pv0003的容量小于需求)
匹配最接近的容量, 且用户所获得的 PV 卷可能会超出所请求的配置, 而不会比请求的配置低。
(4) Deploymnet:web-pv4,PVC:test-pvc4(storage 30Gi)
[root@k8s-master ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv0001 5Gi RWX Recycle Bound default/test-pvc1 56m
pv0002 10Gi RWX Recycle Bound default/test-pvc2 56m
pv0003 20Gi RWX Recycle Available 56m
pv0004 50Gi RWX Recycle Bound default/test-pvc3 56m
[root@k8s-master ~]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
test-pvc1 Bound pv0001 5Gi RWX 53m
test-pvc2 Bound pv0002 10Gi RWX 10m
test-pvc3 Bound pv0004 50Gi RWX 6m39s
test-pvc4 Pending 12s
[root@k8s-master ~]#需求30G, 没有符合条件的PV, PVC处于Pending状态 (20G容量的pv0003的容量小于需求)
再创建一个容量30G的PV, 同时在NFS服务器创建对应的路径
[root@k8s-master ~]# vim test-pv0005.yaml
[root@k8s-master ~]# kubectl apply -f test-pv0005.yaml
persistentvolume/pv0005 created
[root@k8s-master ~]#
NFS:
[root@k8s-node2 nfstest]# mkdir pv0005
[root@k8s-node2 nfstest]# pwd
/nfstest
[root@k8s-node2 nfstest]#
可以看到需求21G的test-pvc4绑定了新创建的30G容量的pv0005
[root@k8s-master ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv0001 5Gi RWX Recycle Bound default/test-pvc1 58m
pv0002 10Gi RWX Recycle Bound default/test-pvc2 58m
pv0003 20Gi RWX Recycle Available 58m
pv0004 50Gi RWX Recycle Bound default/test-pvc3 58m
pv0005 30Gi RWX Recycle Bound default/test-pvc4 18s
[root@k8s-master ~]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
test-pvc1 Bound pv0001 5Gi RWX 55m
test-pvc2 Bound pv0002 10Gi RWX 12m
test-pvc3 Bound pv0004 50Gi RWX 8m46s
test-pvc4 Bound pv0005 30Gi RWX 2m19s
[root@k8s-master ~]#如果找不到匹配的 PV 卷，PVC 申领会无限期地处于未绑定状态。 当与之匹配的 PV 卷可用时，PVC 申领会被绑定。