Principles of shared storage
PV is an abstraction of the underlying network shared storage and defines shared storage as a "resource". For example, Node is also a resource that can be "consumed" by container applications. The PV is created and configured by the administrator, and it is directly related to the specific implementation of shared storage, such as shared storage provided by GlusterFS, iSCSI, RBD or GCE or AWS public cloud, and is connected to shared storage through a plug-in mechanism for supply. Accessing and using
PVC is an "application" for storage resources. Just like Pods "consume" Node resources, PVC can "consume" PV resources. PVC can apply for specific storage space and access mode
, the mechanism of StorageClass and dynamic resource supply has been improved, the on-demand creation of storage volumes has been realized, and an important step has been achieved in the automated management process of shared storage. Through the definition of StorageClass, Administrators can define storage resources as a certain class (Class), just like the storage device's own configuration description (Profile), such as "fast storage", "slow storage", "with data redundancy", "no data redundancy", etc. . Users can intuitively know the characteristics of various storage resources according to the description of StorageClass, and they can apply for storage resources according to the application's demand for storage resources.
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv1
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
storageClassName: slow
nfs:
path: /tmp
server: 172.17.0.2
1. Storage capacity (Capacity)
describes the capabilities of storage devices, currently only supports storage space settings (storage = xx), in the future may add IOPS, throughput rate and other indicators settings
2. Storage Volume Mode (Volume Mode)
Kubernetes from Version 1.13 introduced the setting of storage volume type (volumeMode = xxx), the options include Filesystem (file system) and Block (block device), the default value is Filesystem
currently has the following PV types support block device type: ◎ AWSElasticBlockStore ◎ AzureDisk ◎ FC ◎ GCEPersistentDisk ◎ iSCSI ◎ Local volume ◎ RBD (Ceph Block Device) ◎ VsphereVolume (alpha)
apiVersion: v1
kind: PersestentVolume
metadata:
name: pv2
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
volumeMode: Block
fc:
targetWWNs: ["50060e801049cdf1"]
lun: 0
readOnly: false
3. Access Modes (Access Modes) Set the access mode of the PV, used to describe the user's application access to storage resources. The access mode is as follows
◎ ReadWriteOnce (RWO): read and write permissions, and can only be mounted by a single Node. ◎ ReadOnlyMany (ROX): read-only permission, allowing to be mounted by multiple Nodes. ◎ ReadWriteMany (RWX): Read and write permissions, allowing mounting by multiple Nodes.
Some PVs may support multiple access modes, but PVs can only use one access mode when mounted, and multiple access modes cannot take effect at the same time
PV can set its storage category, and specify the name of a StorageClass resource object through the storageClassName parameter. PVs with a specific category can only be bound to PVCs that have requested that category. PVs of unspecified categories can only be bound to PVCs that do not request any category.
5. Reclaim Policy
is set by the persistentVolumeReclaimPolicy field in the PV definition. The options are as follows. ◎ Retain: Retaining data requires manual processing. ◎ Reclaim space: simple operation to clear files (for example, execute rm -rf / thevolume / * command). ◎ Deletion: The back-end storage connected to the PV completes the volume deletion operation (such as the internal volume cleaning of AWS EBS, GCE PD, Azure Disk, OpenStack Cinder and other devices). Currently, only two types of storage, NFS and HostPath, support the Recycle strategy; AWS EBS, GCE PD, Azure Disk, and Cinder volumes support the Delete strategy
. 6. Mounting parameters (Mount Options)
When mounting PV to a Node, according to The characteristics of back-end storage may require additional mounting parameters, which can be set according to the mountOptions field in the PV definition
apiVersion: v1
kind: PersistentVolume
metadata:
name: gce-disk-l
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
mountOptions:
- hard
- nolock
- nfservers=3
gcePersistentDisk:
fsType: ext4
pdName: gce-disk-l
Currently, the following PV types support setting mounting parameters:
◎ AWSElasticBlockStore ◎ AzureDisk ◎ AzureFile ◎ CephFS ◎ Cinder (OpenStack block storage) ◎ GCEPersistentDisk ◎ Glusterfs ◎ NFS ◎ Quobyte Volumes ◎ RBD (Ceph Block Device) ◎ StorageOS ◎ VsphereVolume ◎ iSCSI
7. Node Affinity (Node Affinity)
PV can set node affinity to restrict access to Volume only through certain Nodes, which can be set in the nodeAffinity field in the PV definition. Pods using these Volumes will be scheduled to the nodes that meet the conditions
A PV may be in one of the following four phases (Phaes) in its life cycle. ◎ Available: Available, not yet bound to a PVC. ◎ Bound: It has been bound to a PVC. ◎ Released: The bound PVC has been deleted and the resources have been released, but it has not been recovered by the cluster. ◎ Failed: Automatic resource recovery failed.
As a user's request for storage resources, PVC mainly includes the setting of information such as storage space request, access mode, PV selection conditions and storage category. The PVC declared in the following example has the following attributes: apply for 8GiB storage space, the access mode is ReadWriteOnce, the PV selection condition is a tag that contains the label "release = stable" and the condition is "environment In [dev]", and the storage category is "slow" (Requires that a StorageClass named slow already exists in the system)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteOnce
resources:
request:
storage: 8Gi
storageClassName: slow
selector:
matchLabels:
release: "stable"
matchExpressions:
- {key: environment,operator: In,values: [dev]}
The key configuration parameters of PVC are described below.
◎ Resource requests (Resources): Describes requests for storage resources, currently only supports request.storage settings, that is, storage space size.
◎ Access Modes: PVC can also set access modes, which are used to describe the access rights of user applications to storage resources. The settings of its three access modes are the same as those of PV.
◎ Storage Modes (Volume Modes): PVC can also set storage volume modes, which are used to describe the PV storage volume modes you want to use, including file systems and block devices.
◎ PV selection conditions (Selector): By setting the Label Selector, the PVC can be used to screen various PVs that already exist in the system. The system will select the appropriate PV to bind to the PVC according to the label. The selection conditions can be set using matchLabels and matchExpressions. If both fields are set, the logic of the Selector will be that both sets of conditions are met at the same time to complete the match
◎ Storage class (Class): PVC can set the required back-end storage class (specified by the storageClassName field) when it is defined, in order to reduce the dependence on the detailed information of the back-end storage characteristics. Only the PV with the Class set can be selected by the system, and the PVC can be bound to the
PVC without the Class requirement. If the value of the storageClassName field is set to empty (storageClassName = ""), it means that the PVC does not require a specific Class, and the system will only select PVs with no Class set to match and bind. PVC can also not set the storageClassName field at all. At this time, the corresponding operation will be performed according to whether the system is enabled with the admission controller named DefaultStorageClass
◎ DefaultStorageClass is not enabled: the value of storageClassName is equivalent to PVC setting storageClassName = "" (storageClassName = ""), that is Only PVs with no Class set can be selected to match and bind to them.
◎ Enable DefaultStorageClass: requires the cluster administrator to define the default StorageClass. If there is no default StorageClass in the system, it is equivalent to not enabling DefaultStorageClass. If there is a default StorageClass, the system will automatically create a PV for the PVC (backend storage using the default StorageClass) and bind them. The method for the cluster administrator to set the default StorageClass is to add an annotation "storageclass.kubernetes.io/is-default-class=true" to the definition of StorageClass. If the administrator defines multiple StorageClasses as default, the system will not be able to create corresponding PVs for PVC because they are not unique
The life cycle of PV and PVC
We can think of PV as an available storage resource, and PVC is the demand for storage resources. The relationship between PV and PVC follows the life cycle shown in Figure 8.1
◎ Static mode: the cluster administrator manually creates many PVs, and the characteristics of the back-end storage need to be set when defining the PV
◎ Dynamic mode: the cluster administrator does not need to manually create the PV, but describes the back-end storage through the setting of StorageClass, Mark as a certain type. At this time, PVC is required to declare the type of storage, and the system will automatically complete the creation of PV and the binding with PVC. PVC can declare Class as "", indicating that the PVC is prohibited to use dynamic mode
After the resource is bound
by the user-defined PVC, the system will select a PV that meets the PVC requirements from the existing PVs according to the PVC's request for storage resources (storage space and access mode). The PVC is bound, and the user's application can use this PVC. If there is no PV that meets the PVC requirements in the system, the PVC will remain in the Pending state indefinitely until the system administrator creates a PV that meets its requirements. Once the PV is bound to a certain PVC, it will be monopolized by this PVC and can no longer be bound to other PVCs. In this case, when the storage space requested by PVC is less than that of PV, the entire PV space can be used by PVC, which may cause waste of resources. If the resource supply uses the dynamic mode, after the system finds the appropriate StorageClass for PVC, it will automatically create a PV and complete the binding of the
resource with the PVC. Use
Pod to use the definition of Volume to mount PVC to a certain container Path to use. The type of Volume is persistentVolumeClaim, which will be described in detail in the following examples. After a PVC is mounted on the container application, it can be used continuously and exclusively. However, multiple Pods can mount the same PVC. The application needs to consider the problem of multiple instances accessing a storage space together.
Resource release.
After the user has finished using the storage resources, the user can delete the PVC. The PV bound to the PVC will be It will be marked as "released", but it cannot be bound to other PVCs immediately. The data written by the previous PVC may still be left on the storage device, and the PV can only be used for
resource recycling after being cleared.
For the PV, the administrator can set a recycling policy for setting the PVC bound to it to release resources How to deal with the problem of legacy data. Only after the PV storage space is recovered can it be used for new PVC binding and use. The recycling strategy is described in the next section
The following two figures illustrate the principle of using PVC in PV, PVC, StorageClass and Pod in static resource supply mode and dynamic resource supply mode
. Figure 8.2 describes the binding of PV and PVC in static resource supply mode , And storage management mechanism for Pod
Figure 8.3 describes the storage management mechanism used by the Pod to dynamically bind resources (the system automatically generates PV) through the StorageClass and PVC in the dynamic resource supply mode.
StorageClass explains
StorageClass as an abstract definition of storage resources. It shields the details of back-end storage from PVC applications set by users. The system automatically completes the creation and binding of PV to achieve dynamic resource supply. The dynamic resource supply mode based on StorageClass will gradually become the standard storage configuration mode of the cloud platform