Source code analysis of storage-related Kubernetes

All the code in this section based on the version 1.13.4.

Foreword

In kubernetes, the storage-related controller is mainly composed of three:
1, AttachDetachController, referred to as AD Controller, mainly deal with real action associated with Volume;
2, PersistentVolumeBinderController, in fact, PV Controller, is responsible for the life cycle of pv and pvc and switching state;
. 3, VolumeExpandController, responsible for volume expansion operation.

PersistentVolumeBinderController

First PersistentVolumeBinderController tracking methods, or enter the Run method.

Intuitive, depends on three goroutines:
. 1, the resync;
2, volumeWorker;
. 3, claimWorker.

resync

Enter resync, the code is very simple, as follows:

Resync main role is to constantly obtain information pv and pvc, passing to the corresponding cache queue. This information is in two queues volumeWorker and claimWorker used data source.

volumeWorker

volumeWorker for loop, constantly resync acquisition buffer queue information for pv, i.e. volume update operation accordingly. The main implementation methodupdateVolume

Into the updateVolume method, call the main syncVolume method, which is the core of volumeWorker. Process is as follows:
1, if the volume is not used, the status is updated PV Available ;
2, pvc volume has been held:

• If the volume has not been bound to the pvc, to update the status of PV Available;
  
• If no information is pvc, pv is not a state Releasednot to Failedupdate the status of PV Released, recovered according to the recovery execution policy configuration pv (the calling reclaimVolumemethod);
• pvc volume specified in the same field and the current volume, PV status is updated Bound;
• If not satisfied, judgment is performed according to the state of the recovery strategy or unbound (unbindVolume) operation.
  

claimWorker

claimWorker workflow and volumeWorker Similarly, the core call the method is updateClaim-->syncClaimmainly dealing with various states pvc lifecycle: Pending, Bound and Lost. Not described in detail here.

to sum up

PersistentVolumeBinderController implementation process is very clear, three-dependent goroutine collaboration, process data acquisition, respectively, the state status updates and life cycle of the life cycle of pv pvc updates. The entire logic, do not operate on specific volume, updated information is only pv and pvc resources kubernetes defined, it means the data etcd. The particular work mainly AttachDetachController, i.e. AD Controller.

AttachDetachController

First, the Runmethod proceeds to start the method of AD Controller, as follows:

Mainly in the following steps constitute the first step:
1, the synchronization information of each resource, including Pod, the Node, the PV, of PVC;
2, calling populateActualStateOfWorld a method to obtain information on the Volume of the Node;
. 3, calls the populateDesiredStateOfWorld method to obtain information Volume Pod corresponding to require;
4, reconciler.Run responsible for checking the mounted state, it is determined whether mounted or unmounted (real work);
5, desiredStateOfWorldPopulator.Run the mount information and synchronization Volume Pod, the corresponding information is supplied to reconciler.Run use;
6, pvcWorker the control flow control pvc;
7, corresponding to the information is registered metrics , the data collected for the use of Prometheus.

populateActualStateOfWorld

populateActualStateOfWorldThe main method of treatment is the relationship between the Node and Volume, Volume Node main role is deposited into the current state actualStateOfWorldof the. The main methods are as follows:

The main steps are as follows:
1. get all the information Node;
2, eleven traversing all Node acquired, for the Node has attached a Volume, respectively, has been placed in state and attached in-user state, the information is added to the Volume actualStateOfWorld , and the Node adding to desiredStateOfWorld the. actualStateOfWorld And desiredStateOfWorld the data will reconciler.Run be used.

populateDesiredStateOfWorld

populateDesiredStateOfWorldThe main method of treatment is the relationship between the Pod and Volume, the main role is to add a desired Pod Volume state to desiredStateOfWorldgo. And populateActualStateOfWorldthe like, mainly for the marking operation Volume, Pod and corresponding to the information in the cache desiredStateOfWorldor from desiredStateOfWorldthe information does not match Pod removed.

desiredStateOfWorldPopulator.Run

desiredStateOfWorldPopulator.RunMethods stop a loop to call findAndAddActivePodsthe method, by acquiring Pod all, whether it is necessary to add to desiredStateOfWorldgo.

reconciler.Run

The main purpose of the foregoing steps to obtain the status of Node Volume and Pod Volume. Which, Volume on the Node already exist, as it is called actualStateOfWorld, and Pod Volume is the ultimate resource needs in effect, so called desiredStateOfWorld. reconciler.RunThe action is non-stop through the acquisition actualStateOfWorldand desiredStateOfWorldstate, and Volume Pod placed corresponding state to ensure the ultimate success or unloaded successfully mount the disk. The main methods are as follows:

Stop the cycle call reconciliationLoopFunc method.
First into the reconcile method, which is where the real work.
reconcile Use the three large loop for processing three types of events:
1, first of all eliminate the need unbundling of Volume, call the UnmountVolume method eventually calls unbundling of storage back-end interfaces;
2, would require volumes Attach or Mount call back execution Attach Storage Interface Mount or operation;
3, would require a Detach Unmount devices or calls back the storage interface or perform Unmount Detach operation.
Wherein, Attach operation refers to the label generated in Volume Node, as is common /dev/xx and the like. Mount and Mount MountDevice operations include two portions, wherein, MountDevice generated label mount path to Node, generally /var/lib/kubelet/xx/kubernetes.io/xx under different storage depends on the path Mount MountDevice generated path will eventually need to use Mount Pod and up general path /var/lib/kubelet/pods/xx/volumes/xx .
sync Volume methods are mainly the subsequent operation is completed. If Volume is not successfully bind, Volume will be rebuilt or unbundling operations.

ExpandController

Kubernetes expansion in operating volumes began to support 1.8, 1.11 function has been in Beta stage. The main code is as follows:

pvcPopulator.Run Monitor change in the PVC, as long as the value of the PVC Storage Request Status field than the large, i.e., the capacity change represents PVC, additional capacity is needed. In this case, the corresponding PV and PVC information to the cache resizeMap go. as follows:

syncResize It is continuously acquired resizeMap data, if there is a change, then call ExpandVolume a method to generate an extended operation, and to complete the expansion disk PV, PVC status update operations. code show as below:

to sum up

Kubernetes storage aimed Node, Pod, PV and PVC four categories of resources. , Which was bound by acquiring Pod in Volume Volume state Node, complete loading volume. The final binding or unbinding and other operations that depend on the background storage, including built-in memory card or plug-own open-source implementation (FlexVolume or CSI).
In kubelet while exist VolumeManagerto manage the information on the resource node Volume thereof, substantially consistent with AD Controller function. By kube-controller-manager of --disable-attach-detach-reconcile-syncparameters or kubelet the --enable-controller-attach-detachattach is performed by the volume kube-controller-manager parameter control / detach operation or kubelet appropriate action.

Reproduced in: https: //juejin.im/post/5cf114d55188253cec305973