In Kubernetes, StatefulSets are used to manage API objects for stateful applications. StatefulSets are only stable in Kubernetes version 1.9. StatefulSet manages Pod deployment and scaling, and provides order and uniqueness guarantees for these Pods. Similar to Deployment, StatefulSet manages Pods based on spec specifications; unlike Deployment, StatefulSet needs to maintain the unique identity of each Pod. These Pods are created based on the same spec, but cannot be replaced with each other. Each Pod retains its own persistent identity.
1. Scenarios using StatefulSet
StatefulSets are valuable for the following application scenarios:
-
Stable and unique network identification
-
Stable, durable storage
-
Deploy and scale in sequence and gracefully
-
In-order, graceful deletion and termination
-
Sequential, automatic rolling updates
The above-mentioned stable is synonymous with persistence. If the application does not require stable identification or sequential deployment, deletion, and expansion, stateless replica sets should be used. Deployment or ReplicaSet controllers are more suitable for stateless business scenarios.
2. Limitations of StatefulSet
-
It was a beta release before Kubernetes 1.9 and was not available until Kubernetes 1.5.
-
Pod storage is provided by PersistentVolume (storage class or pre-created by the administrator).
-
Deleting or shrinking a StatefulSet will not delete the data volume associated with the StatefulSet, which ensures data security.
-
The current StatefulSets require a Headless service to provide network identification for Pods, and this Headless service needs to be created manually.
3. Components
The following is an example of a StatefuleSet composition:
-
A Headless service named nginx to control the network domain.
-
A statefulSet named web that holds 3 replica sets of the nginx container (started on the unique Pod).
-
VolumeClaimTemplates that provide stable storage using PersistenVolumes (provided by PersistentVolume Provisioner).
apiVersion:v1
kind:Service
metadata:
name:nginx
labels:
app:nginx
spec:
ports:
-port:80
name:web
clusterIP:None //Headless服务
selector:
app:nginx
---
apiVersion:apps/v1
kind:StatefulSet
metadata:
name:web
spec:
selector:
matchLabels:
app:nginx# has to match .spec.template.metadata.labels
serviceName:"nginx"
replicas:3 # by default is 1
template:
metadata:
labels:
app:nginx # has to match .spec.selector.matchLabels
spec:
terminationGracePeriodSeconds:10
containers:
-name:nginx
image:k8s.gcr.io/nginx-slim:0.8
ports:
-containerPort:80
name:web
volumeMounts: #挂接数据卷
-name:www
mountPath:/usr/share/nginx/html #挂接路径为容器的/usr/share/nginx/html
volumeClaimTemplates: #数据卷生命模板
-metadata:
name:www
spec:
accessModes:["ReadWriteOnce"]
storageClassName:my-storage-class
resources:
requests:
storage:1Gi
4. Pod selector
StatefulSet's sepc.selector must be set to match .spec.template.metadata.labels. Before Kubernetes 1.8, spec.selector was ignorable and it was set to a default value. In version 1.8 or later, if sepc.selector is not set, the creation of StatefulSet will fail.
5. Pod identity
StatfuleSet Pod has a unique identity, which consists of order, stable network identity and stable storage. This identity always follows the Pod, but it is scheduled to that Node.
5.1 Ordinal Index
For a StatefulSet with N replica sets, each Pod in the StatefulSet will be assigned an integer ordinal between 0 and N, which is unique in the entire set.
5.2 Network ID (Stable Network ID)
In a StatefulSet, the host name of each Pod consists of the StatefulSet name and ordinal. The format of the host name of the Pod: $(statefulset name)-$(ordinal). If three Pods are created, their hostnames are web-0, web-1, web-2. StatefulSet can use the Headless service to control the domain of the Pod. The format of the domain managed by the Service is: $(service name).$(namespace).svc.cluster.local, where cluster.local is the cluster domain. For each created Pod, it will get a DNS subdomain in the format: $(podname).$(governing service domain), where the management service is in the StatefulSet, set by serviceName.
Here is the DNS name of the Pod in the StatefulSet:
5.3 Stable Storage
kubernetes creates a corresponding PersistentVolume for each VolumeClaimTemplate. In the previous nginx instance, each Pod will have a single instance of PersistenVolume of my-storage-class storage type and 1Gib of storage space.
If no storage class is specified, the default storage will be used. But when a Pod is scheduled to a Node, its volumeMounts will mount PersistentVolumes and associate it with PersistentVolumeClaims. It should be noted that even after the Pod is deleted, the association between PersistentVolumes and PersistentVolumeClaims will not be deleted.
5.4 Pod naming labels
When the StatefulSet controller creates a Pod, it will add a label with the set of Pod names. This label will be able to manage services to the specified Pod.
6. Deployment and expansion guarantee
-
For a StatefulSet with N replica sets, when Pods are deployed, they will be created in order 0 to N-1.
-
When a Pod is deleted, they will be terminated in reverse order from N-1 to 0.
-
All dependent Pods should be running and ready before scaling a Pod.
-
Before a Pod can be terminated, all Pods that depend on it must be completely stopped.
In the nginx example created earlier, web-0, web-1 and web-2 will be deployed in order. web-1 can only be deployed after web-0 is running and ready, and web-2 can only be deployed after web-1 is running and ready. If web-0 fails, even if web-1 is running, web-2 will not start normally unless web-0 is restarted and runs normally.
If you scale down the above example and set replicas=1, then web-2 is terminated first, followed by web-1. If web-0 fails after web-2 is terminated, but before web-1 is terminated, web-1 will not be terminated unless web-0 is in a healthy state.
6.1 Pod management strategy
After Kubernetes 1.7, the unique identification of StatefulSet can be guaranteed by the value of .spec.podManagementPolicy.
6.1.1 OrderedReady Pod Management
OrderedReady pod management is the default management mode of StatefulSets, which starts or terminates pods in order of installation.
6.1.2 Parallel Pod Management
Parallel Pod Management tells the StatefulSet controller to start or terminate all Pods in parallel.
7. Update strategy
After Kubernetes 1.7, run the .spec.updateStrategy that configures the StatefulSet to automatically update the Pod's containers, labels, resource requests/limits, and annotations.
7.1 On Delete Strategy
The OnDelete update strategy was the behavior of versions prior to 1.6. When the StatefulSet's .spec.updateStrategy.type is set to OnDelete, the StatefulSet controller will not know to update the Pod.
7.2 Rolling Updates Strategy
The RollingUpdate update strategy will implement the automatic rolling update of the Pods in the StatefulSet, which is the default update mode of the StatefulSet. If .spec.updateStrategy.type is set to RollingUpdate, the StatefulSet controller will delete and rebuild each Pod in the StatefulSet. It will terminate Pods in ascending order from largest to smallest and rebuild Pods in ascending order.
7.3 Partitions
RollingUpdate update strategies can be separated by specifying .spec.updateStrategy.rollingUpdate.partition. When separation is specified, all Pods with ordinal greater than or equal to the separation will be updated, and other Pods will not be updated. In most cases, splits are not used; splits are useful when you want to do a canary release, or perform a staged release.
About the Author:
Ji Xiangyuan, Product Manager of Beijing Shenzhou Aerospace Software Technology Co., Ltd. The copyright of this article belongs to the original author.