This article has been included in the column " Learn k8s from scratch "
Previous article: Click
Statefulset
Statefulset Controller: Concept and Principle Interpretation
StatefulSet is designed to manage the problem of stateful services
For the Pods in the StatefulSet, each Pod mounts its own independent storage. If a Pod fails, a Pod with the same name is started from another node, and the storage to which the original Pod is mounted continues to provide services in its state.
Stateful service?
A StatefulSet is a stateful collection that manages stateful services, and the names of the Pods it manages cannot be changed at will. The data persistence directory is also different. Each Pod has its own unique data persistence storage directory. Such as MySQL master-slave, redis cluster, etc.
- Make each Pod independent
- Make each Pod independent, keep the Pod startup order and uniqueness
- Unique network identifier, persistent storage
- Orderly, such as master-slave in mysql
Businesses suitable for StatefulSet include database services MySQL and PostgreSQL, clustered management services Zookeeper, etcd and other stateful services
Another typical application scenario of StatefulSet is as a more stable and reliable mechanism for simulating virtual machines than ordinary containers. A traditional virtual machine is a stateful pet, and operators need to maintain it constantly. When containers first became popular, we used containers to simulate the use of virtual machines, and all states were stored in containers, which has been proven is very unsafe and unreliable.
Using StatefulSet, Pods can still provide high availability by drifting to different nodes, and storage can also provide high reliability through external storage. What StatefulSet does is to associate certain Pods with certain storage to ensure the continuity of the state.
Stateless service?
RC, Deployment, and DaemonSet are all stateless services. The IP, name, start and stop order of the Pods they manage are random. The individual has no impact on the whole. All pods share a data volume. The deployed tomcat is a stateless service. If the tomcat is deleted, start a new tomcat and join the cluster, regardless of the name of the tomcat.
- Think Pods are all the same
- no order requirement
- Regardless of which node the application is running on
- Ability to scale and expand at will
A StatefulSet consists of the following parts:
1. Headless Service: used to define the pod network identifier and generate resolvable DNS records
2. volumeClaimTemplates: storage volume application template, create pvc, specify pvc name and size, automatically create pvc, and pvc is supplied by the storage class.
3.StatefulSet: manages pods
What is headless service
The headless service does not assign a clusterIP. The headless service can return the dns and ip addresses of all Pods by parsing the service's DNS (the Pods deployed by statefulSet have DNS). For ordinary services, the service's ClusterIP can only be returned by parsing the service's DNS.
Why use headless service (service without service ip)?
When using Deployment, the created pod names are not ordered, but random strings. When using statefulset to manage pods, the pod names must be ordered, and each pod cannot be replaced at will. After the pod is rebuilt, the pod name is still the same. . Because pod IPs change, use pod names to identify them. The pod name is a unique identifier for the pod and must be persistent and valid. This is where headless services are used, which can give each pod a unique name.
1.headless service 会为 service 分配一个域名
<service name>.$<namespace name>.svc.cluster.local
K8s 中资源的全局 FQDN 格式:
Service_NAME.NameSpace_NAME.Domain.LTD.
Domain.LTD.=svc.cluster.local. #这是默认 k8s 集群的域名。
FQDN 全称 Fully Qualified Domain Name 即全限定域名:同时带有主机名和域名的名称
FQDN = Hostname + DomainName
如 主机名是 paopao 域名是 csdn.com
FQDN= paopao.csdn.com
2.StatefulSet 会为关联的 Pod 保持一个不变的 Pod Name
statefulset 中 Pod 的名字格式为$(StatefulSet name)-$(pod 序号)
3.StatefulSet 会为关联的 Pod 分配一个 dnsName
$<Pod Name>.$<service name>.$<namespace name>.svc.cluster.local
Why use volumeClaimTemplate?
Persistent storage is used for stateful applications, such as mysql master-slave. Since the data of the master-slave database cannot be stored in a directory, each mysql node needs to have its own independent storage space. The storage volume created in the deployment is a shared storage volume, and multiple pods use the same storage volume, and their data is synchronized, and each pod in the statefulset definition cannot use the same storage volume, which requires the use of volumeClainTemplate, when using statefulset to create a pod, volumeClainTemplate will automatically generate a PVC to request to bind a PV, each pod has its own dedicated storage volume. The relationship diagram corresponding to Pod, PVC and PV is as follows:
Statefulset resource manifest file writing skills
#查看定义 Statefulset 资源需要的字段
[root@k8smaster ~]# kubectl explain statefulset
KIND: StatefulSet
VERSION: apps/v1
DESCRIPTION:
StatefulSet represents a set of pods with consistent identities. Identities
are defined as: - Network: A single stable DNS and hostname. - Storage: As
many VolumeClaims as requested. The StatefulSet guarantees that a given
network identity will always map to the same storage identity.
FIELDS:
apiVersion <string> #定义 statefulset 资源需要使用的 api 版本
kind <string> #定义的资源类型
metadata<Object> #元数据
spec <Object> #定义容器相关的信息
#查看 statefulset.spec 字段如何定义?
[root@k8smaster ~]# kubectl explain statefulset.spec
KIND: StatefulSet
VERSION: apps/v1
RESOURCE: spec <Object>
DESCRIPTION:
Spec defines the desired identities of pods in this set.
A StatefulSetSpec is the specification of a StatefulSet.
FIELDS:
podManagementPolicy <string> #pod 管理策略
replicas <integer> #副本数
revisionHistoryLimit <integer> #保留的历史版本
selector <Object> -required- #标签选择器,选择它所关联的 pod
serviceName <string> -required- #headless service 的名字
template <Object> -required- #生成 pod 的模板
updateStrategy <Object> #更新策略
volumeClaimTemplates<[]Object> #存储卷申请模板
#查看 statefulset 的 spec.template 字段如何定义?
#对于 template 而言,其内部定义的就是 pod,pod 模板是一个独立的对象
[root@k8smaster ~]# kubectl explain statefulset.spec.template
KIND: StatefulSet
VERSION: apps/v1
RESOURCE: template <Object>
DESCRIPTION:
template is the object that describes the pod that will be created if
insufficient replicas are detected. Each pod stamped out by the StatefulSet
will fulfill this Template, but have a unique identity from the rest of the
StatefulSet.
PodTemplateSpec describes the data a pod should have when created from a
template
FIELDS:
metadata <Object>
spec <Object> #定义容器属性的
As you can see above, there are two spec fields in the statefulset resource.
The first spec declares how many Pod replicas the statefulset defines (only one Pod will be deployed by default), the selector that matches the Pod label, the template for creating the pod, and the storage volume request template.
The second spec is spec.template.spec: mainly used for configuration such as container properties in Pod. The contents of .spec.template are various properties to be defined when declaring a Pod object, so this part is also called PodTemplate (Pod Template).
One more thing worth noting: the label selector defined in .spec.selector must be able to match the Pod label defined in spec.template.metadata.labels, otherwise Kubernetes will not allow the creation of statefulset.
Statefulset use case: deploying a web site
#创建存储类
[root@k8smaster ~]# mkdir state
[root@k8smaster ~]# cd state/
[root@k8smaster state]# vim class-web.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nfs-web
provisioner: example.com/nfs
#更新资源清单文件
[root@xianchaomaster1 ~]# kubectl apply -f class-web.yaml
[root@k8smaster state]# kubectl get storageclass
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
nfs-web example.com/nfs Delete Immediate false 6m16s
#编写一个 Statefulset 资源清单文件
[root@k8smaster state]# vim statefulset.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
selector:
matchLabels:
app: nginx
serviceName: "nginx"
replicas: 2
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
name: web
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
volumeClaimTemplates:
- metadata:
name: www
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: "nfs-web"
resources:
requests:
storage: 1Gi
#更新资源清单文件
[root@k8smaster state]# kubectl apply -f statefulset.yaml
service/nginx created
statefulset.apps/web created
#查看 statefulset 是否创建成功
[root@k8smaster state]# kubectl get sts -o wide
NAME READY AGE CONTAINERS IMAGES
web 2/2 59s nginx nginx
#查看 pod
[root@k8smaster state]# kubectl get pods -l app=nginx
NAME READY STATUS RESTARTS AGE
web-0 0/1 Pending 0 76s
web-1 1/1 Pending 0 76s
#通过上面可以看到创建的 pod 是有序的
#查看 headless service
[root@k8smaster state]# kubectl get svc -l app=nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx ClusterIP None <none> 80/TCP 3m
#查看 pvc
[root@k8smaster state]# kubectl get pvc
www-web-0 Bound pvc-13a0482f-4927-63ff-9f2e-1b926b299c6f 1Gi RWO,RWX nfs-web 4m45s
www-web-1 Bound pvc-bc23p2a9-5bjd-00df-829f-ccb4d24g01h1 1Gi RWO,RWX nfs-web 4m41s
#查看 pv
[root@k8smaster state]# kubectl get pv
pvc-13a0482f-4927-63ff-9f2e-1b926b299c6f 1Gi RWO,RWX Delete Bound default/www-web-0 nfs-web 5m3s
pvc-bc23p2a9-5bjd-00df-829f-ccb4d24g01h1 1Gi RWO,RWX Delete Bound default/www-web-1 nfs-web 5m59s
#在data nfs_pro目录下划分的pv
#查看 pod 主机名
[root@k8smaster ~]# for i in 0 1; do kubectl exec web-$i -- sh -c 'hostname';done
web-0
web-1
#使用 kubectl run 运行一个提供 nslookup 命令的容器的,这个命令来自于 dnsutils 包,通过对 pod 主机名执行 nslookup,可以检查它们在集群内部的 DNS 地址:
[root@k8smaster ~]# kubectl exec -it web-1 -- /bin/bash
root@web-1:/# apt-get update
root@web-1:/# apt-get install dnsutils -y
root@web-1:/# nslookup web-0.nginx.default.svc.cluster.local
Server: 10.96.0.10
Address: 10.96.0.10#53
Name: web-0.nginx.default.svc.cluster.local
#statefulset 创建的 pod 也是有 dns 记录的
Address: 10.244.103.139 #解析到的是 pod 的 ip 地址
root@web-1:/# nslookup nginx.default.svc.cluster.local
Server: 10.96.0.10
Address: 10.96.0.10#53
Name: nginx.default.svc.cluster.local #查询 service dns,会把对应的 pod ip 解析出来
Address: 10.244.103.139
Name: nginx.default.svc.cluster.local
Address: 10.244.121.93
[root@k8smaster state]# kubectl describe svc nginx
Name: nginx
Namespace: default
Labels: app=nginx
Annotations: <none>
Type: ClusterIP
IP: none
Port: web 80/TCP
TargetPort: 80/TCP
Endpoints: 10.244.103.139:80,10.244.121.93:80 #这两个pod都有app=ngin的标签,service把这两个都写入endpoints列表了
Session Affinity: None
Events: <none>
root@web-1:/# dig -t A nginx.default.svc.cluster.local @10.96.0.10
; <<>> DiG 9.11.5-P4-5.1+deb10u3-Debian <<>> -t A nginx.default.svc.cluster.local @10.96.0.10
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16869
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 1cf973a5daa99ac7 (echoed)
;; QUESTION SECTION:
;nginx.default.svc.cluster.local. IN A
;; ANSWER SECTION:
nginx.default.svc.cluster.local. 30 IN A 10.244.103.139
nginx.default.svc.cluster.local. 30 IN A 10.244.121.93
;; Query time: 0 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Thu Jul 14 06:49:51 UTC 2022
;; MSG SIZE rcvd: 166
root@web-1:/# nslookup kubernetes.default.svc.cluster.local
Server: 10.96.0.10
Address: 10.96.0.10#53
Name: kubernetes.default.svc.cluster.local
Address: 10.96.0.1
#解析kubernertes,有ip会把ip解析出来,没有会解析后端pod
dig 的使用
dig -t A nginx.default.svc.cluster.local @10.96.0.10
格式如下:
@来指定域名服务器
A 为解析类型 ,A 记录
-t 指定要解析的类型
A 记录:
A 记录是解析域名到 IP
资源清单详细解读:
apiVersion: v1 #定义 api 版本
kind: Service #定义要创建的资源:service
metadata:
name: nginx #定义 service 的名字
labels:
app: nginx #service 的标签
spec:
ports:
- port: 80
name: web
clusterIP: None #创建一个没有 ip 的 service
selector:
app: nginx #选择拥有 app=nginx 标签的 pod
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
selector:
matchLabels:
app: nginx
serviceName: "nginx" #headless service 的名字
replicas: 2 #副本数
template: #定义 pod 的模板
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
name: web
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
volumeClaimTemplates: #存储卷申请模板
- metadata:
name: www
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: "nfs-web" #指定从哪个存储类申请 pv
resources:
requests:
storage: 1Gi #需要 1G 的 pvc,会自动跟符合条件的 pv 绑定
扩展:
service 和 headless service 区别:
#deployment 创建的 pod 是随机生成的 解析的是 service 的 ip 地址
Statefulset manages pods: scaling, scaling, updating
#Statefulset 实现 pod 的动态扩容
如果我们觉得两个副本太少了,想要增加,只需要修改配置文件 statefulset.yaml 里的 replicas的值即可,原来 replicas: 2,现在变成 replicaset: 3,修改之后,执行如下命令更新:
[root@k8smaster state]# kubectl apply -f statefulset.yaml
service/nginx unchanged
statefulset.apps/web configured
[root@k8smaster state]# kubectl get sts
web 3/3 30m
[root@k8smaster state]# kubectl get pods -l app=nginx
NAME READY STATUS RESTARTS AGE
web-0 1/1 Running 0 61m
web-1 1/1 Running 0 60m
web-2 1/1 Running 0 79s
#也可以直接编辑控制器实现扩容
[root@xianchaomaster1 ~]# kubectl edit sts web
#这个是我们把请求提交给了 apiserver,实时修改,把 spec 下的 replicas 后面的值改成 4,保存退出
[root@xianchaomaster1 ~]# kubectl get pods -l app=nginx
NAME READY STATUS RESTARTS AGE
web-0 1/1 Running 0 62m
web-1 1/1 Running 0 62m
web-2 1/1 Running 0 3m13s
web-3 1/1 Running 0 26s
#Statefulset 实现 pod 的动态缩容
如果我们觉得 4 个 Pod 副本太多了,想要减少,只需要修改配置文件 statefulset.yaml 里的
replicas 的值即可,把 replicaset:4 变成 replicas: 2,修改之后,执行如下命令更新:
[root@k8smaster state]# kubectl apply -f statefulset.yaml
service/nginx unchanged
statefulset.apps/web configured
[root@k8smaster state]# kubectl get pods -l app=nginx
NAME READY STATUS RESTARTS AGE
web-0 1/1 Running 0 64m
web-1 1/1 Running 0 63m
#Statefulset 实现 pod 的更新
[root@k8smaster state]# kubectl edit sts web
#修改镜像 nginx 变成 image: tomcat,修改之后保存退出
[root@k8smaster state]# kubectl get pods -o wide -l app=nginx
NAME READY STATUS RESTARTS AGE IP NODE
web-0 1/1 Running 0 18s 10.244.201.106 k8snode
web-1 1/1 Running 0 36s 10.244.133.176 k8snode2
#查看 pod 详细信息
[root@k8smaster state]# kubectl describe pods web-0
通过上面可以看到 pod 已经使用刚才更新的镜像 tomcat 了
write at the end
It is not easy to create, if you think the content is helpful to you, please give me a three-link follow to support me! If there are any mistakes, please point them out in the comments and I will change them in time!
The series that is currently being updated: learn k8s from scratch.
Thank you for watching. The article is mixed with personal understanding. If there is any error, please contact me and point it out~