Explanation of Kubernetes pod controller DaemonSet and StatefulSet

The use of DaemonSet and StatefulSet

Today we will explain to you how to use another Podcontroller. We mainly explained Deploymentthe use of this kind of object resources. Next, we will explain the controllers used in specific occasions: DaemonSetand StatefulSet.

Use of DaemonSets

From the name of the controller, we can see its usage: Daemon is used to deploy the daemon process, and is used to run a copy of the daemon process as a background process DaemonSetin each Kubernetesnode. To put it bluntly, it is to deploy a daemon process on each node PodCopy, when a node joins Kubernetesthe cluster, Podit will be scheduled to run on the node. When the node can only be removed from the cluster, this node on the node Podwill also be removed. Of course, if we delete DaemonSet, all and this object The related ones Podswill be deleted.

Under what circumstances do we need to use this business scenario? In fact, this scenario is relatively common, such as:

• A cluster storage daemon, e.g. glusterdto cephbe deployed on each node to provide persistent storage;
• Node monitoring daemons, such as Prometheusmonitoring clusters, can run a process on each node node-exporterto collect information about monitoring nodes;
• A log collection daemon, such as fluentdor logstash, runs on each node to collect logs for containers

One thing that needs to be specially explained here is the scheduling problem of DaemonSetrunning Pod. Under normal circumstances, Podwhich node to run on is Kubernetesdetermined by the scheduler policy. However, the node DaemonSetcreated by the controller Podhas actually been determined in advance. ( Podspecified at creation time .spec.nodeName), so:

• DaemonSetIt doesn't care about the fields of a node unshedulable, which we will explain to you in the scheduling chapter later.
• DaemonSetCan be created Podeven if the scheduler has not been started, which is very important.

Below we directly use an example to demonstrate, deploying one on each node Nginx Pod: (nginx-ds.yaml)

kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: nginx-ds
  labels:
    k8s-app: nginx
spec:
  selector:
    matchLabels:
      k8s-app: nginx
  template:
    metadata:
      labels:
        k8s-app: nginx
    spec:
      containers:
      - image: nginx:1.7.9
        name: nginx
        ports:
        - name: http
          containerPort: 80

Then create it directly:

$ kubectl create -f nginx-ds.yaml

Then we can observe Podwhether it is distributed to each node:

$ kubectl get nodes
$ kubectl get pods -o wide

Use of StatefulSets

Before learning StatefulSetthis kind of controller, we have to understand a concept: what is a stateful service? What is a stateless service?

• Stateless Service (Stateless Service): The running instance of this service will not store locally the data that needs to be persisted, and the results of multiple instances responding to the same request are completely consistent, such as the instance we explained earlier, do WordPresswe Multiple instances can be started at the same time, but the results we get from any instance are the same, right? Because the only data he needs to persist is stored in MySQLthe database, we can say that WordPressthis application is a stateless service, but MySQLthe database is not, because he needs to persist the data locally.

• Stateful Service (Stateful Service): It is opposite to the above concept. The running instance of this service needs to store persistent data locally. For example, the above database, if you are running on node A now, its data will be stored MySQLin On node A, if you migrate the service to node B at this time, there will be no previous data, because he needs to restore the data in the corresponding data directory, and there is no data at this time.

Now everyone has a certain understanding of stateful and stateless. For example, our common WEB application sessionmaintains the user's login status. If we sessionpersist it to the node, then the application is a stateful service Yes, because I am now logged in and you have persisted my data to node A. The next time I log in, the request may be routed to node B, but there is no current data sessionon node B at all, so it will sessionIt is considered as not logged in, which caused the results of my two requests to be inconsistent. So in general, for horizontal expansion, we will change this type of WEB application into a stateless service, how to change it? Is it enough to sessionstore the data in a public place, such as inside, for some client requests, we don't use it to keep the user status, and it is also possible to use it instead.redisAPIsessiontoken

Stateless services can be well controlled by using our previous or stateful services. For stateful services, there are many more details to consider. One of the most difficult tasks for containerized applications is to design the deployment of stateful distributed components Deployment. RCArchitecture. Since stateless components may not have predefined startup sequences, clustering requirements, point-to-point TCP connections, unique network identifiers, graceful startup and termination requirements, etc., they can be easily containerized. Systems such as databases, big data analysis systems, distributed key/value stores, and message brokers may have complex distributed architectures, and may use the above functions. To this end, resources Kubernetesare introduced StatefulSetto support this complex requirement.

StatefulSetSimilar to ReplicaSet, but it can handle Podthe startup sequence, Podsetting a unique identifier for preserving the state of each, while having the following functions:

• Stable, unique network identifier
• Stable, persistent storage
• Orderly, graceful deployment and scaling
• Orderly, graceful deletion and termination
• Orderly, automatic rolling updates

Create a StatefulSet

Next, we will show you StatefulSethow to use the object. Before we start, we will prepare two 1G storage volumes (PV). In the following courses, we will also explain in detail how to use PV and PVC. Let's not delve into it here: (pv001.yaml)

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv001
  labels:
    release: stable
spec:
  capacity:
    storage: 1Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  hostPath:
    path: /tmp/data

The other one only needs to change the name to pv002, and then create:

$ kubectl create -f pv001.yaml && kubectl create -f pv002.yaml
persistentvolume "pv001" created
persistentvolume "pv002" created
$ kubectl get pv
kubectl get pv
NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM     STORAGECLASS   REASON    AGE
pv001     1Gi        RWO            Recycle          Available                                      12s
pv002     1Gi        RWO            Recycle          Available                                      11s

You can see that two PV objects have been created successfully, and the status is: Available.

Then we use it StatefulSetto create an Nginx Pod. For this type of resource, we generally Headless Serviceexpose the service by creating a type of service, which will be clusterIPset Noneas a headless service: (statefulset-demo.yaml)

apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
    role: stateful

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  serviceName: "nginx"
  replicas: 2
  selector:
    matchLabels:
      app: nginx
      role: stateful
  template:
    metadata:
      labels:
        app: nginx
        role: stateful
    spec:
      containers:
      - name: nginx
        image: cnych/nginx-slim:0.8
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: www
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi

volumeMountsNote that a new attribute is associated with the above YAML file : volumeClaimTemplates , which automatically declares a pvc object and pv for management:

Then here we open two terminal windows. In the first terminal, use kubectl get to view the creation of the StatefulSet's Pods.

$ kubectl get pods -w -l role=stateful

In another terminal, use kubectl create to create the Headless Service and StatefulSet defined in statefulset-demo.yaml.

$ kubectl create -f statefulset-demo.yaml
service "nginx" created
statefulset.apps "web" created

Check the pod's order index

For a StatefulSet with N replicas, Pods are created in the order of {0…N-1} when they are deployed. In the first terminal we can see the following information:

$ kubectl get pods -w -l role=stateful
NAME      READY     STATUS    RESTARTS   AGE
web-0     0/1       Pending   0          0s
web-0     0/1       Pending   0         0s
web-0     0/1       ContainerCreating   0         0s
web-0     1/1       Running   0         19s
web-1     0/1       Pending   0         0s
web-1     0/1       Pending   0         0s
web-1     0/1       ContainerCreating   0         0s
web-1     1/1       Running   0         18s

Note that the web-1 Pod will not be started until the web-0 Pod is in the Running and Ready states.

As mentioned in the StatefulSets concept, Pods in a StatefulSet have a stable and unique identity. This flag is based on the unique sequential index assigned to each Pod by the StatefulSet controller. Pod names are of the form <statefulset name>-<ordinal index>. The web StatefulSet has two replicas, so it creates two Pods: web-0 and web-1.

The command above creates two Pods, each running an NGINX web server. Get the nginx Service and web StatefulSet to verify that they were created successfully.

$ kubectl get service nginx
NAME      CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
nginx     None         <none>        80/TCP    12s
$ kubectl get statefulset web
NAME      DESIRED   CURRENT   AGE
web       2         1         20s

Use a stable network identity

Each Pod has a stable hostname based on its sequential index. Execute hostname in each Pod with kubectl exec.

$ for i in 0 1; do kubectl exec web-$i -- sh -c 'hostname'; done
web-0
web-1

We then use kubectl run to run a container that provides the nslookup command. By doing an nslookup on the Pods' hostnames, you can check their DNS addresses inside the cluster.

$ kubectl run -i --tty --image busybox dns-test --restart=Never --rm /bin/sh 
nslookup web-0.nginx
Server:    10.0.0.10
Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local

Name:      web-0.nginx
Address 1: 10.244.1.6

nslookup web-1.nginx
Server:    10.0.0.10
Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local

Name:      web-1.nginx
Address 1: 10.244.2.6

The CNAME of the headless service points to the SRV record (logging each Pod in the Running and Ready state). The SRV record points to a record entry that contains the Pod's IP address.

Then let's look at deleting the Pod under the StatefulSet:

View StatefulSet Pods in a terminal:

$ kubectl get pod -w -l role=stateful

Use kubectl delete in another terminal to delete all Pods in the StatefulSet.

$ kubectl delete pod -l role=stateful
pod "web-0" deleted
pod "web-1" deleted

Wait for the StatefulSet to restart them and both Pods become Running and Ready.

$ kubectl get pod -w -l app=nginx
NAME      READY     STATUS              RESTARTS   AGE
web-0     0/1       ContainerCreating   0          0s
web-0     1/1       Running   0          2s
web-1     0/1       Pending   0         0s
web-1     0/1       Pending   0         0s
web-1     0/1       ContainerCreating   0         0s
web-1     1/1       Running   0         34s

Then use kubectl exec and kubectl run again to view the hostname of the Pod and the DNS entries inside the cluster.

$ for i in 0 1; do kubectl exec web-$i -- sh -c 'hostname'; done
web-0
web-1
$ kubectl run -i --tty --image busybox dns-test --restart=Never --rm /bin/sh 
nslookup web-0.nginx
Server:    10.0.0.10
Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local

Name:      web-0.nginx
Address 1: 10.244.1.7

nslookup web-1.nginx
Server:    10.0.0.10
Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local

Name:      web-1.nginx
Address 1: 10.244.2.8

We can see that the pod's ordinal number, hostname, SRV entry, and record name have not changed, but the IP address associated with the pod may have changed. So this is why it is important not to use the IP address of the Pod in the StatefulSet to connect from other applications. In general, we can directly connect through SRV records: web-0.nginx, web-1.nginx, because they are stable, and when your Pod status changes to Running and Ready, your application can discover them the address of.

Similarly, we can check the final binding status of PV and PVC:

$ kubectl get pv
NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM               STORAGECLASS   REASON    AGE
pv001     1Gi        RWO            Recycle          Bound     default/www-web-0                            1h
pv002     1Gi        RWO            Recycle          Bound     default/www-web-1                            1h
$ kubectl get pvc
NAME        STATUS    VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
www-web-0   Bound     pv001     1Gi        RWO                           22m
www-web-1   Bound     pv002     1Gi        RWO                           22m

The use of storage volumes will be explained later, so we will not explain it here for the time being to avoid confusion.

Of course, StatefulSet also has other features. In actual projects, we still rarely go back to deploy our stateful services directly through StatefulSet, unless you can fully hold it yourself. For some specific services, we may use more Advanced Operator to deploy, such as etcd-operator, prometheus-operator, etc. These applications can manage stateful services well, instead of simply using a StatefulSet to deploy a Pod, because for stateful applications The most important thing is data recovery, failover and so on.

---