Kubernetes notes (08) - Daemonset generation reason, yaml description, usage method, definition and usage of taint and tolerance, static Pod

DaemonSet, which runs one on each node of Kubernetesthe cluster Pod, Linuxlike a "daemon" ( Daemon) in the system.

1. Cause

Deployment, it can create any number of Podinstances and maintain Podthe normal operation of these to ensure that the application is always available.

However, Deploymentit doesn't care which nodes of the cluster these Podwill run on. From its point of view, Podthe operating environment and functions of s are irrelevant. As long Podas the number of s is sufficient, the application should work normally.

This assumption is no problem for most businesses, for example Nginx, , WordPress, MySQL, they do not need to know the details of the cluster and nodes, as long as the environment variables and storage volumes are configured, it is the same wherever they "run".

However, there are some special services that are not completely independent of the system, but have a "binding" relationship with the host, and must be attached to the node to generate value, for example:

• A network application such as kube-proxy, must run one on each node Pod, otherwise the node cannot join Kubernetesthe network .
• For monitoring applications (such as Prometheus), each node must have a node Podto monitor the status of the node and report information in real time.
• A log application (such as Fluentd), must run one on each node Podto be able to collect log data generated when the container is running.
• For security applications, similarly, each node must have one Podto perform security audits, intrusion checks, and vulnerability scans.

These services are not suitable Deploymentfor deployment, because the numberDeployment managed by is fixed and may "drift" in the cluster, but the actual requirement is to run on every node in the cluster , and That is to say, the number of is kept in sync with the number of nodes.PodPodPod

Therefore, Kubernetesa new APIobject DaemonSet, which is Deploymentsimilar is both management control Pod, but the management scheduling strategy is different.

DaemonSetThe goal is to run on each node of the cluster and only run one Pod, as if a node is equipped with a "watchdog" to faithfully "guard" the node, which is the origin of DaemonSetthe name .

2. YAML description

DaemonSetDeploymentand belong to online business, so they are also appsgroup, you kubectl api-resourcescan use the command to know its abbreviation isds

$ kubectl api-resources | grep daemonset
daemonsets                        ds           apps/v1                                true         DaemonSet

YAMLThe file header information should be:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: xxx-ds

kubectlDoes not provide the function of automatically creating DaemonSet YAMLtemplates , that is, we cannot use the command kubectl createto directly create a DaemonSetobject .

However, you Kubernetescan find an example of the official website of ( https://kubernetes.io/zh-cn/docs/concepts/workloads/controllers/daemonset/ ) , copy it, and then remove the redundant part, you can Make your own boilerplate file, which probably looks like this:DaemonSetYAML

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: redis-ds
  labels:
    app: redis-ds

spec:
  selector:
    matchLabels:
      name: redis-ds

  template:
    metadata:
      labels:
        name: redis-ds
    spec:
      containers:
      - image: redis:5-alpine
        name: redis
        ports:
        - containerPort: 6379

YAMLSimply compare this Deploymentobject with the object in the previous lesson, and you will find that:

• The previous part kindis metadatathe unique information of the object, which is naturally different, but specthe part DaemonSetalso has selectorthe field , which matches the tag intemplate the , which is almost exactly the same as the object .PodlabelsDeployment

• However, there is no field inDaemonSet , which is a key difference between it and , which means that it will not create multiple copies of in the cluster, but only one instance .specreplicasDeploymentPodPod

That is to say, DaemonSetit is only different from in Podits deployment scheduling strategy Deployment, and everything else is the same. To some extent, we can also DaemonSetregard as Deploymenta special case of .

The difference between the two:

Knowing these differences, now we can use a workaround to DaemonSetcreate YAMLthe template of the , you only need to use kubectl createto create a Deploymentobject first, then kindchange to DaemonSet, and spec.replicasthen , for example:

export out="--dry-run=client -o yaml"

# change "kind" to DaemonSet
kubectl create deploy redis-ds --image=redis:5-alpine $out

3. Use DaemonSet

Execute the command kubectl apply, YAMLsend to Kubernetes, let it create DaemonSetthe object , and then use kubectl getto view the status of the object:

Although we didn't DaemonSetspecify Podthe number to run in , it will find the nodes in the cluster and create them in the nodes Pod. Because there is Masterone Worker, and does not run the application Masterby default , so DaemonSetonly one is generated Podand runs on workerthe node .

According DaemonSetto the original intention of , an Podinstance of should be running on each node, but Masterthe node is excluded, which is not in line with our original idea.

Obviously, DaemonSetthe duty of "gatekeeper" has not been fulfilled, and its design conflicts with the working mechanism of Kubernetes the cluster . Is there any way to solve it?

Of course, Kubernetesthis has been thought of for a long time. In order to deal with the "scheduling" and "eviction" problems Podat certain nodes, it defines two new concepts: taint ( taint) and tolerance ( toleration).

4. Stains and Tolerances

"Taint" and "tolerance" are two important concepts DaemonSetrelated , which belong to Nodeand respectively Pod, and jointly determine the scheduling strategy Podof .

• "Taint" is an attribute of a Kubernetesnode , and its function is also to "label" the node, but in order not to be labelsconfused with the existing fields, it was changed taint.

• The opposite of "stain" is Podyour "tolerance", as the name suggests, is Podwhether you can "tolerate" stains.

It's easier to understand if we put the two together. There are all kinds of nodes in the cluster. Some nodes are "pure and flawless" without "stains"; while some nodes have "stains" because of "mud" stuck to them for some reason. PodThey also have different tempers. Some have serious "cleanliness" and can't tolerate "stains" and can only choose "clean" nodes; while Podothers are less demanding, and can tolerate some small "stains" appropriately. ".

KubernetesWhen creating a cluster, it will automatically Nodeadd to facilitate Podscheduling and deployment. You can use kubectl describe nodeto view the status ofMaster and :Worker

It can be seen that Masterthe node has one by default taint, the name is node-role.kubernetes.io/master, and its effect is NoSchedule, that is to say, the taint will refuse Podto be scheduled to run on this node, and the fieldWorker of the node is empty.taint

This is exactly the difference between Masterand Workerin the Podscheduling strategy. Generally speaking Pod, cannot tolerate any "taint", so the node with taintthe attribute Masterwill Podbe .

How to DaemonSetmake Masterit run on the node (or any other node), there are two ways.

4.1 Remove the taint on the Master node

The first method is to remove Masterthe node taint, so that Masterbecomes as Worker"pure and flawless" as , and DaemonSetnaturally there is no need to distinguish Master/Worker.

NodeThe "taint" attribute on the operation needs to use the command kubectl taint, and then specify the node name, taint name and the effect of the taint, and an additional one is added to remove the taint -.

For example, to Masterremove NoSchedulethe effect of the node, use this command:

$ kubectl taint node pylot  node-role.kubernetes.io/master:NoSchedule-

Because DaemonSetit has been monitoring the status of the cluster nodes, after the command is executed Master, the node has no "taint", so it will immediately find the change, and then create a "guard" on Masterthe node Pod. You can use kubectl getto see this change:

However, this method modifies the state Nodeof , and the impact area will be relatively large, which may cause many Podto run on this node, so we can keep the "taint" Nodeof , Podadd "tolerance" for the required ones, and only let some PodRun on individual nodes to achieve "fine-grained" scheduling.

4.2 Add tolerance to Pod

PodAdd fields tolerationsto make it "tolerant" to certain "taints", and it can run on any node.

tolerationsIt is an array, which can list multiple "stains" that are "tolerated", and the name and effect of the "stains" need to be clearly written. The comparison especially uses operatorthe field to specify how to match the "taint", which is generally used by us Exists, that is to say, there is a "taint" with this name and effect.

If we want DaemonSetthe in Podto be able to run on Masterthe node , we have to write this one tolerations, which tolerates node-role.kubernetes.io/master:NoSchedulethis :

tolerations:
- key: node-role.kubernetes.io/master
  effect: NoSchedule
  operator: Exists

The complete yml file content is as follows:

apiVersion: apps/v1
kind: DaemonSet 
metadata:
  labels:
    app: mq-ds
  name: mq-ds
  
spec:
  selector:
    matchLabels:
      app: mq-ds
      
  template:
    metadata:
      labels:
        app: mq-ds
    spec:
      containers:
      - image: rabbitmq:latest
        name: rabbitmq
      tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
        operator: Exists

Now we first use kubectl taintthe command to Masteradd the "stain" of :

kubectl taint node pylot  node-role.kubernetes.io/master:NoSchedule

Delete the daemonset.

Then we redeployed with "tolerance" DaemonSet:

You will see that there are DaemonSetstill two Pod, running on the Masterand Workernodes respectively, which has the same effect as the first method.

It needs to be specially explained that "tolerance" is not a DaemonSetunique concept, but subordinate to it Pod, so after understanding "taint" and "tolerance", you can also add it to manage them Job/CronJobin , so as to be more accurate. Flexible scheduling of applications.DeploymentPodtolerations

As for the stains and the effects of the stains, I won’t go into details. The Kubernetesofficial website documents ( https://kubernetes.io/zh-cn/docs/concepts/scheduling-eviction/taint-and-toleration/ ) are listed very clear.

5. Static Pods

DaemonSetIt is the most common way to run node-specific Kubernetesin Pod, but it is not the only way. It Kubernetesalso supports another application deployment method called "static Pod".

"Static Pod" is very special. It is not controlled by Kubernetesthe system and has no relationship with apiserverand scheduler, so it is "static".

But since it is Pod, it must "run" on the container runtime, and there will be YAMLfiles to describe it, and the only Kubernetescomponent that can manage it kubeletis .

YAMLThe files of "Static Pod" are stored in /etc/kubernetes/manifeststhe directory of the node by default, which is a dedicated directory Kubernetesof .

The following is the situation of the directory in Masterthe node :

$ sudo ls -l /etc/kubernetes/manifests
总用量 16
-rw------- 1 root root 2242 2月   3 09:03 etcd.yaml
-rw------- 1 root root 4038 2月   3 09:03 kube-apiserver.yaml
-rw------- 1 root root 3543 2月   3 09:03 kube-controller-manager.yaml
-rw------- 1 root root 1464 2月   3 09:03 kube-scheduler.yaml

You can see that Kubernetesthe 4 core components apiserver, etcd, scheduler, controller-manageroriginally existed in a Podstatic form, which is why they can be Kubernetes started before the cluster.

If you have some special needs DaemonSetthat cannot be met, you can consider using static Pod, write a YAMLfile put it in this directory, and the node kubeletwill periodically check the files in the directory, and if any changes are found, it will call the container runtime to create or delete the static Pod.

Static can Podalso achieve DaemonSetthe same effect as and , but it is not Kubernetescontrolled by and must be deployed purely manually on the node, so it should be used with caution.

The network plug-in Flannel is a DaemonSet, which is located in the namespace kube-system and can be seen with kubectl get ds -n kube-system.

Another concept related to "taint" and "tolerance" is "node affinity", which prefers which node to choose, but the usage is a little more complicated.

In Kubernetes v1.24, the master node will no longer use the taint node-role.kuberneters.io/master but node-role.kuberneters.io/control-plane.

6. Questions

1. If a taint nodeis added to , podwill the taint running on it that cannot tolerate the taint automatically run nodeto .

Answer: If it has been scheduled, running on the node podwill not care taintabout , because it taintonly takes effect when scheduling, and you can also try it yourself.
Test conclusion: When mastera node removes taint, podit will be dispatched master; masterwhen a node adds taint, podit will not leave, and will not increase after manual deletion.
Taint tolerance configuration path location:

kubectl explain ds.spec.template.spec.tolerations

2. Why is the tolerance in the field podof and not dson.

Answer: Because the unit of Kubernetesscheduling podis DaemonSetjust management pod.