Kubernetes two Kubernetes actual combat and pod detailed explanation

Getting Started with Kubernetes

A Kubernetes combat

This chapter will introduce how to deploy an nginx service in a kubernetes cluster and be able to access it.

1.1 Namespace

Namespace is a very important resource in the kubernetes system. Its main function is to realize resource isolation of multiple environments or resource isolation of multi-tenants .

By default, all Pods in a kubernetes cluster are mutually accessible. But in practice, you may not want to allow two Pods to access each other, then you can divide the two Pods into different namespaces. Kubernetes can form logical "groups" by allocating resources within the cluster to different Namespaces, so as to facilitate the isolated use and management of resources in different groups.

Through the kubernetes authorization mechanism, different namespaces can be handed over to different tenants for management, thus realizing multi-tenant resource isolation. At this time, the resource quota mechanism of kubernetes can also be combined to limit the resources that different tenants can occupy, such as CPU usage, memory usage, etc., to realize the management of tenants' available resources.

After the cluster is started, kubernetes will create the following four namespaces by default

[root@master ~]# kubectl  get namespace
NAME              STATUS   AGE
default           Active   45h     #  所有未指定Namespace的对象（pod）都会被分配在default命名空间
kube-node-lease   Active   45h     #  集群节点之间的心跳维护，v1.13开始引入
kube-public       Active   45h     #  此命名空间下的资源可以被所有人访问（包括未认证用户）
kube-system       Active   45h     #  所有由Kubernetes系统创建的资源都处于这个命名空间

Let's look at the specific operations of namespace resources:

Command mode to operate

Check

# 1 查看所有的ns  命令：kubectl get ns
[root@master ~]# kubectl get ns
NAME              STATUS   AGE
default           Active   45h
kube-node-lease   Active   45h
kube-public       Active   45h     
kube-system       Active   45h     

# 2 查看指定的ns   命令：kubectl get ns ns名称
[root@master ~]# kubectl get ns default
NAME      STATUS   AGE
default   Active   45h

# 3 指定输出格式  命令：kubectl get ns ns名称  -o 格式参数
# kubernetes支持的格式有很多，比较常见的是wide、json、yaml
[root@master ~]# kubectl get ns default -o yaml
apiVersion: v1
kind: Namespace
metadata:
  creationTimestamp: "2021-05-08T04:44:16Z"
  name: default
  resourceVersion: "151"
  selfLink: /api/v1/namespaces/default
  uid: 7405f73a-e486-43d4-9db6-145f1409f090
spec:
  finalizers:
  - kubernetes
status:
  phase: Active
  
# 4 查看ns详情  命令：kubectl describe ns ns名称
[root@master ~]# kubectl describe ns default
Name:         default
Labels:       <none>
Annotations:  <none>
Status:       Active  # Active：命名空间正在使用中  Terminating：正在删除命名空间

# ResourceQuota 针对namespace做的资源限制
# LimitRange针对namespace中的每个组件做的资源限制
No resource quota.
No LimitRange resource.

create

# 创建namespace
[root@master ~]# kubectl create ns dev
namespace/dev created

delete

# 删除namespace
[root@master ~]# kubectl delete ns dev
namespace "dev" deleted

Configure to operate

First prepare a yaml file: ns-dev.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: dev

Then you can execute the corresponding create and delete commands:

Create: kubectl create -f ns-dev.yaml

Delete: kubectl delete -f ns-dev.yaml

1.2 Pod

Pod is the smallest unit of kubernetes cluster management. To run a program, it must be deployed in a container (the program runs in a container), and the container must exist in a Pod.

Pod can be regarded as the package of containers, and one or more containers can exist in a Pod.

In the figure below, the bottom container is the root container, and the other containers are user containers (will be explained in detail in Chapter 2)

After the kubernetes cluster is started, each component in the cluster also runs as a Pod. You can view it with the following command:

[root@master ~]# kubectl get pod -n kube-system
NAMESPACE     NAME                             READY   STATUS    RESTARTS   AGE
kube-system   coredns-6955765f44-68g6v         1/1     Running   0          2d1h
kube-system   coredns-6955765f44-cs5r8         1/1     Running   0          2d1h
kube-system   etcd-master                      1/1     Running   0          2d1h
kube-system   kube-apiserver-master            1/1     Running   0          2d1h
kube-system   kube-controller-manager-master   1/1     Running   0          2d1h
kube-system   kube-flannel-ds-amd64-47r25      1/1     Running   0          2d1h
kube-system   kube-flannel-ds-amd64-ls5lh      1/1     Running   0          2d1h
kube-system   kube-proxy-685tk                 1/1     Running   0          2d1h
kube-system   kube-proxy-87spt                 1/1     Running   0          2d1h
kube-system   kube-scheduler-master            1/1     Running   0          2d1h

Create and run pods

kubernetes does not provide commands to run Pods individually, they are all implemented through Pod controllers

# 命令格式： kubectl run (pod控制器名称) [参数] 
# --image  指定Pod的镜像
# --port   指定端口
# --namespace  指定namespace
#下边第一个nginx指的是pod名称
[root@master ~]# kubectl run nginx --image=nginx:latest --port=80 --namespace dev 
deployment.apps/nginx created

View pod information

# 查看Pod基本信息
[root@master ~]# kubectl get pods -n dev
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          43s

# 查看Pod的详细信息，pod创建失败的话，在这个命令输出的最下边会有提示信息
[root@master ~]# kubectl describe pod nginx -n dev
Name:         nginx
Namespace:    dev
Priority:     0
Node:         node1/192.168.5.4
Start Time:   Wed, 08 May 2021 09:29:24 +0800
Labels:       pod-template-hash=5ff7956ff6
              run=nginx
Annotations:  <none>
Status:       Running
IP:           10.244.1.23
IPs:
  IP:           10.244.1.23
Controlled By:  ReplicaSet/nginx
Containers:
  nginx:
    Container ID:   docker://4c62b8c0648d2512380f4ffa5da2c99d16e05634979973449c98e9b829f6253c
    Image:          nginx:latest
    Image ID:       docker-pullable://nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Wed, 08 May 2021 09:30:01 +0800
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-hwvvw (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  default-token-hwvvw:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-hwvvw
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age        From               Message
  ----    ------     ----       ----               -------
  Normal  Scheduled  <unknown>  default-scheduler  Successfully assigned dev/nginx-5ff7956ff6-fg2db to node1
  Normal  Pulling    4m11s      kubelet, node1     Pulling image "nginx:latest"
  Normal  Pulled     3m36s      kubelet, node1     Successfully pulled image "nginx:latest"
  Normal  Created    3m36s      kubelet, node1     Created container nginx
  Normal  Started    3m36s      kubelet, node1     Started container nginx

Access Pods

# 获取podIP
# 下边的READY列指的是有几个容器（不包含根容器Pause），且容器里有几个是正在运行着的
# RESTARTS：重启次数
# IP：当前pod的ip（以后要用这个ip访问里边的容器下的程序），每次创建pod时，这个ip都不一样，即ip不稳定，后边会介绍怎么解决
# Node：被调到哪个node去运行了
[root@master ~]# kubectl get pods -n dev -o wide
NAME    READY   STATUS    RESTARTS   AGE    IP             NODE    ... 
nginx   1/1     Running   0          190s   10.244.1.23   node1   ...

#访问POD
[root@master ~]# curl http://10.244.1.23:80
<!DOCTYPE html>
<html>
<head>
	<title>Welcome to nginx!</title>
</head>
<body>
	<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Delete the specified Pod

# 删除指定Pod
[root@master ~]# kubectl delete pod nginx -n dev
pod "nginx" deleted

# 此时，显示删除Pod成功，但是再查询，发现又新产生了一个 
[root@master ~]# kubectl get pods -n dev
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          21s

# 这是因为当前Pod是由Pod控制器创建的（因为我们使用的是kubectl run创建的），控制器会监控Pod状况，一旦发现Pod死亡，会立即重建
# 此时要想删除Pod，必须删除Pod控制器

# 先来查询一下当前namespace下的Pod控制器
[root@master ~]# kubectl get deploy -n  dev
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   1/1     1            1           9m7s

# 接下来，删除此PodPod控制器
[root@master ~]# kubectl delete deploy nginx -n dev
deployment.apps "nginx" deleted

# 稍等片刻，再查询Pod，发现Pod被删除了
[root@master ~]# kubectl get pods -n dev
No resources found in dev namespace.

configuration operation

Create a pod-nginx.yaml with the following content:

# 注意，此时起的是单独的pod，不是pod控制器
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  namespace: dev
spec:
  containers:
  - image: nginx:latest # -指的是数组
    name: pod
    ports:
    - name: nginx-port
      containerPort: 80
      protocol: TCP

Then you can execute the corresponding create and delete commands:

Create: kubectl create -f pod-nginx.yaml

Delete: kubectl delete -f pod-nginx.yaml

1.3 Label

Label is an important concept in the kubernetes system. Its role is to add identification to resources to distinguish and select them.

Label features:

• A Label will be attached to various objects in the form of key/value key-value pairs, such as Node, Pod, Service, etc.
• A resource object can define any number of Labels, and the same Label can also be added to any number of resource objects
• Label is usually determined when the resource object is defined, of course, it can also be dynamically added or deleted after the object is created

Multi-dimensional grouping of resources can be realized through Label, so that resource allocation, scheduling, configuration, deployment and other management work can be performed flexibly and conveniently.

Some commonly used Label examples are as follows:

• Version tags: "version": "release", "version": "stable"…
• Environment tags: "environment": "dev", "environment": "test", "environment": "pro"
• Schema tags: "tier": "frontend" (foreground), "tier": "backend" (backend)

After the label is defined, the selection of the label must also be considered, which requires the use of the Label Selector, namely:

Label is used to define an identifier for a resource object

Label Selector is used to query and filter resource objects with certain labels

There are currently two Label Selectors:

• Equation-based Label Selector

  name = slave: Select all objects containing key="name" and value="slave" in Label

  env != production: select all objects including key="env" in Label and value not equal to "production"

• Set-based Label Selector

  name in (master, slave): Select all objects containing key="name" and value="master" or "slave" in Label

  name not in (frontend): Select all objects that contain the key="name" in the Label and the value is not equal to "frontend"

Multiple label selection criteria can be used. In this case, multiple Label Selectors can be combined and separated by commas ",". For example:

name=slave，env!=production

name not in (frontend)，env!=production

command mode

# 为pod资源打标签  标签名字是：version=1.0
[root@master ~]# kubectl label pod nginx-pod version=1.0 -n dev
pod/nginx-pod labeled

# 为pod资源更新标签
[root@master ~]# kubectl label pod nginx-pod version=2.0 -n dev --overwrite
pod/nginx-pod labeled

# 查看标签
[root@master ~]# kubectl get pod nginx-pod  -n dev --show-labels
NAME        READY   STATUS    RESTARTS   AGE   LABELS
nginx-pod   1/1     Running   0          10m   version=2.0

# 筛选标签
[root@master ~]# kubectl get pod -n dev -l version=2.0  --show-labels
NAME        READY   STATUS    RESTARTS   AGE   LABELS
nginx-pod   1/1     Running   0          17m   version=2.0
[root@master ~]# kubectl get pod -n dev -l version!=2.0 --show-labels
No resources found in dev namespace.

#删除标签  version-: 代表删除version标签
[root@master ~]# kubectl label pod nginx-pod version- -n dev
pod/nginx-pod labeled

configuration method

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  namespace: dev
  labels:
    version: "3.0" 
    env: "test"
spec:
  containers:
  - image: nginx:latest
    name: pod
    ports:
    - name: nginx-port
      containerPort: 80
      protocol: TCP

Then you can execute the corresponding update command: kubectl apply -f pod-nginx.yaml

1.4 Deployment

In kubernetes, pod is the smallest control unit, but kubernetes seldom directly controls pods, usually through the pod controller. The Pod controller is used for pod management to ensure that the pod resources meet the expected state. When the pod resources fail, it will try to restart or rebuild the pod. .

There are many types of Pod controllers in kubernetes, and this chapter only introduces one: Deployment.

The relationship between pod and Deployment is established through label;

command operation

# 创建deployment，名称为nginx,并且创建deployment下的pod
# 命令格式: kubectl create deployment 名称  [参数] 
# --image  指定pod的镜像
# --port   指定端口
# --replicas  指定创建pod数量，默认1个
# --namespace  指定namespace
[root@master ~]# kubectl create deploy nginx --image=nginx:latest --port=80 --replicas=3 -n dev
deployment.apps/nginx created

# 查看创建的Pod
[root@master ~]# kubectl get pods -n dev
NAME                     READY   STATUS    RESTARTS   AGE
nginx-5ff7956ff6-6k8cb   1/1     Running   0          19s
nginx-5ff7956ff6-jxfjt   1/1     Running   0          19s
nginx-5ff7956ff6-v6jqw   1/1     Running   0          19s

# 查看deployment的信息
[root@master ~]# kubectl get deploy -n dev
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   3/3     3            3           2m42s

# UP-TO-DATE：成功升级的副本数量
# AVAILABLE：可用副本的数量
[root@master ~]# kubectl get deploy -n dev -o wide
NAME    READY UP-TO-DATE  AVAILABLE   AGE     CONTAINERS   IMAGES              SELECTOR
nginx   3/3     3         3           2m51s   nginx        nginx:latest        run=nginx

# 查看deployment的详细信息
[root@master ~]# kubectl describe deploy nginx -n dev
Name:                   nginx
Namespace:              dev
CreationTimestamp:      Wed, 08 May 2021 11:14:14 +0800
Labels:                 run=nginx
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               run=nginx
Replicas:               3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  run=nginx
  Containers:
   nginx:
    Image:        nginx:latest
    Port:         80/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   nginx-5ff7956ff6 (3/3 replicas created)
Events:
  Type    Reason             Age    From                   Message
  ----    ------             ----   ----                   -------
  Normal  ScalingReplicaSet  5m43s  deployment-controller  Scaled up replicaset nginx-5ff7956ff6 to 3
  
# 删除 
[root@master ~]# kubectl delete deploy nginx -n dev
deployment.apps "nginx" deleted

Check that the namespace name is the deployment and pod under dev

kubectl get deployment,pods -n dev

configuration operation

Create a deploy-nginx.yaml with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: dev
spec:
  replicas: 3 # pod个数
  selector:
    matchLabels:
      run: nginx #标签名，与下边pod的的labels一致时，Deployment才能与下边的pod挂钩
  template: #pod模版，下边就是pod信息
    metadata:
      labels:
        run: nginx
    spec:
      containers:
      - image: nginx:latest
        name: nginx
        ports:
        - containerPort: 80
          protocol: TCP

Then you can execute the corresponding create and delete commands:

Create: kubectl create -f deploy-nginx.yaml

Delete: kubectl delete -f deploy-nginx.yaml

1.5 Service

Through the study in the previous chapter, you have been able to use Deployment to create a set of Pods to provide services with high availability.

Although each Pod will be assigned a separate Pod IP, there are two problems as follows:

• The Pod IP will change as the Pod is rebuilt (the pod will be rebuilt automatically after it hangs, and the ip of the pod will change after the rebuild)
• Pod IP is only a virtual IP visible in the cluster and cannot be accessed externally

This makes it difficult to access the service. Therefore, kubernetes designed Service to solve this problem.

Service can be regarded as an external access interface for a group of Pods of the same type . With Service, applications can easily implement service discovery and load balancing.

Operation 1: Create a Service accessible within the cluster

# 暴露Service  service的80端口与pod的80端口做映射 --name：service名称  --type：类型有很多，后边会讲
[root@master ~]# kubectl expose deploy nginx --name=svc-nginx1 --type=ClusterIP --port=80 --target-port=80 -n dev
service/svc-nginx1 exposed

# 查看service  查看所有srvice：kubectl get svc  -n dev
[root@master ~]# kubectl get svc svc-nginx1 -n dev -o wide
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE     SELECTOR
svc-nginx1   ClusterIP   10.109.179.231   <none>        80/TCP    3m51s   run=nginx

# 这里产生了一个CLUSTER-IP，这就是service的IP，在Service的生命周期中，这个地址是不会变动的
# 可以通过这个IP访问当前service对应的POD
[root@master ~]# curl 10.109.179.231:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
</head>
<body>
<h1>Welcome to nginx!</h1>
.......
</body>
</html>

Operation 2: Create a Service that can also be accessed outside the cluster

# 上面创建的Service的type类型为ClusterIP，这个ip地址只用集群内部可访问
# 如果需要创建外部也可以访问的Service，需要修改type为NodePort
[root@master ~]# kubectl expose deploy nginx --name=svc-nginx2 --type=NodePort --port=80 --target-port=80 -n dev
service/svc-nginx2 exposed

# 此时查看，会发现出现了NodePort类型的Service，而且有一对Port（80:31928/TC）
[root@master ~]# kubectl get svc  svc-nginx2  -n dev -o wide
NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE    SELECTOR
svc-nginx2    NodePort    10.100.94.0      <none>        80:31928/TCP   9s     run=nginx

# 接下来就可以通过集群外的主机访问 节点IP:31928访问服务了
# 例如在的电脑主机上通过浏览器访问下面的地址
http://192.168.5.4:31928/

Delete Service

[root@master ~]# kubectl delete svc svc-nginx-1 -n dev service "svc-nginx-1" deleted

configuration method

Create a svc-nginx.yaml with the following content:

apiVersion: v1
kind: Service
metadata:
  name: svc-nginx
  namespace: dev
spec:
  clusterIP: 10.109.179.231 #固定svc的内网ip
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: ClusterIP

Then you can execute the corresponding create and delete commands:

Create: kubectl create -f svc-nginx.yaml

Delete: kubectl delete -f svc-nginx.yaml

summary

So far, you have mastered the basic operations of Namespace, Pod, Deployment, and Service resources. With these operations, you can implement simple deployment and access of a service in a kubernetes cluster, but if you want to use kubernetes better, you need Learn the details and rationale of each of these resources in depth.

Two Pod detailed explanation

2.1 Pod introduction

2.1.1 Pod structure

Each Pod can contain one or more containers, which can be divided into two categories:

• The number of containers where user programs reside can be large or small

• The Pause container, which is a root container that every Pod will have , has two functions:

  • It can be used as a basis to evaluate the health status of the entire Pod

  • You can set the Ip address on the root container, and other containers share this Ip (Pod IP) to realize network communication inside the Pod (the outside world can also access each container in the pod through this IP plus different ports)

    这里是Pod内部的通讯，Pod的之间的通讯采用虚拟二层网络技术来实现，我们当前环境用的是Flannel
    

2.1.2 Pod definition

The following is the resource list of Pod:

apiVersion: v1     #必选，版本号，例如v1
kind: Pod       　 #必选，资源类型，例如 Pod
metadata:       　 #必选，元数据
  name: string     #必选，Pod名称
  namespace: string  #Pod所属的命名空间,默认为"default"
  labels:       　　  #自定义标签列表
    - name: string      　          
spec:  #必选，Pod中容器的详细定义
  containers:  #必选，Pod中容器列表
  - name: string   #必选，容器名称
    image: string  #必选，容器的镜像名称
    imagePullPolicy: [ Always|Never|IfNotPresent ]  #获取镜像的策略 
    command: [string]   #容器的启动命令列表，如不指定，使用打包时使用的启动命令
    args: [string]      #容器的启动命令参数列表
    workingDir: string  #容器的工作目录
    volumeMounts:       #挂载到容器内部的存储卷配置
    - name: string      #引用pod定义的共享存储卷的名称，需用volumes[]部分定义的的卷名
      mountPath: string #存储卷在容器内mount的绝对路径，应少于512字符
      readOnly: boolean #是否为只读模式
    ports: #需要暴露的端口库号列表
    - name: string        #端口的名称
      containerPort: int  #容器需要监听的端口号
      hostPort: int       #容器所在主机需要监听的端口号，默认与Container相同
      protocol: string    #端口协议，支持TCP和UDP，默认TCP
    env:   #容器运行前需设置的环境变量列表
    - name: string  #环境变量名称
      value: string #环境变量的值
    resources: #资源限制和请求的设置
      limits:  #资源限制的设置
        cpu: string     #Cpu的限制，单位为core数，将用于docker run --cpu-shares参数
        memory: string  #内存限制，单位可以为Mib/Gib，将用于docker run --memory参数
      requests: #资源请求的设置
        cpu: string    #Cpu请求，容器启动的初始可用数量
        memory: string #内存请求,容器启动的初始可用数量
    lifecycle: #生命周期钩子
        postStart: #容器启动后立即执行此钩子,如果执行失败,会根据重启策略进行重启
        preStop: #容器终止前执行此钩子,无论结果如何,容器都会终止
    livenessProbe:  #对Pod内各容器健康检查的设置，当探测无响应几次后将自动重启该容器
      exec:       　 #对Pod容器内检查方式设置为exec方式
        command: [string]  #exec方式需要制定的命令或脚本
      httpGet:       #对Pod内个容器健康检查方法设置为HttpGet，需要制定Path、port
        path: string
        port: number
        host: string
        scheme: string
        HttpHeaders:
        - name: string
          value: string
      tcpSocket:     #对Pod内个容器健康检查方式设置为tcpSocket方式
         port: number
       initialDelaySeconds: 0       #容器启动完成后首次探测的时间，单位为秒
       timeoutSeconds: 0    　　    #对容器健康检查探测等待响应的超时时间，单位秒，默认1秒
       periodSeconds: 0     　　    #对容器监控检查的定期探测时间设置，单位秒，默认10秒一次
       successThreshold: 0
       failureThreshold: 0
       securityContext:
         privileged: false
  restartPolicy: [Always | Never | OnFailure]  #Pod的重启策略
  nodeName: <string> #设置NodeName表示将该Pod调度到指定到名称的node节点上
  nodeSelector: obeject #设置NodeSelector表示将该Pod调度到包含这个label的node上
  imagePullSecrets: #Pull镜像时使用的secret名称，以key：secretkey格式指定
  - name: string
  hostNetwork: false   #是否使用主机网络模式，默认为false，如果设置为true，表示使用宿主机网络
  volumes:   #在该pod上定义共享存储卷列表
  - name: string    #共享存储卷名称 （volumes类型有很多种）
    emptyDir: {
    
    }       #类型为emtyDir的存储卷，与Pod同生命周期的一个临时目录。为空值
    hostPath: string   #类型为hostPath的存储卷，表示挂载Pod所在宿主机的目录
      path: string      　　        #Pod所在宿主机的目录，将被用于同期中mount的目录
    secret:       　　　#类型为secret的存储卷，挂载集群与定义的secret对象到容器内部
      scretname: string  
      items:     
      - key: string
        path: string
    configMap:         #类型为configMap的存储卷，挂载预定义的configMap对象到容器内部
      name: string
      items:
      - key: string
        path: string

#小提示：
#   在这里，可通过一个命令来查看每种资源的可配置项
#   kubectl explain 资源类型         查看某种资源可以配置的一级属性
#   kubectl explain 资源类型.属性     查看属性的子属性
[root@k8s-master01 ~]# kubectl explain pod
KIND:     Pod
VERSION:  v1
FIELDS:
   apiVersion   <string>
   kind <string>
   metadata     <Object>
   spec <Object>
   status       <Object>

[root@k8s-master01 ~]# kubectl explain pod.metadata
KIND:     Pod
VERSION:  v1
RESOURCE: metadata <Object>
FIELDS:
   annotations  <map[string]string>
   clusterName  <string>
   creationTimestamp    <string>
   deletionGracePeriodSeconds   <integer>
   deletionTimestamp    <string>
   finalizers   <[]string>
   generateName <string>
   generation   <integer>
   labels       <map[string]string>
   managedFields        <[]Object>
   name <string>
   namespace    <string>
   ownerReferences      <[]Object>
   resourceVersion      <string>
   selfLink     <string>
   uid  <string>

In kubernetes, the first-level attributes of basically all resources are the same, mainly including 5 parts:

• apiVersion version, defined internally by kubernetes, the version number must be queried with kubectl api-versions
• kind type, defined internally by kubernetes, the version number must be queried with kubectl api-resources
• metadata Metadata, mainly resource identification and description, commonly used are name, namespace, labels, etc.
• Spec detailed description, which is the most important part of the configuration, contains detailed descriptions of various resource configurations
• status status information, the content inside does not need to be defined, it is automatically generated by kubernetes

Among the above attributes, spec is the focus of the next research, continue to look at its common sub-attributes:

• containers <[]Object> container list, used to define the details of the container
• nodeName dispatches the pod to the specified Node node according to the value of nodeName
• nodeSelector <map[]> According to the information defined in NodeSelector, select to schedule the Pod to the Node containing these labels
• hostNetwork Whether to use the host network mode, the default is false, if set to true, it means to use the host network
• volumes <[]Object> Storage volume, used to define the storage information mounted on the Pod
• restartPolicy Restart strategy, indicating the processing strategy of the Pod when it encounters a failure

2.2 Pod configuration

This section mainly studies pod.spec.containersattributes, which is also the most critical configuration in pod configuration.

[root@k8s-master01 ~]# kubectl explain pod.spec.containers
KIND:     Pod
VERSION:  v1
RESOURCE: containers <[]Object>   # 数组，代表可以有多个容器
FIELDS:
   name  <string>     # 容器名称
   image <string>     # 容器需要的镜像地址
   imagePullPolicy  <string> # 镜像拉取策略 
   command  <[]string> # 容器的启动命令列表，如不指定，使用打包时使用的启动命令
   args     <[]string> # 容器的启动命令需要的参数列表
   env      <[]Object> # 容器环境变量的配置
   ports    <[]Object>     # 容器需要暴露的端口号列表
   resources <Object>      # 资源限制和资源请求的设置

2.2.1 Basic configuration

Create a pod-base.yaml file with the following content:

apiVersion: v1
kind: Pod
metadata:
  name: pod-base
  namespace: dev
  labels:
    user: heima
spec:
  containers: #定义两个容器，一个是nginx，一个是busybox
  - name: nginx  # 前边有-，代表下边有相同的配置
    image: nginx:1.17.1 # 本地没有该镜像的话，会自动去远程仓库下载（具体下载策略下边会讲）
  - name: busybox # 一个小的linux命令集合工具
    image: busybox:1.30

The configuration of a relatively simple Pod is defined above, which contains two containers:

• nginx: Created with the nginx image of version 1.17.1, (nginx is a lightweight web container)
• busybox: Created with the busybox image version 1.30, (busybox is a small collection of linux commands)

# 创建Pod
[root@k8s-master01 pod]# kubectl apply -f pod-base.yaml
pod/pod-base created

# 查看Pod状况
# READY 1/2 : 表示当前Pod中有2个容器，其中1个准备就绪，1个未就绪
# RESTARTS  : 重启次数，因为有1个容器故障了，Pod一直在重启试图恢复它（下边的重启次数是4了）
[root@k8s-master01 pod]# kubectl get pod -n dev
NAME       READY   STATUS    RESTARTS   AGE
pod-base   1/2     Running   4          95s

# 可以通过describe查看内部的详情
# 此时已经运行起来了一个基本的Pod，虽然它暂时有问题
[root@k8s-master01 pod]# kubectl describe pod pod-base -n dev

2.2.2 Image pull

Create a pod-imagepullpolicy.yaml file with the following content:

apiVersion: v1
kind: Pod
metadata:
  name: pod-imagepullpolicy #不能有大写字母
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
    imagePullPolicy: Never # 用于设置镜像拉取策略（never：一直用本地镜像）
  - name: busybox
    image: busybox:1.30

imagePullPolicy, used to set the image pull policy, kubernetes supports configuration of three pull policies:

• Always: Always pull the image from the remote warehouse (always download remotely; even if it is locally, it must be downloaded remotely)
• IfNotPresent: Use the local image if it is available locally, and pull the image from the remote warehouse if it is not available locally (If the image is available locally, it will not be downloaded remotely if the image has a specific version number)
• Never: only use the local image, never go to the remote warehouse to pull, if there is no local image, an error will be reported (always use the local image)

Description of default values:

If the image tag is a specific version number, the default policy is: IfNotPresent

If the image tag is: latest (final version), the default policy is always

# 创建Pod
[root@k8s-master01 pod]# kubectl create -f pod-imagepullpolicy.yaml
pod/pod-imagepullpolicy created

# 查看Pod详情
# 此时明显可以看到nginx镜像有一步Pulling image "nginx:1.17.1"的过程
[root@k8s-master01 pod]# kubectl describe pod pod-imagepullpolicy -n dev
......
Events:
  Type     Reason     Age               From               Message
  ----     ------     ----              ----               -------
  Normal   Scheduled  <unknown>         default-scheduler  Successfully assigned dev/pod-imagePullPolicy to node1
  Normal   Pulling    32s               kubelet, node1     Pulling image "nginx:1.17.1"
  Normal   Pulled     26s               kubelet, node1     Successfully pulled image "nginx:1.17.1"
  Normal   Created    26s               kubelet, node1     Created container nginx
  Normal   Started    25s               kubelet, node1     Started container nginx
  Normal   Pulled     7s (x3 over 25s)  kubelet, node1     Container image "busybox:1.30" already present on machine
  Normal   Created    7s (x3 over 25s)  kubelet, node1     Created container busybox
  Normal   Started    7s (x3 over 25s)  kubelet, node1     Started container busybox

2.2.3 Start command

In the previous case, there has always been a problem that has not been resolved, that is, the busybox container has not been successfully run, so what caused the failure of this container?

It turns out that busybox is not a program, but a collection of tools. After the kubernetes cluster is started and managed, it will automatically shut down. The solution is to keep it running (infinite loop), which uses the command configuration.

Create a pod-command.yaml file with the following content:

apiVersion: v1
kind: Pod
metadata:
  name: pod-command
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
  - name: busybox
    image: busybox:1.30
    command: ["/bin/sh","-c","touch /tmp/hello.txt;while true;do /bin/echo $(date +%T) >> /tmp/hello.txt; sleep 3; done;"]

command, used to run a command after the container in the pod is initialized.

Explain the meaning of the above command a little bit:

"/bin/sh", "-c", use sh to execute the command

touch /tmp/hello.txt; create a /tmp/hello.txt file

while true;do /bin/echo $(date +%T) >> /tmp/hello.txt; sleep 3; done; Write the current time to the file every 3 seconds

# 创建Pod
[root@k8s-master01 pod]# kubectl create  -f pod-command.yaml
pod/pod-command created

# 查看Pod状态
# 此时发现两个pod都正常运行了
[root@k8s-master01 pod]# kubectl get pods pod-command -n dev
NAME          READY   STATUS   RESTARTS   AGE
pod-command   2/2     Runing   0          2s

# 进入pod中的busybox容器，查看文件内容
# 补充一个命令: kubectl exec  pod名称 -n 命名空间 -it -c 容器名称 /bin/sh  在容器内部执行命令
# 使用这个命令就可以进入某个容器的内部，然后进行相关操作了
# 比如，可以查看txt文件的内容
[root@k8s-master01 pod]# kubectl exec pod-command -n dev -it -c busybox /bin/sh
/ # tail -f /tmp/hello.txt
14:44:19
14:44:22
14:44:25

特别说明：
    通过上面发现command已经可以完成启动命令和传递参数的功能，为什么这里还要提供一个args选项，用于传递参数呢?这其实跟docker有点关系，kubernetes中的command、args两项其实是实现覆盖Dockerfile中ENTRYPOINT的功能。
 1 如果command和args均没有写，那么用Dockerfile的配置。
 2 如果command写了，但args没有写，那么Dockerfile默认的配置会被忽略，执行输入的command
 3 如果command没写，但args写了，那么Dockerfile中配置的ENTRYPOINT的命令会被执行，使用当前args的参数
 4 如果command和args都写了，那么Dockerfile的配置被忽略，执行command并追加上args参数

2.2.4 Environment variables

Create a pod-env.yaml file with the following content:

apiVersion: v1
kind: Pod
metadata:
  name: pod-env
  namespace: dev
spec:
  containers:
  - name: busybox
    image: busybox:1.30
    command: ["/bin/sh","-c","while true;do /bin/echo $(date +%T);sleep 60; done;"]
    env: # 设置环境变量列表（下边设置了两个环境变量）
    - name: "username"
      value: "admin"
    - name: "password"
      value: "123456"

env: Environment variables, used to set environment variables in the containers in the pod.

# 创建Pod
[root@k8s-master01 ~]# kubectl create -f pod-env.yaml
pod/pod-env created

# 进入容器，输出环境变量
[root@k8s-master01 ~]# kubectl exec pod-env -n dev -c busybox -it /bin/sh
/ # echo $username
admin
/ # echo $password
123456

This method is not very recommended. It is recommended to store these configurations separately in the configuration file, which will be introduced later.

2.2.5 Port Settings

This section introduces the port settings of the container, that is, the ports option of containers.

First look at the sub-options supported by ports:

[root@k8s-master01 ~]# kubectl explain pod.spec.containers.ports
KIND:     Pod
VERSION:  v1
RESOURCE: ports <[]Object>
FIELDS:
   name         <string>  # 端口名称，如果指定，必须保证name在pod中是唯一的		
   containerPort<integer> # 容器要监听的端口(0<x<65536)，重点掌握，使用pod的ip加这个端口来访问
   hostPort     <integer> # 容器要在主机上公开的端口，如果设置，主机上只能运行容器的一个副本(一般省略) 
   hostIP       <string>  # 要将外部端口绑定到的主机IP(一般省略)
   protocol     <string>  # 端口协议。必须是UDP、TCP或SCTP。默认为“TCP”。

Next, write a test case, create pod-ports.yaml

apiVersion: v1
kind: Pod
metadata:
  name: pod-ports
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
    ports: # 设置容器暴露的端口列表
    - name: nginx-port
      containerPort: 80 #以后使用pod的ip加这个端口来访问容器里的程序
      protocol: TCP

# 创建Pod
[root@k8s-master01 ~]# kubectl create -f pod-ports.yaml
pod/pod-ports created

# 查看pod
# 在下面可以明显看到配置信息
[root@k8s-master01 ~]# kubectl get pod pod-ports -n dev -o yaml
......
spec:
  containers:
  - image: nginx:1.17.1
    imagePullPolicy: IfNotPresent
    name: nginx
    ports:
    - containerPort: 80
      name: nginx-port
      protocol: TCP
......

To access the program in the container, you need to use thePodip:containerPort

2.2.6 Resource Quotas

To run a program in a container, it must occupy certain resources, such as CPU and memory. If the resources of a container are not limited, it may consume a lot of resources, causing other containers to fail to run. In response to this situation, kubernetes provides a mechanism for quotas on memory and cpu resources. This mechanism is mainly implemented through the resources option, which has two sub-options:

• limits: Used to limit the maximum resource usage of the runtime container. When the container resource usage exceeds the limits, it will be terminated and restarted
• requests : Used to set the minimum resources required by the container. If the environment resources are not enough, the container will fail to start

The upper and lower limits of resources can be set through the above two options.

Next, write a test case, create pod-resources.yaml

apiVersion: v1
kind: Pod
metadata:
  name: pod-resources
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
    resources: # 资源配额
      limits:  # 限制资源（上限）
        cpu: "2" # CPU限制，单位是core数
        memory: "10Gi" # 内存限制
      requests: # 请求资源（下限）
        cpu: "1"  # CPU限制，单位是core数
        memory: "10Mi"  # 内存限制

Here is a description of the units of cpu and memory:

• cpu: core number, can be integer or decimal
• memory: memory size, you can use Gi, Mi, G, M, etc.

# 运行Pod
[root@k8s-master01 ~]# kubectl create  -f pod-resources.yaml
pod/pod-resources created

# 查看发现pod运行正常
[root@k8s-master01 ~]# kubectl get pod pod-resources -n dev
NAME            READY   STATUS    RESTARTS   AGE  
pod-resources   1/1     Running   0          39s   

# 接下来，停止Pod
[root@k8s-master01 ~]# kubectl delete  -f pod-resources.yaml
pod "pod-resources" deleted

# 编辑pod，修改resources.requests.memory的值为10Gi
[root@k8s-master01 ~]# vim pod-resources.yaml

# 再次启动pod
[root@k8s-master01 ~]# kubectl create  -f pod-resources.yaml
pod/pod-resources created

# 查看Pod状态，发现Pod启动失败
[root@k8s-master01 ~]# kubectl get pod pod-resources -n dev -o wide
NAME            READY   STATUS    RESTARTS   AGE          
pod-resources   0/1     Pending   0          20s    

# 查看pod详情会发现，如下提示
[root@k8s-master01 ~]# kubectl describe pod pod-resources -n dev
......
Warning  FailedScheduling  35s   default-scheduler  0/3 nodes are available: 1 node(s) had taint {
    
    node-role.kubernetes.io/master: }, that the pod didn't tolerate, 2 Insufficient memory.(内存不足)

2.3 Pod life cycle

We generally refer to the period of time from the creation of the pod object to the end as the life cycle of the pod, which mainly includes the following processes:

• pod creation process
• Run the init container process
• Run the main container (main container)
  • Hook after container startup (post start), hook before container termination (pre stop)
  • Container liveness probe (liveness probe), readiness probe (readiness probe)
• pod termination process

Throughout the life cycle, the Pod will appear in five states ( phases ), as follows:

• Pending: The apiserver has created the pod resource object, but it has not been scheduled or is still in the process of downloading the image
• Running (Running): The pod has been scheduled to a node, and all containers have been created by the kubelet
• Succeeded: All containers in the pod have terminated successfully and will not be restarted
• Failed: All containers have terminated, but at least one container failed to terminate, that is, the container returned a non-zero exit status
• Unknown (Unknown): The apiserver cannot normally obtain the status information of the pod object, usually caused by network communication failure

2.3.1 Creation and Termination

pod creation process

1. The user submits the pod information to be created to apiServer through kubectl or other api clients

2. apiServer starts to generate the information of the pod object, stores the information in etcd, and then returns a confirmation message to the client (the message is not returned when the creation is successful, but is returned when the creation is started)

3. The apiServer starts to reflect the changes of the pod object in etcd, and other components use the watch mechanism to track and check the changes on the apiServer

4. The scheduler finds that there is a new pod object to be created, starts to assign hosts to the pod and updates the result information to apiServer

5. The kubelet on the node node finds that a pod is dispatched, tries to call docker to start the container, and sends the result back to apiServer

6. apiServer stores the received pod status information in etcd

pod termination process

1. The user sends a command to apiServer to delete the pod object
2. The pod object information in apiServcer will be updated over time. During the grace period (default 30s), the pod is considered dead
3. Mark the pod as terminating
4. The kubelet starts the pod shutdown process while monitoring that the pod object turns to the terminating state
5. When the endpoint controller monitors the shutdown behavior of the pod object, it will be removed from the endpoint list of all service resources matching this endpoint
6. If the current pod object defines a preStop hook processor, it will start executing synchronously after it is marked as terminating
7. The container process in the pod object received a stop signal
8. After the grace period ends, if there are still running processes in the pod, the pod object will receive an immediate termination signal
9. Kubelet requests apiServer to set the grace period of this pod resource to 0 to complete the deletion operation. At this time, the pod is no longer visible to the user

2.3.2 Initialize the container

The initialization container is a container to be run before the pod's main container is started. It mainly does some pre-work for the main container. It has two characteristics:

1. The initialization container must run to completion until the end. If an initialization container fails to run, then kubernetes needs to restart it until it completes successfully
2. The initialization containers must be executed in the defined order, and the subsequent one can run only after the previous one succeeds

There are many application scenarios for initializing containers, the most common ones are listed below:

• Provide utilities or custom code that are not present in the main container image
• The initialization container starts and runs serially before the application container, so it can be used to delay the start of the application container until the conditions it depends on are met

Next, make a case to simulate the following requirement:

Suppose you want to run nginx as the main container, but you need to be able to connect to the server where mysql and redis are located before running nginx

In order to simplify the test, the addresses of the mysql (192.168.5.4)and redis servers are specified in advance(192.168.5.5)

Create pod-initcontainer.yaml with the following content:

apiVersion: v1
kind: Pod
metadata:
  name: pod-initcontainer
  namespace: dev
spec:
  containers:
  - name: main-container
    image: nginx:1.17.1
    ports: 
    - name: nginx-port
      containerPort: 80
  initContainers:
  - name: test-mysql
    image: busybox:1.30
    command: ['sh', '-c', 'until ping 192.168.5.14 -c 1 ; do echo waiting for mysql...; sleep 2; done;']
  - name: test-redis
    image: busybox:1.30
    command: ['sh', '-c', 'until ping 192.168.5.15 -c 1 ; do echo waiting for reids...; sleep 2; done;']

# 创建pod
[root@k8s-master01 ~]# kubectl create -f pod-initcontainer.yaml
pod/pod-initcontainer created

# 查看pod状态
# 发现pod卡在启动第一个初始化容器过程中，后面的容器不会运行
root@k8s-master01 ~]# kubectl describe pod  pod-initcontainer -n dev
........
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  49s   default-scheduler  Successfully assigned dev/pod-initcontainer to node1
  Normal  Pulled     48s   kubelet, node1     Container image "busybox:1.30" already present on machine
  Normal  Created    48s   kubelet, node1     Created container test-mysql
  Normal  Started    48s   kubelet, node1     Started container test-mysql

# 动态查看pod
[root@k8s-master01 ~]# kubectl get pods pod-initcontainer -n dev -w
NAME                             READY   STATUS     RESTARTS   AGE
pod-initcontainer                0/1     Init:0/2   0          15s
pod-initcontainer                0/1     Init:1/2   0          52s
pod-initcontainer                0/1     Init:1/2   0          53s
pod-initcontainer                0/1     PodInitializing   0          89s
pod-initcontainer                1/1     Running           0          90s

# 接下来新开一个shell，为当前服务器新增两个ip，观察pod的变化
[root@k8s-master01 ~]# ifconfig ens33:1 192.168.5.14 netmask 255.255.255.0 up
[root@k8s-master01 ~]# ifconfig ens33:2 192.168.5.15 netmask 255.255.255.0 up

2.3.3 Hook function

The hook function can perceive the events in its own life cycle, and run the program code specified by the user when the corresponding moment arrives.

Kubernetes provides two hook functions after starting and before stopping the main container:

• post start: Executed after the container is created, if it fails, the container will be restarted
• pre stop : Executed before the container terminates, the container will be terminated successfully after the execution is completed, and the operation of deleting the container will be blocked before it is completed

The hook handler supports the following three ways to define actions:

• Exec command: Execute a command in the container (used more)

  ……
    lifecycle:
      postStart: 
        exec:
          command:
          - cat
          - /tmp/healthy
  ……
  

• TCPSocket: Try to access the specified socket in the current container

  ……      
    lifecycle:
      postStart:
        tcpSocket:
          port: 8080
  ……
  

• HTTPGet: initiate an http request to a url in the current container

  ……
    lifecycle:
      postStart:
        httpGet:
          path: / #URI地址
          port: 80 #端口号
          host: 192.168.5.3 #主机地址
          scheme: HTTP #支持的协议，http或者https
  ……
  

Next, take the exec method as an example to demonstrate the use of the hook function and create a pod-hook-exec.yaml file with the following content:

apiVersion: v1
kind: Pod
metadata:
  name: pod-hook-exec
  namespace: dev
spec:
  containers:
  - name: main-container
    image: nginx:1.17.1
    ports:
    - name: nginx-port
      containerPort: 80
    lifecycle:
      postStart: 
        exec: # 在容器启动的时候执行一个命令，修改掉nginx的默认首页内容
          command: ["/bin/sh", "-c", "echo postStart... > /usr/share/nginx/html/index.html"]
      preStop:
        exec: # 在容器停止之前停止nginx服务
          command: ["/usr/sbin/nginx","-s","quit"]

# 创建pod
[root@k8s-master01 ~]# kubectl create -f pod-hook-exec.yaml
pod/pod-hook-exec created

# 查看pod
[root@k8s-master01 ~]# kubectl get pods  pod-hook-exec -n dev -o wide
NAME           READY   STATUS     RESTARTS   AGE    IP            NODE    
pod-hook-exec  1/1     Running    0          29s    10.244.2.48   node2   

# 访问pod
[root@k8s-master01 ~]# curl 10.244.2.48
postStart...

2.3.4 Container detection

Container detection is used to detect whether the application instance in the container is working normally, and it is a traditional mechanism to ensure business availability. If after detection, the state of the instance does not meet expectations, then kubernetes will "remove" the problem instance and not bear the business traffic. Kubernetes provides two probes to implement container detection, namely:

• liveness probes: liveness probes, used to detect whether the application instance is currently running normally, if not, k8s will restart the container
• Readiness probes: readiness probes, used to detect whether the application instance can currently receive requests, if not, k8s will not forward traffic

livenessProbe decides whether to restart the container, and readinessProbe decides whether to forward the request to the container.

The above two probes currently support three detection methods:

• Exec command: Execute a command in the container. If the exit code of command execution is 0, the program is considered normal, otherwise it is abnormal

  ……
    livenessProbe:
      exec:
        command:
        - cat
        - /tmp/healthy
  ……
  

• TCPSocket: It will try to access the port of a user container. If this connection can be established, the program is considered normal, otherwise it is not normal

  ……      
    livenessProbe:
      tcpSocket:
        port: 8080
  ……
  

• HTTPGet: Call the URL of the Web application in the container. If the returned status code is between 200 and 399, the program is considered normal, otherwise it is abnormal

  ……
    livenessProbe:
      httpGet:
        path: / #URI地址
        port: 80 #端口号
        host: 127.0.0.1 #主机地址
        scheme: HTTP #支持的协议，http或者https
  ……
  

Let's take liveness probes as an example to make a few demonstrations:

Method 1: Exec

Create pod-liveness-exec.yaml

apiVersion: v1
kind: Pod
metadata:
  name: pod-liveness-exec
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
    ports: 
    - name: nginx-port
      containerPort: 80
    livenessProbe:
      exec:
        command: ["/bin/cat","/tmp/hello.txt"] # 执行一个查看文件的命令

Create a pod and observe the effect

# 创建Pod
[root@k8s-master01 ~]# kubectl create -f pod-liveness-exec.yaml
pod/pod-liveness-exec created

# 查看Pod详情
[root@k8s-master01 ~]# kubectl describe pods pod-liveness-exec -n dev
......
  Normal   Created    20s (x2 over 50s)  kubelet, node1     Created container nginx
  Normal   Started    20s (x2 over 50s)  kubelet, node1     Started container nginx
  Normal   Killing    20s                kubelet, node1     Container nginx failed liveness probe, will be restarted
  Warning  Unhealthy  0s (x5 over 40s)   kubelet, node1     Liveness probe failed: cat: can't open '/tmp/hello11.txt': No such file or directory
  
# 观察上面的信息就会发现nginx容器启动之后就进行了健康检查
# 检查失败之后，容器被kill掉，然后尝试进行重启（这是重启策略的作用，后面讲解）
# 稍等一会之后，再观察pod信息，就可以看到RESTARTS不再是0，而是一直增长
[root@k8s-master01 ~]# kubectl get pods pod-liveness-exec -n dev
NAME                READY   STATUS             RESTARTS   AGE
pod-liveness-exec   0/1     CrashLoopBackOff   2          3m19s

# 当然接下来，可以修改成一个存在的文件，比如/tmp/hello.txt，再试，结果就正常了......

Method 2: TCPSocket

Create pod-liveness-tcpsocket.yaml

apiVersion: v1
kind: Pod
metadata:
  name: pod-liveness-tcpsocket
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
    ports: 
    - name: nginx-port
      containerPort: 80
    livenessProbe:
      tcpSocket:
        port: 8080 # 尝试访问8080端口

Create a pod and observe the effect

# 创建Pod
[root@k8s-master01 ~]# kubectl create -f pod-liveness-tcpsocket.yaml
pod/pod-liveness-tcpsocket created

# 查看Pod详情
[root@k8s-master01 ~]# kubectl describe pods pod-liveness-tcpsocket -n dev
......
  Normal   Scheduled  31s                            default-scheduler  Successfully assigned dev/pod-liveness-tcpsocket to node2
  Normal   Pulled     <invalid>                      kubelet, node2     Container image "nginx:1.17.1" already present on machine
  Normal   Created    <invalid>                      kubelet, node2     Created container nginx
  Normal   Started    <invalid>                      kubelet, node2     Started container nginx
  Warning  Unhealthy  <invalid> (x2 over <invalid>)  kubelet, node2     Liveness probe failed: dial tcp 10.244.2.44:8080: connect: connection refused
  
# 观察上面的信息，发现尝试访问8080端口,但是失败了
# 稍等一会之后，再观察pod信息，就可以看到RESTARTS不再是0，而是一直增长
[root@k8s-master01 ~]# kubectl get pods pod-liveness-tcpsocket  -n dev
NAME                     READY   STATUS             RESTARTS   AGE
pod-liveness-tcpsocket   0/1     CrashLoopBackOff   2          3m19s

# 当然接下来，可以修改成一个可以访问的端口，比如80，再试，结果就正常了......

Method 3: HTTPGet

Create pod-liveness-httpget.yaml

apiVersion: v1
kind: Pod
metadata:
  name: pod-liveness-httpget
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
    ports:
    - name: nginx-port
      containerPort: 80
    livenessProbe:
      httpGet:  # 其实就是访问http://127.0.0.1:80/hello  
        scheme: HTTP #支持的协议，http或者https
        port: 80 #端口号
        path: /hello #URI地址

Create a pod and observe the effect

# 创建Pod
[root@k8s-master01 ~]# kubectl create -f pod-liveness-httpget.yaml
pod/pod-liveness-httpget created

# 查看Pod详情
[root@k8s-master01 ~]# kubectl describe pod pod-liveness-httpget -n dev
.......
  Normal   Pulled     6s (x3 over 64s)  kubelet, node1     Container image "nginx:1.17.1" already present on machine
  Normal   Created    6s (x3 over 64s)  kubelet, node1     Created container nginx
  Normal   Started    6s (x3 over 63s)  kubelet, node1     Started container nginx
  Warning  Unhealthy  6s (x6 over 56s)  kubelet, node1     Liveness probe failed: HTTP probe failed with statuscode: 404
  Normal   Killing    6s (x2 over 36s)  kubelet, node1     Container nginx failed liveness probe, will be restarted
  
# 观察上面信息，尝试访问路径，但是未找到,出现404错误
# 稍等一会之后，再观察pod信息，就可以看到RESTARTS不再是0，而是一直增长
[root@k8s-master01 ~]# kubectl get pod pod-liveness-httpget -n dev
NAME                   READY   STATUS    RESTARTS   AGE
pod-liveness-httpget   1/1     Running   5          3m17s

# 当然接下来，可以修改成一个可以访问的路径path，比如/，再试，结果就正常了......

So far, liveness Probe has been used to demonstrate three detection methods, but looking at the sub-properties of livenessProbe, you will find that in addition to these three methods, there are some other configurations, which are explained here:

[root@k8s-master01 ~]# kubectl explain pod.spec.containers.livenessProbe
FIELDS:
   exec <Object>  
   tcpSocket    <Object>
   httpGet      <Object>
   initialDelaySeconds  <integer>  # 容器启动后等待多少秒执行第一次探测
   timeoutSeconds       <integer>  # 探测超时时间。默认1秒，最小1秒
   periodSeconds        <integer>  # 执行探测的频率。默认是10秒，最小1秒
   failureThreshold     <integer>  # 连续探测失败多少次才被认定为失败。默认是3。最小值是1
   successThreshold     <integer>  # 连续探测成功多少次才被认定为成功。默认是1

Let’s configure two slightly below to demonstrate the effect:

[root@k8s-master01 ~]# more pod-liveness-httpget.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-liveness-httpget
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
    ports:
    - name: nginx-port
      containerPort: 80
    livenessProbe:
      httpGet:
        scheme: HTTP
        port: 80 
        path: /
      initialDelaySeconds: 30 # 容器启动后30s开始探测
      timeoutSeconds: 5 # 探测超时时间为5s

2.3.5 Restart strategy

In the previous section, once there is a problem with container detection, kubernetes will restart the Pod where the container is located. In fact, this is determined by the restart strategy of the pod. There are three restart strategies for the pod, as follows:

• Always : When the container fails, the container is automatically restarted, which is also the default value.
• OnFailure: Restart when the container terminates and the exit code is not 0 (not 0: abnormal termination)
• Never : Do not restart the container regardless of the state

The restart policy applies to all containers in the pod object. The container that needs to be restarted for the first time will be restarted immediately when it needs to be restarted, and the operation that needs to be restarted again will be delayed by kubelet for a period of time, and the delay time of repeated restart operations is based on this 10s, 20s, 40s, 80s, 160s and 300s, 300s is the maximum delay time.

Create pod-restartpolicy.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: pod-restartpolicy
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
    ports:
    - name: nginx-port
      containerPort: 80
    livenessProbe:
      httpGet:
        scheme: HTTP
        port: 80
        path: /hello
  restartPolicy: Never # 设置重启策略为Never

run pod test

# 创建Pod
[root@k8s-master01 ~]# kubectl create -f pod-restartpolicy.yaml
pod/pod-restartpolicy created

# 查看Pod详情，发现nginx容器失败
[root@k8s-master01 ~]# kubectl  describe pods pod-restartpolicy  -n dev
......
  Warning  Unhealthy  15s (x3 over 35s)  kubelet, node1     Liveness probe failed: HTTP probe failed with statuscode: 404
  Normal   Killing    15s                kubelet, node1     Container nginx failed liveness probe
  
# 多等一会，再观察pod的重启次数，发现一直是0，并未重启   
[root@k8s-master01 ~]# kubectl  get pods pod-restartpolicy -n dev
NAME                   READY   STATUS    RESTARTS   AGE
pod-restartpolicy      0/1     Running   0          5min42s

2.4 Pod scheduling

By default, which Node node a Pod runs on is calculated by the Scheduler component using the corresponding algorithm, and this process is not subject to manual control. But in actual use, this does not meet the needs, because in many cases, we want to control certain Pods to reach certain nodes, so how should we do it? This requires understanding the scheduling rules of kubernetes for pods. kubernetes provides four types of scheduling methods:

• Automatic scheduling: Which node to run on is completely calculated by the Scheduler through a series of algorithms
• Directed scheduling: NodeName, NodeSelector
• Affinity scheduling: NodeAffinity, PodAffinity, PodAntiAffinity
• Taint (tolerance) scheduling: Taints, Toleration

2.4.1 Directed Scheduling

Directed scheduling refers to scheduling the Pod to the desired node by declaring nodeName or nodeSelector on the pod. Note that the scheduling here is mandatory, which means that even if the target Node to be scheduled does not exist, it will be scheduled to it, but the pod fails to run.

NodeName

NodeName is used to enforce constraints to schedule Pods on the Node node of the specified Name. In this way, the scheduling logic of the Scheduler is skipped directly, and the Pod is directly scheduled to the node with the specified name.

Next, experiment: Create a pod-nodename.yaml file

apiVersion: v1
kind: Pod
metadata:
  name: pod-nodename
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
  nodeName: node1 # 把当前pod，指定调度到node1节点上

#创建Pod
[root@k8s-master01 ~]# kubectl create -f pod-nodename.yaml
pod/pod-nodename created

#查看Pod调度到NODE属性，确实是调度到了node1节点上
[root@k8s-master01 ~]# kubectl get pods pod-nodename -n dev -o wide
NAME           READY   STATUS    RESTARTS   AGE   IP            NODE      ......
pod-nodename   1/1     Running   0          56s   10.244.1.87   node1     ......   

# 接下来，删除pod，修改nodeName的值为node3（并没有node3节点）
[root@k8s-master01 ~]# kubectl delete -f pod-nodename.yaml
pod "pod-nodename" deleted
[root@k8s-master01 ~]# vim pod-nodename.yaml
[root@k8s-master01 ~]# kubectl create -f pod-nodename.yaml
pod/pod-nodename created

#再次查看，发现已经向Node3节点调度，但是由于不存在node3节点，所以pod无法正常运行
[root@k8s-master01 ~]# kubectl get pods pod-nodename -n dev -o wide
NAME           READY   STATUS    RESTARTS   AGE   IP       NODE    ......
pod-nodename   0/1     Pending   0          6s    <none>   node3   ......           

NodeSelector

NodeSelector is used to schedule pods to node nodes with specified labels. It is implemented through the label-selector mechanism of kubernetes. That is to say, before the pod is created, the scheduler will use the MatchNodeSelector scheduling strategy to perform label matching, find out the target node, and then schedule the pod to the target node. This matching rule is mandatory constraint.

Next, experiment:

1 First add labels to node nodes respectively

[root@k8s-master01 ~]# kubectl label nodes node1 nodeenv=pro
node/node2 labeled
[root@k8s-master01 ~]# kubectl label nodes node2 nodeenv=test
node/node2 labeled

2 Create a pod-nodeselector.yaml file and use it to create Pods

apiVersion: v1
kind: Pod
metadata:
  name: pod-nodeselector
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
  nodeSelector: 
    nodeenv: pro # 指定调度到具有nodeenv=pro标签的节点上

#创建Pod
[root@k8s-master01 ~]# kubectl create -f pod-nodeselector.yaml
pod/pod-nodeselector created

#查看Pod调度到NODE属性，确实是调度到了node1节点上
[root@k8s-master01 ~]# kubectl get pods pod-nodeselector -n dev -o wide
NAME               READY   STATUS    RESTARTS   AGE     IP          NODE    ......
pod-nodeselector   1/1     Running   0          47s   10.244.1.87   node1   ......

# 接下来，删除pod，修改nodeSelector的值为nodeenv: xxxx（不存在打有此标签的节点）
[root@k8s-master01 ~]# kubectl delete -f pod-nodeselector.yaml
pod "pod-nodeselector" deleted
[root@k8s-master01 ~]# vim pod-nodeselector.yaml
[root@k8s-master01 ~]# kubectl create -f pod-nodeselector.yaml
pod/pod-nodeselector created

#再次查看，发现pod无法正常运行,Node的值为none
[root@k8s-master01 ~]# kubectl get pods -n dev -o wide
NAME               READY   STATUS    RESTARTS   AGE     IP       NODE    
pod-nodeselector   0/1     Pending   0          2m20s   <none>   <none>

# 查看详情,发现node selector匹配失败的提示
[root@k8s-master01 ~]# kubectl describe pods pod-nodeselector -n dev
.......
Events:
  Type     Reason            Age        From               Message
  ----     ------            ----       ----               -------
  Warning  FailedScheduling  <unknown>  default-scheduler  0/3 nodes are available: 3 node(s) didn't match node selector.

2.4.2 Affinity Scheduling

In the previous section, two directional scheduling methods were introduced, which are very convenient to use, but there are certain problems, that is, if there is no Node that meets the conditions, the Pod will not be run, even if there is a list of available Nodes in the cluster No, this limits its usage scenarios.

Based on the above problems, kubernetes also provides an affinity scheduling (Affinity). It has been extended on the basis of NodeSelector. Through the form of configuration, the Node that meets the conditions can be preferentially selected for scheduling. If not, it can also be scheduled to the nodes that do not meet the conditions, making the scheduling more flexible.

Affinity is mainly divided into three categories:

• nodeAffinity (node ​​affinity): Target the node to solve the problem of which nodes the pod can be scheduled to
• podAffinity (pod affinity): With pod as the target, solve the problem of which pods can be deployed in the same topology domain as existing pods
• podAntiAffinity (pod anti-affinity): Target pods to solve the problem that pods cannot be deployed in the same topology domain as existing pods

Explanation on affinity (anti-affinity) usage scenarios:

Affinity : If two applications interact frequently, it is necessary to use affinity to make the two applications as close as possible, which can reduce the performance loss caused by network communication.

Anti-affinity : When the application is deployed with multiple copies, it is necessary to use anti-affinity to distribute each application instance on each node, which can improve the high availability of the service.

NodeAffinity

First look at NodeAffinitythe configurable items:

pod.spec.affinity.nodeAffinity
  requiredDuringSchedulingIgnoredDuringExecution  Node节点必须满足指定的所有规则才可以，相当于硬限制
    nodeSelectorTerms  节点选择列表
      matchFields   按节点字段列出的节点选择器要求列表
      matchExpressions   按节点标签列出的节点选择器要求列表(推荐)
        key    键
        values 值
        operator 关系符 支持Exists, DoesNotExist, In, NotIn, Gt, Lt
  preferredDuringSchedulingIgnoredDuringExecution 优先调度到满足指定的规则的Node，相当于软限制 (倾向)
    preference   一个节点选择器项，与相应的权重相关联
      matchFields   按节点字段列出的节点选择器要求列表
      matchExpressions   按节点标签列出的节点选择器要求列表(推荐)
        key    键
        values 值
        operator 关系符 支持In, NotIn, Exists, DoesNotExist, Gt, Lt
	weight 倾向权重，在范围1-100。

关系符的使用说明:

- matchExpressions:
  - key: nodeenv              # 匹配存在标签的key为nodeenv的节点
    operator: Exists
  - key: nodeenv              # 匹配标签的key为nodeenv,且value是"xxx"或"yyy"的节点
    operator: In
    values: ["xxx","yyy"]
  - key: nodeenv              # 匹配标签的key为nodeenv,且value大于"xxx"的节点
    operator: Gt
    values: "xxx"

Next, let's demonstrate first requiredDuringSchedulingIgnoredDuringExecution,

Create pod-nodeaffinity-required.yaml

apiVersion: v1
kind: Pod
metadata:
  name: pod-nodeaffinity-required
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
  affinity:  #亲和性设置
    nodeAffinity: #设置node亲和性
      requiredDuringSchedulingIgnoredDuringExecution: # 硬限制
        nodeSelectorTerms:
        - matchExpressions: # 匹配env的值在["xxx","yyy"]中的标签
          - key: nodeenv
            operator: In
            values: ["xxx","yyy"]

# 创建pod
[root@k8s-master01 ~]# kubectl create -f pod-nodeaffinity-required.yaml
pod/pod-nodeaffinity-required created

# 查看pod状态 （运行失败）
[root@k8s-master01 ~]# kubectl get pods pod-nodeaffinity-required -n dev -o wide
NAME                        READY   STATUS    RESTARTS   AGE   IP       NODE    ...... 
pod-nodeaffinity-required   0/1     Pending   0          16s   <none>   <none>  ......

# 查看Pod的详情
# 发现调度失败，提示node选择失败
[root@k8s-master01 ~]# kubectl describe pod pod-nodeaffinity-required -n dev
......
  Warning  FailedScheduling  <unknown>  default-scheduler  0/3 nodes are available: 3 node(s) didn't match node selector.
  Warning  FailedScheduling  <unknown>  default-scheduler  0/3 nodes are available: 3 node(s) didn't match node selector.

#接下来，停止pod
[root@k8s-master01 ~]# kubectl delete -f pod-nodeaffinity-required.yaml
pod "pod-nodeaffinity-required" deleted

# 修改文件，将values: ["xxx","yyy"]------> ["pro","yyy"]
[root@k8s-master01 ~]# vim pod-nodeaffinity-required.yaml

# 再次启动
[root@k8s-master01 ~]# kubectl create -f pod-nodeaffinity-required.yaml
pod/pod-nodeaffinity-required created

# 此时查看，发现调度成功，已经将pod调度到了node1上
[root@k8s-master01 ~]# kubectl get pods pod-nodeaffinity-required -n dev -o wide
NAME                        READY   STATUS    RESTARTS   AGE   IP            NODE  ...... 
pod-nodeaffinity-required   1/1     Running   0          11s   10.244.1.89   node1 ......

Next, let's demonstrate again requiredDuringSchedulingIgnoredDuringExecution,

Create pod-nodeaffinity-preferred.yaml

apiVersion: v1
kind: Pod
metadata:
  name: pod-nodeaffinity-preferred
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
  affinity:  #亲和性设置
    nodeAffinity: #设置node亲和性
      preferredDuringSchedulingIgnoredDuringExecution: # 软限制
      - weight: 1
        preference:
          matchExpressions: # 匹配env的值在["xxx","yyy"]中的标签(当前环境没有)
          - key: nodeenv
            operator: In
            values: ["xxx","yyy"]

# 创建pod
[root@k8s-master01 ~]# kubectl create -f pod-nodeaffinity-preferred.yaml
pod/pod-nodeaffinity-preferred created

# 查看pod状态 （运行成功）
[root@k8s-master01 ~]# kubectl get pod pod-nodeaffinity-preferred -n dev
NAME                         READY   STATUS    RESTARTS   AGE
pod-nodeaffinity-preferred   1/1     Running   0          40s

NodeAffinity规则设置的注意事项：
    1 如果同时定义了nodeSelector和nodeAffinity，那么必须两个条件都得到满足，Pod才能运行在指定的Node上
    2 如果nodeAffinity指定了多个nodeSelectorTerms，那么只需要其中一个能够匹配成功即可
    3 如果一个nodeSelectorTerms中有多个matchExpressions ，则一个节点必须满足所有的才能匹配成功
    4 如果一个pod所在的Node在Pod运行期间其标签发生了改变，不再符合该Pod的节点亲和性需求，则系统将忽略此变化

PodAffinity

PodAffinity mainly implements the function of using the running Pod as a reference to make the newly created Pod and the reference pod in the same area.

First look at PodAffinitythe configurable items:

pod.spec.affinity.podAffinity
  requiredDuringSchedulingIgnoredDuringExecution  硬限制
    namespaces       指定参照pod的namespace
    topologyKey      指定调度作用域
    labelSelector    标签选择器
      matchExpressions  按节点标签列出的节点选择器要求列表(推荐)
        key    键
        values 值
        operator 关系符 支持In, NotIn, Exists, DoesNotExist.
      matchLabels    指多个matchExpressions映射的内容
  preferredDuringSchedulingIgnoredDuringExecution 软限制
    podAffinityTerm  选项
      namespaces      
      topologyKey
      labelSelector
        matchExpressions  
          key    键
          values 值
          operator
        matchLabels 
    weight 倾向权重，在范围1-100

topologyKey用于指定调度时作用域,例如:
    如果指定为kubernetes.io/hostname，那就是以Node节点为区分范围
	如果指定为beta.kubernetes.io/os,则以Node节点的操作系统类型来区分

Next, demo requiredDuringSchedulingIgnoredDuringExecution,

1) First create a reference Pod, pod-podaffinity-target.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: pod-podaffinity-target
  namespace: dev
  labels:
    podenv: pro #设置标签
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
  nodeName: node1 # 将目标pod名确指定到node1上

# 启动目标pod
[root@k8s-master01 ~]# kubectl create -f pod-podaffinity-target.yaml
pod/pod-podaffinity-target created

# 查看pod状况
[root@k8s-master01 ~]# kubectl get pods  pod-podaffinity-target -n dev
NAME                     READY   STATUS    RESTARTS   AGE
pod-podaffinity-target   1/1     Running   0          4s

2) Create pod-podaffinity-required.yaml with the following content:

apiVersion: v1
kind: Pod
metadata:
  name: pod-podaffinity-required
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
  affinity:  #亲和性设置
    podAffinity: #设置pod亲和性
      requiredDuringSchedulingIgnoredDuringExecution: # 硬限制
      - labelSelector:
          matchExpressions: # 匹配env的值在["xxx","yyy"]中的标签
          - key: podenv
            operator: In
            values: ["xxx","yyy"]
        topologyKey: kubernetes.io/hostname

The above configuration means that the new Pod must be on the same Node as the pod with the label nodeenv=xxx or nodeenv=yyy. Obviously, there is no such pod now. Next, run the test.

# 启动pod
[root@k8s-master01 ~]# kubectl create -f pod-podaffinity-required.yaml
pod/pod-podaffinity-required created

# 查看pod状态，发现未运行
[root@k8s-master01 ~]# kubectl get pods pod-podaffinity-required -n dev
NAME                       READY   STATUS    RESTARTS   AGE
pod-podaffinity-required   0/1     Pending   0          9s

# 查看详细信息
[root@k8s-master01 ~]# kubectl describe pods pod-podaffinity-required  -n dev
......
Events:
  Type     Reason            Age        From               Message
  ----     ------            ----       ----               -------
  Warning  FailedScheduling  <unknown>  default-scheduler  0/3 nodes are available: 2 node(s) didn't match pod affinity rules, 1 node(s) had taints that the pod didn't tolerate.

# 接下来修改  values: ["xxx","yyy"]----->values:["pro","yyy"]
# 意思是：新Pod必须要与拥有标签nodeenv=xxx或者nodeenv=yyy的pod在同一Node上
[root@k8s-master01 ~]# vim pod-podaffinity-required.yaml

# 然后重新创建pod，查看效果
[root@k8s-master01 ~]# kubectl delete -f  pod-podaffinity-required.yaml
pod "pod-podaffinity-required" deleted
[root@k8s-master01 ~]# kubectl create -f pod-podaffinity-required.yaml
pod/pod-podaffinity-required created

# 发现此时Pod运行正常
[root@k8s-master01 ~]# kubectl get pods pod-podaffinity-required -n dev
NAME                       READY   STATUS    RESTARTS   AGE   LABELS
pod-podaffinity-required   1/1     Running   0          6s    <none>

About PodAffinity, preferredDuringSchedulingIgnoredDuringExecutionit will not be demonstrated here.

PodAntiAffinity anti-affinity

PodAntiAffinity mainly implements the function of using the running Pod as a reference, so that the newly created Pod and the reference pod are not in the same area.

Its configuration method and options are the same as PodAffinty, so I won’t explain it in detail here, but just make a test case.

1) Continue to use the target pod in the previous case

[root@k8s-master01 ~]# kubectl get pods -n dev -o wide --show-labels
NAME                     READY   STATUS    RESTARTS   AGE     IP            NODE    LABELS
pod-podaffinity-required 1/1     Running   0          3m29s   10.244.1.38   node1   <none>     
pod-podaffinity-target   1/1     Running   0          9m25s   10.244.1.37   node1   podenv=pro

2) Create pod-podantiaffinity-required.yaml with the following content:

apiVersion: v1
kind: Pod
metadata:
  name: pod-podantiaffinity-required
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
  affinity:  #亲和性设置
    podAntiAffinity: #设置pod亲和性
      requiredDuringSchedulingIgnoredDuringExecution: # 硬限制
      - labelSelector:
          matchExpressions: # 匹配podenv的值在["pro"]中的标签
          - key: podenv
            operator: In
            values: ["pro"]
        topologyKey: kubernetes.io/hostname

The above configuration means that the new Pod must not be on the same Node as the pod with the label nodeenv=pro. Run the test.

# 创建pod
[root@k8s-master01 ~]# kubectl create -f pod-podantiaffinity-required.yaml
pod/pod-podantiaffinity-required created

# 查看pod
# 发现调度到了node2上
[root@k8s-master01 ~]# kubectl get pods pod-podantiaffinity-required -n dev -o wide
NAME                           READY   STATUS    RESTARTS   AGE   IP            NODE   .. 
pod-podantiaffinity-required   1/1     Running   0          30s   10.244.1.96   node2  ..

2.4.3 Taints and tolerances

Taints

The previous scheduling methods are all from the perspective of the Pod, by adding attributes to the Pod to determine whether the Pod should be scheduled to the specified Node, in fact, we can also stand on the perspective of the Node, by adding the taint attribute to the Node , to decide whether to allow Pod scheduling.

After the Node is tainted, there is a repulsive relationship with the Pod, and then the Pod is refused to be scheduled, and even the existing Pod can be expelled.

The format of the taint is: key=value:effect, key and value are the labels of the taint, and the effect describes the effect of the taint. The following three options are supported:

• PreferNoSchedule: kubernetes will try to avoid scheduling Pods on Nodes with this taint unless there are no other nodes to schedule
• NoSchedule: kubernetes will not schedule Pods to Nodes with this taint, but will not affect existing Pods on the current Node
• NoExecute: kubernetes will not schedule Pods to Nodes with this taint, and will also drive existing Pods on Nodes away

Examples of commands to set and remove taints using kubectl are as follows:

# 设置污点
kubectl taint nodes node1 key=value:effect

# 去除污点
kubectl taint nodes node1 key:effect-

# 去除所有污点
kubectl taint nodes node1 key-

Next, demonstrate the effect of the stain:

1. Prepare node node1 (to make the demonstration effect more obvious, temporarily stop the node2 node)
2. Set a taint for the node1 node: tag=heima:PreferNoSchedule; then create pod1 (pod1 is ok)
3. Modify to set a taint for the node1 node: tag=heima:NoSchedule; then create pod2 (pod1 normal pod2 fails)
4. Modify to set a taint for the node1 node: tag=heima:NoExecute; then create pod3 (all 3 pods fail)

# 为node1设置污点(PreferNoSchedule)
[root@k8s-master01 ~]# kubectl taint nodes node1 tag=heima:PreferNoSchedule

# 创建pod1
[root@k8s-master01 ~]# kubectl run taint1 --image=nginx:1.17.1 -n dev
[root@k8s-master01 ~]# kubectl get pods -n dev -o wide
NAME                      READY   STATUS    RESTARTS   AGE     IP           NODE   
taint1-7665f7fd85-574h4   1/1     Running   0          2m24s   10.244.1.59   node1    

# 为node1设置污点(取消PreferNoSchedule，设置NoSchedule)
[root@k8s-master01 ~]# kubectl taint nodes node1 tag:PreferNoSchedule-
[root@k8s-master01 ~]# kubectl taint nodes node1 tag=heima:NoSchedule

# 创建pod2
[root@k8s-master01 ~]# kubectl run taint2 --image=nginx:1.17.1 -n dev
[root@k8s-master01 ~]# kubectl get pods taint2 -n dev -o wide
NAME                      READY   STATUS    RESTARTS   AGE     IP            NODE
taint1-7665f7fd85-574h4   1/1     Running   0          2m24s   10.244.1.59   node1 
taint2-544694789-6zmlf    0/1     Pending   0          21s     <none>        <none>   

# 为node1设置污点(取消NoSchedule，设置NoExecute)
[root@k8s-master01 ~]# kubectl taint nodes node1 tag:NoSchedule-
[root@k8s-master01 ~]# kubectl taint nodes node1 tag=heima:NoExecute

# 创建pod3
[root@k8s-master01 ~]# kubectl run taint3 --image=nginx:1.17.1 -n dev
[root@k8s-master01 ~]# kubectl get pods -n dev -o wide
NAME                      READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED 
taint1-7665f7fd85-htkmp   0/1     Pending   0          35s   <none>   <none>   <none>    
taint2-544694789-bn7wb    0/1     Pending   0          35s   <none>   <none>   <none>     
taint3-6d78dbd749-tktkq   0/1     Pending   0          6s    <none>   <none>   <none>     

Tip:
A cluster built with kubeadm will add a taint mark to the master node by default, so pods will not be scheduled to the master node.

kubectl describe node master

Toleration

The function of taint is introduced above. We can add taint to node to refuse pod scheduling. But if we just want to schedule a pod to a node with taint, what should we do at this time? This is where tolerance is used . (The left side below is tolerance)

Stain means rejection, and tolerance means ignoring. Node rejects pod scheduling through taint, and Pod rejects through tolerance and ignore.

Let's take a look at the effect through a case:

1. In the previous section, the stain has been marked on the node1 node NoExecute. At this time, the pod cannot be scheduled.
2. In this section, you can add tolerance to the pod and then schedule it

Create pod-toleration.yaml with the following content

apiVersion: v1
kind: Pod
metadata:
  name: pod-toleration
  namespace: dev
spec:
  containers:
  - name: nginx
    image: nginx:1.17.1
  tolerations:      # 添加容忍
  - key: "tag"        # 要容忍的污点的key
    operator: "Equal" # 操作符
    value: "heima"    # 容忍的污点的value
    effect: "NoExecute"   # 添加容忍的规则，这里必须和标记的污点规则相同

# 添加容忍之前的pod
[root@k8s-master01 ~]# kubectl get pods -n dev -o wide
NAME             READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED 
pod-toleration   0/1     Pending   0          3s    <none>   <none>   <none>           

# 添加容忍之后的pod
[root@k8s-master01 ~]# kubectl get pods -n dev -o wide
NAME             READY   STATUS    RESTARTS   AGE   IP            NODE    NOMINATED
pod-toleration   1/1     Running   0          3s    10.244.1.62   node1   <none>        

Let's take a look at the detailed configuration of tolerance:

[root@k8s-master01 ~]# kubectl explain pod.spec.tolerations
......
FIELDS:
   key       # 对应着要容忍的污点的键，空意味着匹配所有的键
   value     # 对应着要容忍的污点的值
   operator  # key-value的运算符，支持Equal和Exists（默认）
   effect    # 对应污点的effect，空意味着匹配所有影响
   tolerationSeconds   # 容忍时间, 当effect为NoExecute时生效，表示pod在Node上的停留时间