Container probe
Kubelet periodic diagnosis probe is performed on the container. To perform diagnostics, kubelet call Handler implemented by the container. There are three types of processing programs:
ExecAction : Executes the specified command within the container. If the exit command return code of 0 is considered diagnostic success.
TCPSocketAction : the specified container port TCP IP address checking. If the port is open, the diagnosis is considered to be a success.
HTTPGetAction : performing HTTP Get request for an IP address of the container on the specified ports and paths. If the response status code greater than or equal to 200 and less than 400, then the diagnosis is considered successful
each probe will have one of three results:
Success: the container through the diagnosis.
Failure: container is not diagnostic.
Unknown: diagnostics fail, it will not take any action
Probe way
livenessProbe: indicate whether the container is running. If the survival of the probe fails, kubelet will kill container and the container will be affected by the restart of its strategy. If the container does not provide survival probe, the default status is Success
readinessProbe: indicate whether the container is ready to service requests. If the probe fails ready, the endpoint controller will match the Pod Service endpoint of all remove the IP address of the Pod. Ready state before the initial delay default Failure. If the container is not ready to provide the probe, the default status of Success.
test
Detection probe - detecting the ready
read.yaml
[root@k8s-master mnt]# cat read.yaml apiVersion: v1 kind: Pod metadata: name: readiness-httpget-pod namespace: default spec: containers: - name: readiness-httpget-container Image: wangyanglinux / myapp: v1 imagePullPolicy: IfNotPresent readinessProbe: httpGet: port: 80 path: /index1.html initialDelaySeconds: 1 periodSeconds: 3 [root@k8s-master mnt]#
[root@k8s-master mnt]# vim read.yaml [root@k8s-master mnt]# kubectl create -f read.yaml pod/readiness-httpget-pod created [root@k8s-master mnt]# kubectl get pod NAME READY STATUS RESTARTS AGE myapp-pod 1/1 Running 0 70m readiness-httpget-pod 0/1 Running 0 17s [root@k8s-master mnt]# kubectl describe pod readiness-httpget-pod Name: readiness-httpget-pod Namespace: default Priority: 0 Node: k8s-node01/192.168.180.133 Start Time: Wed, 18 Dec 2019 23:12:59 +0800 Labels: <none> Annotations: <none> Status: Running IP: 10.244.2.10 IPs: IP: 10.244.2.10 Containers: readiness-httpget-container: Container ID: docker://566ff6cdcf44daaba316b796fb8bf6f9563ddd44000c9ae9f572fd0a6719684c Image: wangyanglinux / myapp: v1 Image ID: docker-pullable://wangyanglinux/myapp@sha256:9c3dc30b5219788b2b8a4b065f548b922a34479577befb54b03330999d30d513 Port: <none> Host Port: <none> State: Running Started: Wed, 18 Dec 2019 23:13:01 +0800 Ready: False Restart Count: 0 Readiness: http-get http://:80/index1.html delay=1s timeout=1s period=3s #success=1 #failure=3 Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-gx2h8 (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: default-token-gx2h8: Type: Secret (a volume populated by a Secret) SecretName: default-token-gx2h8 Optional: false QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled <unknown> default-scheduler Successfully assigned default/readiness-httpget-pod to k8s-node01 Normal Pulled 66s kubelet, k8s-node01 Container image "wangyanglinux/myapp:v1" already present on machine Normal Created 66s kubelet, k8s-node01 Created container readiness-httpget-container Normal Started 66s kubelet, k8s-node01 Started container readiness-httpget-container Warning Unhealthy 0s (x22 over 63s) kubelet, k8s-node01 Readiness probe failed: HTTP probe failed with statuscode: 404 [root@k8s-master mnt]# kubectl exec readiness-httpget-pod -it /bin/sh / # ls bin dev etc home lib media mnt proc root run sbin srv sys tmp usr var / # cd /usr/share/nginx /usr/share/nginx # ls -l total 0 drwxr-xr-x 1 root root 24 Feb 25 2018 html /usr/share/nginx # cd html/ /usr/share/nginx/html # ls -l total 8 -rw-r--r-- 1 root root 537 Jan 10 2018 50x.html -rw-r--r-- 1 root root 65 Mar 2 2018 index.html /usr/share/nginx/html # cat index.html Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a> /usr/share/nginx/html # echo "123" >> index1.html /usr/share/nginx/html # exit [root@k8s-master mnt]# kubectl get pod NAME READY STATUS RESTARTS AGE myapp-pod 1/1 Running 1 73m readiness-httpget-pod 1/1 Running 0 3m41s
Note: Due to index1.html does not exist, will lead him to restart, created after the normal manual.
Detection probe - Test of survival
[root@k8s-master mnt]# cat live-exec.yaml apiVersion: v1 kind: Pod metadata: name: liveness-exec-pod namespace: default spec: containers: - name: liveness-exec-container image: busybox imagePullPolicy: IfNotPresent command: ["/bin/sh","-c","touch /tmp/live ; sleep 60; rm -rf /tmp/live; sleep 3600"] livenessProbe: exec: command: ["test","-e","/tmp/live"] initialDelaySeconds: 1 periodSeconds: 3 [root@k8s-master mnt]#
[root@k8s-master mnt]# vim live-exec.yaml [root@k8s-master mnt]# kubectl create -f live-exec.yaml pod/liveness-exec-pod created [root@k8s-master mnt]# kubectl get pod -w NAME READY STATUS RESTARTS AGE liveness-exec-pod 1/1 Running 0 25s myapp-pod 1/1 Running 1 81m readiness-httpget-pod 1/1 Running 0 11m liveness-exec-pod 1/1 Running 1 101s liveness-exec-pod 1/1 Running 2 3m19s ^ Z [ 1] + Stopped kubectl get pod -w
Note: Due to / tmp / live does not exist, it would have been restarted
[root@k8s-master mnt]# cat live-http.yaml apiVersion: v1 kind: Pod metadata: name: liveness-httpget-pod namespace: default spec: containers: - name: liveness-httpget-container Image: wangyanglinux / myapp: v1 imagePullPolicy: IfNotPresent ports: - name: http containerPort: 80 livenessProbe: httpGet: port: http path: /index.html initialDelaySeconds: 1 periodSeconds: 3 timeoutSeconds: 10 [root@k8s-master mnt]#
[root@k8s-master mnt]# kubectl create -f live-http.yaml pod/liveness-httpget-pod created [root@k8s-master mnt]# kubectl get pod NAME READY STATUS RESTARTS AGE liveness-httpget-pod 1/1 Running 0 14s myapp-pod 1/1 Running 1 90m readiness-httpget-pod 1/1 Running 0 20m [root@k8s-master mnt]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES liveness-httpget-pod 1/1 Running 0 26s 10.244.2.12 k8s-node01 <none> <none> myapp-pod 1/1 Running 1 90m 10.244.1.9 k8s-node02 <none> <none> readiness-httpget-pod 1/1 Running 0 20m 10.244.2.10 k8s-node01 <none> <none> [root@k8s-master mnt]# curl 10.244.2.12 Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a> [root@k8s-master mnt]# curl 10.244.2.12/index.html Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a> [root@k8s-master mnt]# kubectl exec liveness-httpget-pod -it -- /bin/sh / # cd /usr/share/nginx/html/ /usr/share/nginx/html # ls -l total 8 -rw-r--r-- 1 root root 537 Jan 10 2018 50x.html -rw-r--r-- 1 root root 65 Mar 2 2018 index.html /usr/share/nginx/html # rm -rf index.html /usr/share/nginx/html # exit [root@k8s-master mnt]# kubectl get pod NAME READY STATUS RESTARTS AGE liveness-httpget-pod 1/1 Running 0 2m24s myapp-pod 1/1 Running 1 92m readiness-httpget-pod 1/1 Running 0 22m [root@k8s-master mnt]# kubectl get pod NAME READY STATUS RESTARTS AGE liveness-httpget-pod 1/1 Running 1 2m41s myapp-pod 1/1 Running 1 92m readiness-httpget-pod 1/1 Running 0 22m
Description: Delete Html, will find Pod start restart.
[root@k8s-master mnt]# cat live-tcp.yaml apiVersion: v1 kind: Pod metadata: name: probe-tcp spec: containers: - name: nginx Image: wangyanglinux / myapp: v1 livenessProbe: initialDelaySeconds: 5 timeoutSeconds: 1 tcpSocket: port: 8080 periodSeconds: 3 [root@k8s-master mnt]#
[root@k8s-master mnt]# vim live-tcp.yaml [root@k8s-master mnt]# kubectl create -f live-tcp.yaml pod/probe-tcp created [root@k8s-master mnt]# kubectl get pod NAME READY STATUS RESTARTS AGE liveness-httpget-pod 1/1 Running 1 9m24s myapp-pod 1/1 Running 1 99m probe-tcp 1/1 Running 0 5s readiness-httpget-pod 1/1 Running 0 29m [root@k8s-master mnt]# kubectl get pod NAME READY STATUS RESTARTS AGE liveness-httpget-pod 1/1 Running 1 9m37s myapp-pod 1/1 Running 1 99m probe-tcp 1/1 Running 1 18s readiness-httpget-pod 1/1 Running 0 29m [root@k8s-master mnt]# kubectl get pod NAME READY STATUS RESTARTS AGE liveness-httpget-pod 1/1 Running 1 9m41s myapp-pod 1/1 Running 1 99m probe-tcp 1/1 Running 1 22s readiness-httpget-pod 1/1 Running 0 29m [root@k8s-master mnt]# kubectl get pod NAME READY STATUS RESTARTS AGE liveness-httpget-pod 1/1 Running 1 9m43s myapp-pod 1/1 Running 1 99m probe-tcp 1/1 Running 1 24s readiness-httpget-pod 1/1 Running 0 29m [root@k8s-master mnt]# kubectl get pod NAME READY STATUS RESTARTS AGE liveness-httpget-pod 1/1 Running 1 9m44s myapp-pod 1/1 Running 1 99m probe-tcp 1/1 Running 1 25s readiness-httpget-pod 1/1 Running 0 29m [root@k8s-master mnt]# kubectl get pod NAME READY STATUS RESTARTS AGE liveness-httpget-pod 1/1 Running 1 10m myapp-pod 1/1 Running 1 100m probe-tcp 1/1 Running 3 47s readiness-httpget-pod 1/1 Running 0 29m [root@k8s-master mnt]# kubectl delete -f live-tcp.yaml pod "probe-tcp" deleted [root@k8s-master mnt]#
Description: Delete Html, will find Pod start restart.
Pod hook
Pod hook (hook) is managed by the Kubernetes kubelet initiated when the process starts before the container or in the container into the
running process before termination, which is included in the life cycle of the container. Pod simultaneously for all the containers are arranged hook
type comprise two Hook:
Exec : the implementation of a command
HTTP : HTTP request
Restart Policy
PodSpec has a restartPolicy field, possible values Always, OnFailure and Never. The default is
Always. Pod restartPolicy suitable for all containers. restartPolicy refers only to pass on the same node
kubelet restart the container. Failure of the vessel by the kubelet to five minutes for the upper limit of the exponential backoff delay (10 seconds, 20 seconds, 40
seconds ...) restart and reset after the successful implementation of ten minutes. Pod as the document, once bound to a node, Pod will
never be re-bound to another node.
Pod phase
Pod status field is a PodStatus objects, PodStatus has a phase field.
Pod phase (Phase) is in its life cycle Pod simple macro overview. The stage is not a comprehensive summary of container or Pod, nor as to the comprehensive state machine
Pod phase number and meaning is strictly specified. In addition to this document include a state, there should no longer assume that other phase values Pod
several common value
- Suspend (Pending): Pod Kubernetes system has been accepted, but there is one or more containers image has not been created. Pod latency time schedule comprises time through the network, and downloading the image, which might take
- Running (Running): The Pod has been bound to a node, Pod all of the containers have been created. At least one container is running, or is in a state to start or restart
- Success (Succeeded): Pod in all containers have been successfully terminated, and will not restart
- Failed (Failed): Pod in all containers have been terminated, and at least one container is terminated because of failure. That is, the container exit 0 state or in a non-terminated system
- Unknown (Unknown): For some reason unable to obtain state Pod, usually because of a failure where the host communication with the Pod
[root@k8s-master mnt]# vim post.yaml [root@k8s-master mnt]# kubectl create -f post.yaml pod/lifecycle-demo created [root@k8s-master mnt]# kubectl get pod NAME READY STATUS RESTARTS AGE lifecycle-demo 1/1 Running 0 9s liveness-httpget-pod 1/1 Running 1 40m myapp-pod 1/1 Running 1 130m readiness-httpget-pod 1/1 Running 0 60m [root@k8s-master mnt]# kubectl exec lifecycle-demo -it -- /bin/bash OCI runtime exec failed: exec failed: container_linux.go:346: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown command terminated with exit code 126 [root@k8s-master mnt]# kubectl exec lifecycle-demo -it -- /bin/sh / # cd /usr/share/message /bin/sh: cd: can't cd to /usr/share/message / # cat /usr/share/message Hello from the postStart handler / # exit [root@k8s-master mnt]# cat post.yaml apiVersion: v1 kind: Pod metadata: name: lifecycle-demo spec: containers: - name: lifecycle-demo-container Image: wangyanglinux / myapp: v1 lifecycle: mail field: exec: command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"] Transfers: exec: command: ["/bin/sh", "-c", "echo Hello from the poststop handler > /usr/share/message"] [root@k8s-master mnt]#