Two mechanisms Kubernetes health check: Liveness detection and Readiness detection, and practical application of health checks Scale Up and Rolling Update scene.
kubelet start using the probe to know when to start Container application. If such a probe configuration, it disables the activity and readiness checks until successful, to ensure that they do not interfere with the probes start the application. This can be used to start slow vessel inspection activities, prevent them from being killed before kubelet up and running.
The definition of an active command exec
In this exercise, you will create a Pod, Pod can be based on the k8s.gcr.io/busybox
image a Container running . This is the Pod profile:
pods/probe/exec-liveness.yaml apiVersion: v1 kind: Pod metadata: labels: test: liveness name: liveness-exec spec: containers: - name: liveness image: k8s.gcr.io/busybox args: - /bin/sh - -c - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600 livenessProbe: exec: command: - cat - /tmp/healthy initialDelaySeconds: 5 periodSeconds: 5
In the configuration file, you can see the Pod has a single Container. This periodSeconds
field specifies kubelet should be performed once every 5 seconds to detect activity. This initialDelaySeconds
field tells kubelet before the implementation of a probe should wait for 5 seconds. To perform the detection, kubelet cat /tmp/healthy
in a container execution command . If the command is successful, it returns 0, and kubelet believe Container active and healthy. If the command returns a non-zero value, the kubelet will kill Container and restart it.
In the first 30 seconds in the life of the container, there is a /tmp/healthy
document. Thus, the previous 30 seconds, the command cat /tmp/healthy
returns a success code. After 30 seconds, cat /tmp/healthy
return failure code.
Within 30 seconds, see the Pod events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 24s 24s 1 {default-scheduler } Normal Scheduled Successfully assigned liveness-exec to worker0 23s 23s 1 {kubelet worker0} spec.containers{liveness} Normal Pulling pulling image "k8s.gcr.io/busybox" 23s 23s 1 {kubelet worker0} spec.containers{liveness} Normal Pulled Successfully pulled image "k8s.gcr.io/busybox" 23s 23s 1 {kubelet worker0} spec.containers{liveness} Normal Created Created container with docker id 86849c15382e; Security:[seccomp=unconfined] 23s 23s 1 {kubelet worker0} spec.containers{liveness} Normal Started Started container with docker id 86849c15382e
After 35 seconds, once again view the Pod event: At the bottom of the output, a message indicates active probe has failed, and the container has been killed and recreated.
FirstSeen LastSeen Count From SubobjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 37s 37s 1 {default-scheduler } Normal Scheduled Successfully assigned liveness-exec to worker0 36s 36s 1 {kubelet worker0} spec.containers{liveness} Normal Pulling pulling image "k8s.gcr.io/busybox" 36s 36s 1 {kubelet worker0} spec.containers{liveness} Normal Pulled Successfully pulled image "k8s.gcr.io/busybox" 36s 36s 1 {kubelet worker0} spec.containers{liveness} Normal Created Created container with docker id 86849c15382e; Security:[seccomp=unconfined] 36s 36s 1 {kubelet worker0} spec.containers{liveness} Normal Started Started container with docker id 86849c15382e 2s 2s 1 {kubelet worker0} spec.containers{liveness} Warning Unhealthy Liveness probe failed: cat: can't open '/tmp/healthy': No such file or directory
Wait 30 seconds, and verify that the vessel has been restarted:
kubectl get pod liveness-exec NAME READY STATUS RESTARTS AGE liveness-exec 1/1 Running 1 1m
The definition of active HTTP request
Another activity probes HTTP GET request. This is based on k8s.gcr.io/liveness Pod profile image to run container.
pods/probe/http-liveness.yaml apiVersion: v1 kind: Pod metadata: labels: test: liveness name: liveness-http spec: containers: - name: liveness image: k8s.gcr.io/liveness args: - /server livenessProbe: httpGet: path: /healthz port: 8080 httpHeaders: - name: Custom-Header value: Awesome initialDelaySeconds: 3 periodSeconds: 3
In the configuration file, you can see the Pod has a single Container. This periodSeconds
field specifies kubelet should be performed once every 3 seconds to detect activity. This initialDelaySeconds
field tells kubelet before the implementation of a probe should wait for three seconds. To perform the detection, kubelet an HTTP GET request to a server running Container and listening on port 8080. If the server /healthz
path handler returns a success code, kubelet believe Container is active and running in good condition. If the handler returns a failure code, kubelet will kill Container and restart it.
Any code that is greater than or equal to 200 and less than 400 were successfully expressed. Any other code both indicate failure.
You can server.go in view the source code for the server .
Container in /healthz
active in the first 10 seconds, the processing procedure returns to state 200. Thereafter, the processing procedure returns to state 500.
http.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) { duration := time.Now().Sub(started) if duration.Seconds() > 10 { w.WriteHeader(500) w.Write([]byte(fmt.Sprintf("error: %v", duration.Seconds()))) } else { w.WriteHeader(200) w.Write([]byte("ok")) } })
Three seconds after the container starts, kubelet will begin performing a health check. Therefore, the previous health check will succeed. But 10 seconds later, the health check will fail, and kubelet will terminate and restart the Container.
Defined TCP liveness probe
https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
Definition of readiness probes
Sometimes, the application is temporarily unable to service traffic. For example, an application may need to load large data files or configuration during startup, or after starting dependent on external services. In this case, you do not want to kill the application but do not want to send the request. Kubernetes provides these circumstances is ready to probe detection and mitigation. Container with a container vessel of its report is not ready and can not receive traffic over Kubernetes Services.
Similar readiness configuration and liveness. The only difference is that you use a readinessProbe
field instead of livenessProbe
the field.
readinessProbe: exec: command: - cat - /tmp/healthy initialDelaySeconds: 5 periodSeconds: 5
HTTP and TCP probe configuration is also the same readiness and activities of the probe.
readiness and liveness can be used in parallel for the same container. Both can be used simultaneously to ensure that traffic does not reach the container is not ready, and can ensure that the container is restarted in case of failure.