When creating a Pod, you can use liveness and readiness to detect the operation of the Pod's inner container.
1. Liveness can be used to check the survival of the application in the container, and to detect whether the container is really "running", that is, "live detection". If the check fails, kubelet will kill the container process and stop the container. Whether to restart the container depends on the restart strategy of the Pod.
2. Readiness checks whether the application in the container can normally provide services to the outside world, that is, "ready". For example, the nginx container cannot provide HTTP services to the outside during the reload configuration. If the detection fails, the Endpoint Controller will delete the Pod's IP from the service.
3. startupProbe can be used to determine whether the container has been started, such as the example of slow start of the container mentioned above. We can use parameters to ensure that there is enough time to deal with the "extra-long" startup time. If the detection fails, the operation is the same as that of livenessProbe. This Probe was only added in version 1.16 and became beta in version 1.18. In other words, if your Kubernetes version is less than 1.18, you need to explicitly enable this function in the feature gate in the startup parameters of kube-apiserver. You can refer to this document to see how to configure this parameter.
All three probes have the following parameters:
initialDelaySeconds: The number of seconds to wait before starting the liveness and readiness probes.
periodSeconds: Check the frequency of the probe.
timeoutSeconds: The number of seconds before marking the probe as timed out (failed the health check).
successThreshold: The minimum number of consecutive successful checks that the probe needs to pass.
failureThreshold: The number of retries before marking the probe as failed. For liveness probes, this will cause the Pod to restart. For readiness probes, the Pod will be marked as unready.
Readiness probe
The readiness probe allows kubelet to know when the application is ready to accept new traffic. If the application needs some time to initialize the state after the process starts, configure the readiness probe to let Kubernetes wait before sending new traffic. The main function of the readiness probe is to direct traffic to the deployment after the service.
One important thing about the readiness probe is that it runs during the entire life cycle of the container. This means that the readiness probe will not only run at startup, but it will also run repeatedly while the Pod is running. This is to handle situations where the application is temporarily unavailable (such as when loading large amounts of data, waiting for external connections). In this case, we don't have to kill the application, we can wait for it to recover. The readiness probe can be used to detect this situation and send traffic to these pods after they pass the readiness check again.
Liveness probe
The liveness probe is used to restart unhealthy containers. Kubelet will periodically ping the liveness probe to determine the health status and kill the Pod if the liveness check fails. The liveness check can help the application recover from a deadlock. If the liveness check is not performed, Kubernetes will think that the Pod in the deadlock is in a healthy state, because from the perspective of Kubernetes, the child process of the Pod is still running and healthy. By configuring the liveness probe, the kubelet can detect that the application is in an unhealthy state and restart the Pod to restore availability.
Startup probe
The startup probe is similar to the readiness probe, but it is only executed at startup and can be optimized for slow-starting containers or applications with unpredictable behavior during initialization. With the readiness probe, we can configure initialDelaySeconds to determine how long the readiness probe will wait before it is ready.
Suppose there is an application that occasionally needs to download a large amount of data. Since initialDelaySeconds is a static number, even if the application does not require such a long initialization waiting time, we must set the most conservative waiting time. Through the startup probe, we can configure failureThreshold and periodSeconds to solve this problem. For example, set failureThreshold to 15 and periodSeconds to 5. This means that the application will have a startup time of 10x5=75s before it fails.
Configure the probe
Now that we understand the different types of probes, here are three different ways to configure each probe.
The HTTP
kubelet sends an HTTP GET request to the endpoint and checks the 2xx or 3xx response. We can reuse existing HTTP endpoints or set up lightweight HTTP servers for detection (for example, Express server with /healthz endpoint). The HTTP probe contains other additional parameters:
host: The host name to connect to (default: pod's IP).
scheme: HTTP (default) or HTTPS.
path: The path on the HTTP/S server.
httpHeaders: Custom headers (if headers are required for authentication, CORS settings, etc.).
port: The port name or port number for accessing the server.
TCP
If you only need to check whether a TCP connection can be established, you can specify a TCP probe. If a TCP connection is established, the Pod is marked as healthy. For gRPC or FTP servers that are not suitable for using HTTP probes, TCP probes may be useful.
Command
The probe can be configured to run shell commands. If the exit code returned by the command is 0, the check passes, otherwise the Pod will be marked as unhealthy. This type of probe is useful if you do not want to expose the HTTP server and port, or if you want to check the initialization steps through commands (for example, check whether a configuration file has been created, run CLI commands).
