Kubernetes in Pod health check

1. What is a health check

Kubernetes architecture, each node will have a kubeletcontainer health check (Container Probe) task is a Kubeletperiodic execution.

Kubelet Pod by calling the container Handlerto perform the inspection operation, Handlerthere are the following three types:

• ExecAction: execute specific commands in a container, command exits with 0 (command return value:? $) Indicates success
• TCPSocketAction: a TCP IP address and check the container according to a specific port, port access / open / exposed for success
• HTTPGetAction: HTTP request to launch a container according to IP, port and access paths, return a status code indicating success if between 200-400

Each inspection actions are likely to have three kinds of return states:

• Success: it represented by the health check
• Failure: that there is no health checks
• Unknown: Check the action represents a failure

2, the probe classification

When you create a Pod, by livenessand readinessoperation of two ways to detect Pod's content.

2.1, LivenessProbe probe (probe viability)

Determining whether the health of the container (Running state) and fed back to Kubelet. In fact, there are many applications for a long time will gradually into the background after running an unusable state, and can only be restored by restarting Pod operation, the 存活性probe mechanism can be found on such issues, combined with restart strategy based on the detection result to trigger subsequent execution .

kubernetes viability detection probe support into three: ExecAction, TCPSocketAction and HTTPGetAction

If a container is not LivenessProbe probe, then kubelet will think LivenessProbe probe of the container return value will always be Success.

2.2, ReadinessProbe probe (probe-ready)

Judge container service is available (Ready status) can provide services only reached Pod Ready to receive request state, when the vessel ran the business up to the state of the container is Ready, charge that the container probe fails, if the probe fails , the system will back-end Service Endpoint list remove their Pod IP, follow-up and then return to Ready, the probe will be its success Pod IP Endpoint added back to the list.

3, the probe implementation

LivenessProbeAnd ReadinessProbecan be configured to achieve the following three ways probe:

ReadinessProbeConfiguration and LivenessProbesimilarly, simply yaml is livenessProberevised toreadinessProbe

3.1、Container Exec

Create a container, by checking whether a file exists to determine whether the normal operation of the vessel, if the file exists will return status code of 0, 30 seconds after the vessel run, the file will be deleted, LivenessProbe check fails will restart vessel.

apiVersion: v1
kind: Pod
metadata:
  name: exec
spec:
  containers:
    - name: nginx
      image: nginx:1.13
      ports:
        - containerPort: 80
      args:
        - /bin/sh
        - -c
        - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
      livenessProbe:
        exec:
          command:
            - cat
            - /tmp/healthy
        initialDelaySeconds: 5
        periodSeconds: 5

Detects the directory does not exist

[root@k8s-master01 health]# kubectl describe pod exec  |grep "Liveness"
    Liveness:       exec [cat /tmp/healthy] delay=5s timeout=1s period=5s #success=1 #failure=3
  Warning  Unhealthy  4s (x3 over 14s)  kubelet, k8s-node01  Liveness probe failed: cat: /tmp/healthy: No such file or directory

3.2、HTTP Check

Nginx create a container, by accessing /index.html to determine whether the survival of the service, remove the file manually, check can lead to failure, thus restarting the container

apiVersion: v1
kind: Pod
metadata:
  name: httpget
spec:
  containers:
    - name: nginx
      image: nginx:1.13
      ports:
        - containerPort: 80
      livenessProbe:
        httpGet:
          path: /index.html # 访问路径
          port: 80          # 容器端口
        initialDelaySeconds: 5
        periodSeconds: 5

Nginx index.html file manually remove the container

[root@k8s-master01 health]# kubectl exec -it httpget bash
root@httpget:/# mv /usr/share/nginx/html/index.html  /tmp/

When accessing index.html not return a status code 200, the container will reboot

[root@k8s-master01 health]# kubectl describe pod httpget |grep "Liveness"
    Liveness:       http-get http://:80/index.html delay=5s timeout=1s period=5s #success=1 #failure=3
  Warning  Unhealthy  59s (x3 over 69s)   kubelet, k8s-node01  Liveness probe failed: HTTP probe failed with statuscode: 404

3.3、TCP Socket Check

Through the IP address (target IP address requests a connection, the default is Pod IP) to perform a TCP inspection and port number, if you can establish a TCP connection, then think of a container healthy, it will be more efficient HTTP-based detection method ratio (HTTP seven layer, TCP is four), more conservation of resources, but the accuracy of micro low, can build a successful does not mean that the page can be displayed.

apiVersion: v1
kind: Pod
metadata:
  name: tcpSocket
spec:
  containers:
    - name: nginx
      image: nginx:1.13
      ports:
        - containerPort: 80
      livenessProbe:
        tcpSocket:
          port: 80
        initialDelaySeconds: 3
        periodSeconds: 3

4, probing behavior parameters

initiaDelaySeconds // 容器启动之后多久开始检测，默认为0秒
periodSeconds      // 每隔多久检测一次，默认为10秒，最小为1秒
failureThreshold   // 检测失败几次后则认为健康检测失败，默认为3次
successThreshold   // 从检测错误到成功需要几次才认为健康检测成功，默认为1次
timeoutSeconds     // 执行检测命令的最长时间，默认为1秒，最小为1秒

httpGet的属性
		host：主机名或IP
		scheme：链接类型，HTTP或HTTPS，默认为HTTP
		path：请求路径
		httpHeaders：自定义请求头
		port：请求端口