Monitoring of Kubernetes nodes in Prometheus

1. Method analysis

  • Each node needs to be monitored, so the DaemonSet type can be used to manage node_exporter
  • Add the tolerance configuration of the node (due to the taint set on the current master node)
  • Mount the system file information in the host (used to obtain the system information of the host of each node)

Second, the yaml file of the pod

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: monitor
  labels:
    app: node-exporter
spec:
  selector:
    matchLabels:
      app: node-exporter
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      hostPID: true
      hostIPC: true
      hostNetwork: true
      nodeSelector:
        kubernetes.io/os: linux
      containers:
      - name: node-exporter
        image: prom/node-exporter:v1.0.1
        args:
        - --web.listen-address=$(HOSTIP):9100
        - --path.procfs=/host/proc
        - --path.sysfs=/host/sys
        - --path.rootfs=/host/root
        - --collector.filesystem.ignored-mount-points=^/(dev|proc|sys|var/lib/docker/.+)($|/)
        - --collector.filesystem.ignored-fs-types=^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|sysfs|tracefs)$
        ports:
        - containerPort: 9100             #对外暴露接口
        env:
        - name: HOSTIP
          valueFrom:
            fieldRef:
              fieldPath: status.hostIP
        resources:
          requests:
            cpu: 150m
            memory: 180Mi
          limits:
            cpu: 150m
            memory: 180Mi
        securityContext:
          runAsNonRoot: true
          runAsUser: 65534
        volumeMounts:
        - name: proc
          mountPath: /host/proc
        - name: sys
          mountPath: /host/sys
        - name: root
          mountPath: /host/root
          mountPropagation: HostToContainer
          readOnly: true
      tolerations:
      - operator: "Exists"       #设置亏点
      volumes:
      - name: proc
        hostPath:
          path: /proc
      - name: dev
        hostPath:
          path: /dev
      - name: sys
        hostPath:
          path: /sys
      - name: root
        hostPath:
          path: /

Monitoring of Kubernetes nodes in Prometheus

Three, on how to connect the monitoring data of each node to Prometheus

Method 1: Add the service of each node-exporter to the target list by static configuration, such as:

    - job_name: 'k8s-slave1'
      static_configs:
      - targets: ['10.3.153.201:9100']

Monitoring of Kubernetes nodes in Prometheus

The problems caused by the above methods:

* 集群节点的增删,都需要手动维护列表
* target列表维护量随着集群规模增加

Method 2: Configure a Service, mount the node-exporter service on the backend, and configure the address of the Service to the target, such as:

Bringing new problems, the status of each node's node-exporter cannot be visually seen in the target

Guess you like

Origin blog.51cto.com/12965094/2679472