1. Method analysis
- Each node needs to be monitored, so the DaemonSet type can be used to manage node_exporter
- Add the tolerance configuration of the node (due to the taint set on the current master node)
- Mount the system file information in the host (used to obtain the system information of the host of each node)
Second, the yaml file of the pod
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: node-exporter
namespace: monitor
labels:
app: node-exporter
spec:
selector:
matchLabels:
app: node-exporter
template:
metadata:
labels:
app: node-exporter
spec:
hostPID: true
hostIPC: true
hostNetwork: true
nodeSelector:
kubernetes.io/os: linux
containers:
- name: node-exporter
image: prom/node-exporter:v1.0.1
args:
- --web.listen-address=$(HOSTIP):9100
- --path.procfs=/host/proc
- --path.sysfs=/host/sys
- --path.rootfs=/host/root
- --collector.filesystem.ignored-mount-points=^/(dev|proc|sys|var/lib/docker/.+)($|/)
- --collector.filesystem.ignored-fs-types=^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|sysfs|tracefs)$
ports:
- containerPort: 9100 #对外暴露接口
env:
- name: HOSTIP
valueFrom:
fieldRef:
fieldPath: status.hostIP
resources:
requests:
cpu: 150m
memory: 180Mi
limits:
cpu: 150m
memory: 180Mi
securityContext:
runAsNonRoot: true
runAsUser: 65534
volumeMounts:
- name: proc
mountPath: /host/proc
- name: sys
mountPath: /host/sys
- name: root
mountPath: /host/root
mountPropagation: HostToContainer
readOnly: true
tolerations:
- operator: "Exists" #设置亏点
volumes:
- name: proc
hostPath:
path: /proc
- name: dev
hostPath:
path: /dev
- name: sys
hostPath:
path: /sys
- name: root
hostPath:
path: /
Three, on how to connect the monitoring data of each node to Prometheus
Method 1: Add the service of each node-exporter to the target list by static configuration, such as:
- job_name: 'k8s-slave1'
static_configs:
- targets: ['10.3.153.201:9100']
The problems caused by the above methods:
* 集群节点的增删,都需要手动维护列表
* target列表维护量随着集群规模增加
Method 2: Configure a Service, mount the node-exporter service on the backend, and configure the address of the Service to the target, such as:
Bringing new problems, the status of each node's node-exporter cannot be visually seen in the target