Environment Description:
| CPU name | Operating system version | ip | docker version | omelets version | helm version | Configuration | Remark |
|---|---|---|---|---|---|---|---|
| master | Centos 07/06/1810 | 172.27.9.131 | Docker 18.09.6 | V1.14.2 | v2.14.3 | 2C2G | master host |
| node01 | Centos 07/06/1810 | 172.27.9.135 | Docker 18.09.6 | V1.14.2 | v2.14.3 | 2C2G | node node |
| node02 | Centos 07/06/1810 | 172.27.9.136 | Docker 18.09.6 | V1.14.2 | v2.14.3 | 2C2G | node node |
k8s cluster deployment see: Centos7.6 deployment k8s (v1.14.2) cluster
k8s learning materials See: basic concepts, kubectl command and information sharing
A, prometheus Profile
PrometheusIs an open source system monitoring and alerting tool kit, originally built in soundcloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, the project has a very active community of developers and users. It is now an independent open source project, independent maintenance in any company, in 2016 joined the native cloud computing foundation, becoming the second kubernetes managed projects.
Features:
- Multidimensional data model names and values with a measure of time-series data identification
- Flexible query language
- It does not depend on the distributed storage; single server node is autonomous
- Time series collected by the pull model on http
- Time series by the intermediate support push gateway
- Static configuration or through service discovery to find the target
- A variety of graphics and dashboards support mode
In the micro-service architecture, with their good support for multidimensional data collection and queries.
Two, prometheus architecture
Prometheus metric data acquired from the jobs, but also obtain metric data temporary jobs directly or via push gateway. It is in the local memory of all the samples are acquired, and the data run rules, the existing data and new polymerization time series record, or generate an alarm. By Grafana or other API consumers can view visual data collected.
Three, Prometheus Operator Introduction
Prometheus Operator是CoreOS开发的基于Prometheus的Kubernetes监控方案
Prometheus Operator：整合Kubernetes和Prometheus的最佳方法。
Prometheus Operator 功能更特点：
- 创建/销毁: 在Kubernetes namespace中更容易启动一个prometheus实例，一个特定的应用程序或团队更容易使用Operator。
- 简单配置: 配置Prometheus的基础，比如versions, persistence, retention policies和来自本机kubernetes资源的副本。
- 通过标签的目标服务: 基于常见的Kubernetes label查询，自动生成监控target 配置；无需学习prometheus特定的配置语言。
工作流程：
四、Prometheus Operator部署
1.安装文件下载
[root@master ~]# git clone https://github.com/coreos/kube-prometheus.git
2.镜像下载
下载镜像：
docker pull registry.cn-hangzhou.aliyuncs.com/loong576/configmap-reload:v0.0.1
docker pull registry.cn-hangzhou.aliyuncs.com/loong576/alertmanager:v0.18.0
docker pull registry.cn-hangzhou.aliyuncs.com/loong576/kube-state-metrics:v1.8.0
docker pull registry.cn-hangzhou.aliyuncs.com/loong576/kube-rbac-proxy:v0.4.1
docker pull registry.cn-hangzhou.aliyuncs.com/loong576/node-exporter:v0.18.1
docker pull registry.cn-hangzhou.aliyuncs.com/loong576/k8s-prometheus-adapter-amd64:v0.5.0
docker pull registry.cn-hangzhou.aliyuncs.com/loong576/prometheus-config-reloader:v0.33.0
docker pull registry.cn-hangzhou.aliyuncs.com/loong576/prometheus:v2.11.0
docker pull registry.cn-hangzhou.aliyuncs.com/loong576/prometheus-operator:v0.33.0
docker pull registry.cn-hangzhou.aliyuncs.com/loong576/grafana:6.4.3
打tag：
docker tag registry.cn-hangzhou.aliyuncs.com/loong576/configmap-reload:v0.0.1 quay.io/coreos/configmap-reload:v0.0.1
docker tag registry.cn-hangzhou.aliyuncs.com/loong576/alertmanager:v0.18.0 quay.io/prometheus/alertmanager:v0.18.0
docker tag registry.cn-hangzhou.aliyuncs.com/loong576/kube-state-metrics:v1.8.0 quay.io/coreos/kube-state-metrics:v1.8.0
docker tag registry.cn-hangzhou.aliyuncs.com/loong576/kube-rbac-proxy:v0.4.1 quay.io/coreos/kube-rbac-proxy:v0.4.1
docker tag registry.cn-hangzhou.aliyuncs.com/loong576/node-exporter:v0.18.1 quay.io/prometheus/node-exporter:v0.18.1
docker tag registry.cn-hangzhou.aliyuncs.com/loong576/k8s-prometheus-adapter-amd64:v0.5.0 quay.io/coreos/k8s-prometheus-adapter-amd64:v0.5.0
docker tag registry.cn-hangzhou.aliyuncs.com/loong576/prometheus-config-reloader:v0.33.0 quay.io/coreos/prometheus-config-reloader:v0.33.0
docker tag registry.cn-hangzhou.aliyuncs.com/loong576/prometheus:v2.11.0 quay.io/prometheus/prometheus:v2.11.0
docker tag registry.cn-hangzhou.aliyuncs.com/loong576/prometheus-operator:v0.33.0 quay.io/coreos/prometheus-operator:v0.33.0
docker tag registry.cn-hangzhou.aliyuncs.com/loong576/grafana:6.4.3 grafana/grafana:6.4.3删除镜像：
docker rmi -f registry.cn-hangzhou.aliyuncs.com/loong576/configmap-reload:v0.0.1
docker rmi -f registry.cn-hangzhou.aliyuncs.com/loong576/alertmanager:v0.18.0
docker rmi -f registry.cn-hangzhou.aliyuncs.com/loong576/kube-state-metrics:v1.8.0
docker rmi -f registry.cn-hangzhou.aliyuncs.com/loong576/kube-rbac-proxy:v0.4.1
docker rmi -f registry.cn-hangzhou.aliyuncs.com/loong576/node-exporter:v0.18.1
docker rmi -f registry.cn-hangzhou.aliyuncs.com/loong576/k8s-prometheus-adapter-amd64:v0.5.0
docker rmi -f registry.cn-hangzhou.aliyuncs.com/loong576/prometheus-config-reloader:v0.33.0
docker rmi -f registry.cn-hangzhou.aliyuncs.com/loong576/prometheus:v2.11.0
docker rmi -f registry.cn-hangzhou.aliyuncs.com/loong576/prometheus-operator:v0.33.0
docker rmi -f registry.cn-hangzhou.aliyuncs.com/loong576/grafana:6.4.3以上三个步骤所有node节点都执行
3.执行安装
[root@master kube-prometheus]# kubectl create -f manifests/setup
[root@master kube-prometheus]# kubectl create -f manifests/4.资源查看
[root@master kube-prometheus]# kubectl get all -n monitoring
[root@master kube-prometheus]# kubectl get prometheus --all-namespaces -o wide
所有资源都在monitoring命名空间中。
5.更改访问方式
[root@master kube-prometheus]# kubectl edit -n monitoring service prometheus-k8s
service/prometheus-k8s edited
[root@master kube-prometheus]# kubectl edit -n monitoring service grafana
service/grafana edited
[root@master kube-prometheus]# kubectl edit -n monitoring service alertmanager-main
service/alertmanager-main edited
分别修改service prometheus-k8s、grafana和alertmanager-main，service类型为NodePort，端口分别为30021、30022、30023
五、页面展示
1.访问 Prometheus
2.访问 Alert Manager
3.访问 Grafana
http://172.27.9.131:30022 用户名密码都为admin
内置模板查看：
集群资源查看：
kubelet查看：
StatefulSets查看：
Pod查看：
API查看：
namespace查看：
Check each node resources:
Prometheus View:
Other templates:
Built-in template is very rich, but you can also download additional templates, such as' 1 Node Exporter for Prometheus monitor display billboards update! ': Https://grafana.com/api/dashboards/8919/revisions/10/download
Six, Prometheus Operator uninstall
[root@master kube-prometheus]# kubectl delete --ignore-not-found=true -f manifests/ -f manifests/setupIn this paper all scripts and configuration files have been uploaded: K8S practice (XII): Prometheus Operator cluster monitoring Kubernetes