快速上手：Prometheus和Grafana入门教程

一、简介

这篇博客详细介绍了如何安装和部署Prometheus和Grafana，以及如何使用它们来监控不同的应用程序和系统资源。还介绍了如何安装Prometheus的各种Exporter，并演示了如何使用node_exporter、blackbox_exporter和mysql_exporter来收集和展示不同的指标数据。利用Grafana提供的丰富的可视化功能，实现了对node、blackbox、mysql等应用程序的可视化监控。
如果您正在寻找一份详细的Prometheus和Grafana安装部署指南，以及如何使用它们来监控不同应用程序和系统资源的实用教程，本文将会是一个很好的参考。

二、安装Prometheus

2.1 二进制安装

github：https://github.com/prometheus/prometheus/releases/tag/v2.40.7

官网地址：https://prometheus.io

2.2 添加到system服务

注意修改ExecStart为你安装Prometheus的目录

cat <<-"EOF" > /etc/systemd/system/prometheus.service
[Unit]
Description="prometheus"
Documentation=https://prometheus.io/
After=network.target

[Service]
Type=simple
ExecStart=/data/prometheus/prometheus  --config.file=/data/prometheus/prometheus.yml --storage.tsdb.path=/data/prometheus/data --web.enable-lifecycle

Restart=on-failure
RestartSecs=5s
SuccessExitStatus=0
LimitNOFILE=655360
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=prometheus


[Install]
WantedBy=multi-user.target
EOF

执行命令启动prometheus服务

systemctl daemon-reload
systemctl restart prometheus
systemctl status prometheus

# 查看端口 进程 日志
ss -ntlp |grep 9090
ps -ef |grep prometheus |grep -v grep 

tail -100  /var/log/messages |grep prometheus

2.3 访问UI

浏览器访问IP+9090端口

三、安装Granfa

github：https://github.com/grafana/grafana

创建映射目录

cd /data
mkdir -p grafana-server/
cd grafana-server/
mkdir -p etc/
mkdir -p mysql/data
mkdir -p grafana/data

拉取镜像

docker pull mysql:8.0
docker pull grafana/grafana

3.1 Docker部署Mysql

创建mysql docker容器

docker run -p 33060:3306 --name mysql8 \
-v /data/grafana-server/mysql/data:/var/lib/mysql \
-e MYSQL_ROOT_PASSWORD=123456 \
-e  MYSQL_DATABASE=grafana \
-e  MYSQL_USER=grafana \
-e  MYSQL_PASSWORD=123456 \
-e  TZ=Asia/Shanghai \
--restart=always \
-d mysql:8.0

192.168.153.128是我的宿主机IP。我将mysql容器的3306端口映射到了宿主机的33060端口

测试连接

3.2 Docker部署Grafana

将grafana容器内的grafana配置文件拷贝出来

docker run -itd --name grafana-server grafana/grafana:latest
docker cp grafana-server:/var/lib/grafana /data/grafana-server/grafana/data/
docker stop grafana-server
docker rm grafana-server

编辑grafana配置文件修改存储库为mysql

vim /data/grafana-server/etc/grafana.ini

[database]
# You can configure the database connection by specifying type, host, name, user and password
# as separate properties or as on string using the url properties.

# Either "mysql", "postgres" or "sqlite3", it's your choice
type = mysql
host = 192.168.153.128:33060
name = grafana
user = grafana
# If the password contains # or ; you have to wrap it with triple quotes. Ex """#password;"""
password = 123456

# Use either URL or the previous fields to configure the database
# Example: mysql://user:secret@host:port/database
; url = mysql://grafana:[email protected]:33060/grafana
max_idle_conn = 2

# Max conn setting default is 0 (mean not set)
max_open_conn = 100

# Connection Max Lifetime default is 14400 (means 14400 seconds or 4 hours)
conn_max_lifetime = 600

创建容器

docker run -p 3000:3000 --name grafana \
-v /data/grafana-server/grafana/data/grafana:/var/lib/grafana \
-v /data/grafana-server/etc/grafana.ini:/etc/grafana/grafana.ini \
-v /etc/localtime:/etc/localtime \
--restart=always \
-d grafana/grafana

docker ps |grep grafana

docker stop grafana

docker rm grafana

docker logs grafana

3.3 访问UI

浏览器访问IP+3000端口
初始用户名：admin
初始密码：admin

登录后提示修改密码，也可以不修改，点击skip跳过。

3.4 添加Prometheus为数据源

四、安装Exporter收集数据

4.1 node_exporter 服务器监控

下载解压

Github：https://github.com/prometheus/node_exporter/releases

解压压缩包，重命名文件

tar -zxvf node_exporter-1.5.0.linux-arm64.tar.gz 
mv node_exporter-1.5.0.linux-arm64 node_exporter

启动node_exporter服务

修改ExecStart为解压目录，然后直接粘贴回车。

cat <<EOF> /etc/systemd/system/node_exporter.service
[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/data/prometheus-exporter/node_exporter/node_exporter
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=node_exporter
[Install]
WantedBy=default.target

EOF

执行命令启动node_exporter服务

systemctl daemon-reload
systemctl restart node_exporter
systemctl status node_exporter

与整合Prometheus

进入Prometheus安装目录，修改prometheus.yaml文件

更新Prometheus数据

# Prometheus 热更新命令
curl -vvv  -X POST localhost:9090/-/reload

导入Grafana模版

https://grafana.com/grafana/dashboards/1860-node-exporter-full/

4.2 blackbox_exporter 接口存活监控

下载解压

Github：https://github.com/prometheus/node_exporter/releases

启动blackbox_exporter服务

cat <<EOF> /etc/systemd/system/blackbox_exporter.service
[Unit]
Description=blackbox_exporter Exporter
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/data/prometheus-exporter/blackbox_exporter/blackbox_exporter --config.file=/data/prometheus-exporter/blackbox_exporter/blackbox.yml
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=blackbox_exporter
[Install]
WantedBy=default.target
EOF

执行以下命令启动blackbox_exporter服务

systemctl daemon-reload
systemctl restart blackbox_exporter
systemctl status blackbox_exporter

启动以后可以使用http://192.168.153.128:9115/probe?target=https://www.baidu.com&module=http_2xx&debug=true进行验证（注意换成你的IP访问）

与Prometheus整合

编辑prometheus.yaml文件:

  - job_name: 'blackbox-http'
    metrics_path: /probe
    params:
      module: [http_2xx]  # Look for a HTTP 200 response.
    static_configs:
      - targets:
        - http://prometheus.io
        - https://www.baidu.com
        - http://www.google.com
        - http://192.168.153.129:31064/hello
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 127.0.0.1:9115  

更新Prometheus数据

curl -vvv  -X POST localhost:9090/-/reload

导入Grafana模版

Grafana 模版：https://grafana.com/grafana/dashboards/13659

4.3 mysqld_exporter 接口存活监控

下载安装

https://github.com/prometheus/mysqld_exporter/releases/tag/v0.14.0

tar -zxvf mysqld_exporter-0.14.0.linux-arm64.tar.gz
mv mysqld_exporter-0.14.0.linux-arm64 mysqld_exporter

创建mysql新用户

# 登陆mysql
 mysql -uroot -p123456
# 创建exporter用户
CREATE USER 'exporter'@'%' IDENTIFIED BY '123456' ;
GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'exporter'@'%';
FLUSH PRIVILEGES;

启动mysqld_exporter服务

修改 Environment 和 ExecStart 值，然后直接粘贴回车

cat <<EOF> /etc/systemd/system/mysqld_exporter.service
[Unit]
Description=mysqld_exporter Exporter
Wants=network-online.target
After=network-online.target

[Service]
Environment=DATA_SOURCE_NAME=exporter:123456@(192.168.153.128:33060)/
ExecStart=/data/prometheus-exporter/mysqld_exporter/mysqld_exporter
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=mysqld_exporter
[Install]
WantedBy=default.target
EOF

执行命令启动mysqld_exporter服务

systemctl daemon-reload
systemctl restart mysqld_exporter

启动服务后可以访问http://192.168.153.128:9104/进行验证：

与Prometheus整合

修改prometheus.yml文件

  - job_name: mysqld_exporter
    honor_timestamps: true
    scrape_interval: 8s
    scrape_timeout: 8s
    metrics_path: /metrics
    scheme: http
    follow_redirects: true
    static_configs:
    - targets:
      - 192.168.153.128:9104

更新Prometheus数据

curl -vvv  -X POST localhost:9090/-/reload

导入Grafana模版

https://grafana.com/grafana/dashboards/11323

导入模版以后需要修改host变量

修改 Hide 为Label
在Query中 删除大括号里面的变量
保存退出即可