使用prometheus监控spring cloud

一起养成写作习惯！这是我参与「掘金日新计划 · 4 月更文挑战」的第6天，点击查看活动详情。

prometheus

Prometheus 是一个云原生计算基金会项目，是一个系统和服务监控系统。它以给定的时间间隔从配置的目标收集指标，评估规则表达式，显示结果，并在观察到指定条件时触发警报。

Prometheus 与其他指标和监控系统的区别在于：

多维数据模型（由指标名称和键/值维度集定义的时间序列）
PromQL，一种强大且灵活的查询语言，可利用此维度
不依赖分布式存储；单个服务器节点是自治的
用于时间序列收集的 HTTP拉取模型
****通过用于批处理作业的中间网关支持推送时间序列
通过服务发现或静态配置发现目标
图形和仪表板支持的多种模式
支持分层和水平联合

架构概述

安装

tar xvfz prometheus-*.tar.gz
cd prometheus-*
复制代码

在启动 Prometheus 之前，让我们对其进行配置。

prometheus.yml

# my global config
global:
  scrape_interval:     10s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 10s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
       - 127.0.0.1:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  - "rules/*.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # 配置job，可以同时配置多个服务
  - job_name: 'name1'
    metrics_path: '/actuator/prometheus'
    scrape_interval: 5s
    scheme: "https"
    static_configs:
      - targets: ['xxx.com']

  - job_name: 'name2'
    metrics_path: '/actuator/prometheus'
    scrape_interval: 5s
    scheme: "http"
    static_configs:
      - targets: ['abc.com']
复制代码

监控报警规则

rules/xx_rules.yml

groups:
- name: nqi-down
  rules:
  - alert: gx-node-down
    expr: up{instance="xxx.com:443"} == 0
    for: 10s
    labels:
      status: High
      team: xxx
    annotations:
      description: "xxx is Down ! ! !"
      summary:  "xxx服务停了,请留意!!!"
      
  - alert: test-node-down
    expr: up{instance="abc.com:80"} == 0
    for: 5s
    labels:
      status: Warn
      team: test
    annotations:
      description: "abc is Down ! ! !"
      summary:  "abc服务停了,请留意!!!"
复制代码

安装alertmanager并解压

下载alertmanager

tar xvfz alertmanager-*.tar.gz
cd alertmanager-*
复制代码

配置

alertmanager.yml

global: 
  resolve_timeout: 5m #解析的超时时间
  smtp_smarthost: 'smtp.xxx.com:465' #邮箱smtp地址
  smtp_from: '[email protected]' #来自哪个邮箱发出的
  smtp_auth_username: '[email protected]' #邮箱的用户名
  smtp_auth_password: 'W7CKmqD2x0iGXM9R' #这里是邮箱的授权密码，不是登录密码
  smtp_require_tls: false #是否启用tls

templates:
  - ./email.html

route:
  group_by: ['abc']
  group_wait: 3s
  group_interval: 10s
  repeat_interval: 3h
  receiver: 'mail'
receivers:
- name: 'mail'
  email_configs: #email的配置
  - to: '[email protected], [email protected]' #报警接收人的邮件地址
    send_resolved: true  #发送恢复通知
    html: '{{ template "email.html" . }}'
inhibit_rules:
  - source_match:
      severity: 'critical'
    target_match:
      severity: 'warning'
    equal: ['alertname', 'dev', 'instance']
复制代码

代码

增加pom依赖

<!-- 监控 -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
复制代码

application.properties增加配置

#监控
management.endpoints.web.exposure.include=prometheus
复制代码

安装Grafana

wget <https://dl.grafana.com/enterprise/release/grafana-enterprise-8.4.6.linux-amd64.tar.gz>\
tar -zxvf grafana-enterprise-8.4.6.linux-amd64.tar.gz
复制代码