prometheus conjunction with E-mail alert alertmanager

Prometheus is Alertmanager with two components separated from each other. The Prometheus server sends an alert to Alertmanager alarm rules, then the Alertmanager silencing, inhibition, aggregation and other information, and other information sent via email, the micro channel,

Installation and deployment alertmanager

Download Package

wget https://github.com/prometheus/alertmanager/releases/download/v0.20.0/alertmanager-0.20.0.linux-amd64.tar.gz

Extracting installer

tar zxvf alertmanager-0.16.0.linux-amd64.tar.gz && cd alertmanager-0.16.0.linux-amd64

Start alertmanager

./alertmanager --config.file=alertmanager.yml

Prometheus arranged to communicate with alertmanager

Prometheus.yml modify configuration files

Disposed in communication with alertmanager

alerting:
  alertmanagers:
  - static_configs:
    - targets:
       - x.x.x.x:9093

Increase alarm rules, monitoring host service, the default is normally 1

#Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:

• "rules/*.yml"

groups:
- name: generals.rules
  rules:
  - alert: InstanceDown
    expr: up == 0
    for: 1m
    labels:
      severity: error
    annotations:
      summary: "Instance {{ $labels.isinstance }} down"
      description: "{{ $labels.instance }} down more than 1 minutes"

Reload the configuration file kill -hup $ pid or restart the service prometheus

View alarms configured rules

Trigger alarm

Alarm rules that have been created, set up e-mail notification alarm, as has been shown in the column qq mailbox, the mailbox qq need to turn on the SMTP,
smtp_require_tls default is true, there needs to be modified to false, otherwise the message is triggered unsuccessful, smtp_auth_password fill authorize Ma

global: global configuration, including a timeout after the alarm solve, SMTP configuration, various channels API addresses, etc. notification.
route: distribution policy is used to set the alarm, it is a tree structure, depth-first match in the order from left to right.
receivers: configure alarm message receiver information, such as conventional email, wechat, slack, webhook other message notification mode.

global:
  resolve_timeout: 5m
  smtp_smarthost: 'smtp.qq.com:465'
  smtp_from: '[email protected]'
  smtp_auth_username: '[email protected]'
  smtp_auth_password: 'xxx'
  #smtp_auth_secret: false
  smtp_require_tls: false
  smtp_hello: 'qq.com'

route:
  group_by: ['alertname']
  group_wait: 10s
  group_interval: 10s
  repeat_interval: 1h
  receiver: 'mail'
receivers:
- name: 'mail'
  email_configs:
  - to: '[email protected]'
    send_resolved: true

An alarm is triggered, manually shut down, es service, after a few minutes receive an email alert