Prometheus Alarm convergence

Prometheus Alarm convergence

　　The biggest problem faced by the alarm, alarm is too much, the equivalent of wolf form. Recipients can easily numb, no longer continue to ignore. The key warning is often flooded. In a question, alertmanger been solved to a certain extent.
　　Prometheus successfully put a warning sent to Altermanager, but not simple Altermanager sent directly, which would lead to excessive alarms, major alarms are submerged. So alarms need to be doing reasonable convergence.

 

Alarm convergence means:

• Packet (group): The similar properties are classified as a single alert notification
  •        1, reducing the skilled warning message
  • 　　2, the same alarm polymerization operation and maintenance help troubleshoot

• Inhibition (Inhibition): When the alarm is issued, thereby stopping repeated transmission caused by other alarm alerts
• 

  Profile: Vim alertmanager.yml
    # inhibit_rules numerals: Reduce the warning convergence, reduction alarm, send alarm key 
  inhibit_rules:
      # source_match: matching the current alarms suppressed by the occurrence of other alarm 
     - source_match:
          # Severity: Specifies the alarm level 
         Severity: ' Critical ' 
       # target_match: suppress alarms 
       target_match:
          # Severity: Specifies the level of alarm suppression 
         Severity: ' warning ' 
       # equal is: only contains the specified label before the establishment of the rule 
       equal is: [ ' AlertName ' , ' dev ' , ' instance ' ]

  Inhibition profile
  •        1, eliminating the honor alarm
• Silent (Silences): is a simple mechanism mute reminder of a particular time
  • 　　1, is expected to resolve the alarm

 

---

Creating silence

• Access address: http: //www.xiangsikai.com: 9093 / # / alerts

1. Create a silent

2, fill in the start and end time, fill in the label name, save

3, create a silent View