SpringBoot+Prometheus+Grafana realizes application monitoring and alarm

Click on the blue font above and select "Star Official Account"

High-quality articles, delivered immediately

Follow the official account backstage to reply to pay or mall to obtain actual project information + video

Author: i smoke

cnblogs.com/2YSP/p/12827487.html 

1. Background

There are many application monitoring solutions for SpringBoot, and SpringBoot+Prometheus+Grafana is currently one of the more commonly used solutions. The relationship between the three is as follows:

relation chart

Two, develop SpringBoot application

First, create a SpringBoot project, the pom file is as follows:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>

<dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <optional>true</optional>
</dependency>

<!-- https://mvnrepository.com/artifact/io.prometheus/simpleclient_spring_boot -->
<dependency>
    <groupId>io.prometheus</groupId>
    <artifactId>simpleclient_spring_boot</artifactId>
    <version>0.8.1</version>
</dependency>

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
</dependency>

Note:  The SpringBoot version here is 1.5.7.RELEASE. The reason why the latest 2.X is not used is because the latest simpleclient_spring_boot only supports 1.5.X, not sure if the 2.X version can be supported.

Added annotations to the MonitorDemoApplication startup class

package cn.sp; 
 
import io.prometheus.client.spring.boot.EnablePrometheusEndpoint; 
import io.prometheus.client.spring.boot.EnableSpringBootMetricsCollector; 
import org.springframework.boot.SpringApplication; 
import org.springframework.boot.autoconfigure.SpringBootApplication; 
@EnablePrometheusEndpoint 
@EnableSpringBootMetricsCollector 
@SpringBootApplication 
public class MonitorDemoApplication { 
 
    public static void main(String[] args) { 
        SpringApplication.run(MonitorDemoApplication.class, args); 
    } 
 
} 

Configuration file application.yml

server: 
  port: 8848 
spring: 
  application: 
    name: monitor-demo 
 
security: 
  user: 
    name: admin 
    password: 1234 
  basic: 
    enabled: true 
    # 安全路径列表,逗号分隔,此处只针对/admin路径进行认证 
    path: /admin 
 
# actuator暴露接口的前缀 
management: 
  context-path: /admin 
  # actuator暴露接口使用的端口,为了和api接口使用的端口进行分离 
  port: 8888 
  security: 
    enabled: true 
    roles: SUPERUSER 

Test code TestController

@RequestMapping("/heap/test")
@RestController
public class TestController {
 
    public static final Map<String, Object> map = new ConcurrentHashMap<>();
 
    @RequestMapping("")
    public String testHeapUsed() {
        for (int i = 0; i < 10000000; i++) {
            map.put(i + "", new Object());
        }
        return "ok";
    }
}

The logic here is to create a large number of objects and save them in the map to increase the heap memory usage after requesting this interface, which is convenient for testing email alarms later.

After starting the project, you can see many Endpoints in IDEA, as shown in the figure:

At first, my IDEA did not display this Endpoints, but later I found that the version of the idea I was using was too old, or 2017.1,
and this requires idea2017.2 or higher to be able to see it.

Later, I had to download and install it again, and it took a long time. . . .

After startup, visit http://localhost:8888/admin/prometheus to see the monitoring indicators exposed by the service. note:

Because the security authentication is turned on, you need to prompt to enter the account/password to access this URL. If it prompts 404, please check whether your requested address is correct. If you do not set management.context-path, the default address is http://ip:port /prometheus

Three, install Prometheus

download link

https://prometheus.io/download/

This article downloads the Windows version of prometheus-2.17.2.windows-amd64.tar.gz.

After decompression, modify the prometheus.yml file to configure the target information for data collection.

scrape_configs: 
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config. 
  # - job_name: 'prometheus' 
 
    # metrics_path defaults to '/metrics' 
    # scheme defaults to 'http'. 
 
    # static_configs: 
    # - targets: ['localhost:9090'] 
  - job_name: 'monitor-demo' 
    scrape_interval: 5s # 刮取的时间间隔 
    scrape_timeout: 5s  
    metrics_path: /admin/prometheus 
    scheme: http  
    basic_auth: #认证信息 
      username: admin 
      password: 1234 
    static_configs: 
      - targets: 
        - 127.0.0.1:8888  #此处填写 Spring Boot 应用的 IP + 端口号

For more configuration information, please check the official documentation.

Now you can start Prometheus. Enter the command line: prometheus.exe --config.file=prometheus.yml to
visit http://localhost:9090/targets to check whether the Spring Boot collection status is normal.

Fourth, install Grafana

download link

https://grafana.com/grafana/download

This article uses the Windows version grafana-6.3.3.windows-amd64.zip.

After decompression, run grafana-server.exe in the bin directory to start, and the browser can access http://localhost:3000 to see the login page. The default account password is admin/admin.

Now start to create your own visual monitoring panel.

1. Set up the data source

2. Create a Dashboard

3. Fill in the collected index points

Note:  The indicator points here cannot be filled in casually, they must be existing and can be seen in Prometheus. 4. Select the chart style
5. Fill in the title description

Finally, click Save in the upper right corner and enter the name of Dashboad.

Tips:  The chart layout here can be dragged with the mouse

Five, add email alarm

In an actual project, when a monitored indicator exceeds a threshold (for example, CPU usage is too high), it is hoped that the monitoring system will automatically alert the operation and maintenance personnel through SMS, DingTalk, and email. Grafana supports this function.

Step 1:  Click [Alerting]——>[Notification channels] to add notification channels

Create channel
edit

There are many options for Type here, including webhook, Dingding, etc. Here, we take email as an example.

Step 2:  Email configuration

Grafana runs by default using defaults.ini in the conf directory as the configuration file. According to official recommendations, we should not change defaults.ini but create a new configuration file custom.ini in the same directory.

Take Tencent enterprise mailbox as an example, the configuration is as follows:

#################################### SMTP / Emailing #####################
[smtp]
enabled = true
host = smtp.exmail.qq.com:465
user = [email protected]
# If the password contains # or ; you have to wrap it with triple quotes. Ex """#password;"""
password = XXX
cert_file =
key_file =
skip_verify = true
from_address = [email protected]
from_name = Grafana
ehlo_identity = ininin.com

After that, demand heavy 启 Grafana, instruction grafana-server.exe -config = E: \ file \ grafana-6.3.3 \ conf \ custom.ini

Step 3:  Add alert to the indicator

Configure alert rules
Configure notification method and information

Evaluate every

Indicates the detection rate, here to test the effect, change to 1 second

For

If the alert rule is configured with For and the query violates the configured threshold, it will first change from OK to Pending. No notification will be sent from OK to Pending Grafana. Once the trigger time of the alert rule exceeds the duration, it will change to Alerting and send an alert notification.

Conditions

When represents the time, of represents the condition, is above represents the trigger value
at the same time, there will be a red line after setting is above.

If no data or all values are null

If there is no data or all values ​​are empty, select trigger alarm here

If execution error or timeout

If the execution is wrong or overtime, choose to trigger the alarm here

Note: The  next time it is triggered, such as 10 seconds later, it will not trigger again to prevent alarm storms from occurring!

Step 4: After the  test
request http://localhost:8848/heap/test interface, the memory rises above the set threshold, and then an alarm email is received.

Alarm mail

The picture here is not shown, I don't understand why.

Six, summary

This set of monitoring functions is quite powerful, but Prometheus has a lot of expressions.

Attach a few links:

  • https://prometheus.io/docs/introduction/first_steps/

  • https://grafana.com/docs/grafana/latest/

  • https://github.com/2YSP/monitor-demo

}



有热门推荐????
10个解放双手实用在线工具,有些代码真的不用手写!微信小程序练手实战:前端+后端(Java)
又一神操作,SpringBoot2.x 集成百度 uidgenerator搞定全局ID为什么阿里不允许用Executors创建线程池,而是通过ThreadPoolExecutor的方式?
为什么要在2021年放弃Jenkins?我已经对他失去耐心了...
Docker + FastDFS + Spring Boot 一键式搭建分布式文件服务器

点击阅读原文,前往学习SpringCloud实战项目

Guess you like

Origin blog.csdn.net/qq_17231297/article/details/114697181