Prometheus monitoring flink

Summary

In this paper, will be sent metric flink task to run Prometheus

Significance of monitoring

flink flow task has certain requirements in terms of stability of real-time, metric flink cluster by Prometheus collection, specify a number of indicators can be monitored alarms. Thus allowing developers to quickly react promptly deal with the problem online.

1.Prometheus Profile

Prometheus is an open source monitoring and alarm systems.https://prometheus.io/docs/introduction/overview/

2.1 Features

• Multi-dimensional data model (by name and label key indicators of identity)
• Flexible query language
• Stand-alone mode of operation does not depend on distributed storage
• Monitoring data collected by a pull mode (HTTP)
• It can support the push to prometheus monitoring data through the use of middleware
• Static configuration or through service discovery to find the target (monitoring data source)
• It supports multi-mode drawing and dashboards

2.2 Components

Prometheus ecosystem includes many components (most are optional)

• Prometheus server (fetch, store time-series data)

• client libraries (help application support prometheus data acquisition)

• a push gateway (Jobs supporting short life cycle, the monitoring data received push) (Prometheus native support pull mode, push mode of operation for compatibility)
• exporters (to support open source monitoring data collection services, such as: HAProxy, StatsD, Graphite, etc.) (ie, agent)

• alertmanager (process alarm)

2.3 Architecture

The picture below shows the prometheus prometheus construction and system components may be required to:

image.png

3 flink integrated prometheus

3.1 flink Configuration

Detailed Configuration Reference
https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#cpu
enter flink directory

image.png

Copy flink-metrics-prometheus-1.7.2.jar opt directory under the directory to lib.

Edit conf / flink-conf.yml

metrics.reporter.promgateway.class: org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporter
metrics.reporter.promgateway.host: test01.cdh6.local
metrics.reporter.promgateway.port: 9091
metrics.reporter.promgateway.jobName: myJob
metrics.reporter.promgateway.randomJobNameSuffix: true
metrics.reporter.promgateway.deleteOnShutdown: false

3.2 pushgateway installation

Reference https://github.com/prometheus/pushgateway
accesshttp://test01.cdh6.local:9091/#

image.png

3.3 prometheus installation

Reference https://www.jianshu.com/p/3a9ede07d963
this embodiment prometheus and installed on the same machine pushgateway
prepared prometheus.yml

scrape_configs:
  - job_name: 'pushgateway'
    static_configs:
      - targets: ['localhost:9091']
        labels:
          instance: 'pushgateway'

Start
./prometheus --config.file=prometheus.ym l
the port 9091 is flink-conf.yml corresponding metrics.reporter.promgateway.port: 9091
Flink to put some metric push the port 9091, and then prometheus acquisition.

4 results

Start flink cluster .bin/start-cluster.sh
accesshttp://test01.cdh6.local:9090

image.png

to sum up

flink metric data stream flow is flink metric -> pushgateway -> prometheus

todo configure alarm policy

Reproduced in: https: //www.jianshu.com/p/fc9af91e6b07