Prometheus Beginners

1, Prometheus About
Prometheus is the beginning of the SoundCloudcompany's open-source a 监控software since 2012年成立since many companies and organizations are usingPrometheus

GitHub official address is: https://github.com/prometheus/prometheus
official address: https://prometheus.io/

2, Prometheus features

• A multidimensional data model, which includes the name of the metrics and by key/valuethe identification of时间序列数据
• PromQLA flexible 查询语言, you can use this dimension
• Not rely on distributed storage, a single server node can work directly
• Acquisition time series of HTTP pull manner
• Push 时间序列through PushGatewaycomponent supports
• Through 服务发现or 静态配置find the target
• And dashboard support a variety of graphics mode (grafana)

Package

• Prometheus Server, mainly for data capture and time-series data storage and also provides query and Alert Rule Configuration Management.
• Alertmanager, it is mainly responsible for implementing alarm.
• Push Gateway, for bulk, summing node short-term monitoring data to achieve various metrics data is received by the Client push over, at a specified time interval, accessed by the main program crawl.
• * _Exporter, a variety of reporting data.

Infrastructure

From this schematic diagram it can be seen that the main module comprises Prometheus, Server, Exporters, Pushgateway, PromQL, Alertmanager, WebUI like.

It is generally used logic:

1. Prometheus server periodically from the static configuration targetsor 服务发现a targetspull data.
2. 当新拉取的数据大于配置内存缓存区的时候，Prometheus会将数据持久化到磁盘（如果使用Remote Storage将持久化到云端）。
3. Prometheus可以配置rules，然后定时查询数据，当条件触发的时候，会将alert 推送到配置的Alertmanager。
4. Alertmanager收到警告的时候，可以根据配置，聚合，去重，降噪，最后发送警告。
5. 可以使用API， Prometheus Console 或者Grafana查询和聚合数据。

Prometheus vs Zabbix

• Zabbix 使用的是 C 和 PHP, Prometheus 使用 Golang, 整体而言 Prometheus 运行速度更快一点
• Zabbix 属于传统主机监控，主要用于物理主机，交换机，网络等 监控，Prometheus 不仅适用主机监控，还适用于 Cloud, SaaS, Openstack，Container 监控
• Zabbix 在传统主机监控方面，有更丰富的插件
• Zabbix 可以在 WebGui 中配置很多事情，但是 Prometheus 需要手动修改文件配置

3、Prometheus部署
1.下载安装包prometheus-2.4.0.linux-amd64.tar.gz
github下载地址：https://github.com/prometheus/prometheus/releases/tag/v2.4.0

2.解压

$ tar xf prometheus-2.14.0.linux-amd64.tar.gz -C /usr/local/
$ ln -s /usr/local/prometheus-2.14.0.linux-amd64/ /usr/local/prometheus

创建system

$ vim /etc/systemd/system/prometheus.service 
[root@linux-node1 ~]# vim /etc/systemd/system/prometheus.service 
[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=root
Group=root
Type=simple
ExecStart=/usr/local/prometheus/prometheus \
--config.file=/usr/local/prometheus/prometheus.yml

[Install]
WantedBy=multi-user.target

3.配置prometheus.yml (保持默认配置)

$ vim /usr/local/prometheus/prometheus.yml 
# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090']

4.启动

$ systemctl daemon-reload
$ systemctl start prometheus.service  
$ systemctl enable prometheus.service  

// Prometheus Server端启动默认端口 - 9090
$ netstat -lntup | grep "9090"
tcp6       0      0 :::9090                 :::*                    LISTEN      3393/./prometheus   

5.访问http://ip:9090/Prometheus自带的监控界面

4、Grafana部署
Grafana是一款采用go语言编写的开源应用，主要用于大规模指标数据的可视化展现，是网络架构和应用分析中最流行的时序数据展示工具，目前已经支持绝大部分常用的时序数据库。

Grafana支持许多不同的数据源。每个数据源都有一个特定的查询编辑器,该编辑器定制的特性和功能是公开的特定数据来源。 官方支持以下数据源：Graphite，Elasticsearch，InfluxDB，Prometheus，Cloudwatch，MySQL和OpenTSDB等。

Grafana文档：https://grafana.com/docs/grafana/latest/installation/rpm/

1.安装Grafana

这里使用rpm包安装

$ wget https://mirrors.tuna.tsinghua.edu.cn/grafana/yum/rpm/grafana-6.3.7-1.x86_64.rpm
$ rpm -i --nodeps grafana-6.3.7-1.x86_64.rpm

2.启动Grafana，并加入开机自启

$ systemctl start grafana-server.service 
$ systemctl enable grafana-server.service  

// Grafana 启动默认端口 - 3000
$ netstat -lntup | grep "3000"
tcp6       0      0 :::3000                 :::*                    LISTEN      3585/grafana-server 

3.访问grafana
浏览器访问IP:3000端口，即可打开grafana页面，默认用户名密码都是admin，初次登录会要求修改默认的登录密码

4.添加数据源
（1）点击主界面的“Add data source”

（2）选择Prometheus

（3）Dashboards页面选择Prometheus 2.0 Stats

（4）Settings页面填写Prometheus地址并保存

（5）切换到我们刚才添加的Prometheus 2.0 Stats即可看到整个监控页面

那么现在Prometheus是没有任何监控数据的，

5、Node-Exporter部署

需要监控服务器CPU、内存、磁盘、I/O等信息，首先需要安装node_exporter。
node_exporter的作用是用于机器系统数据收集。

Download:
https://github.com/prometheus/node_exporter/releases/
https://prometheus.io/download/

1. Install node_exporter

$ tar xf node_exporter-0.18.1.linux-amd64.tar.gz -C /usr/local/
$ ln -s  /usr/local/node_exporter-0.18.1.linux-amd64/ /usr/local/node_exporter

Create a system service

$ vim /etc/systemd/system/node_exporter.service
[Unit]
Description=node_exporter
After=network.target

[Service]
Type=simple
User=root
ExecStart=/usr/local/node_exporter/node_exporter
Restart=on-failure

[Install]
WantedBy=multi-user.target

2. Start node-exporter

$ systemctl daemon-reload
$ systemctl start node_exporter.service 
$ systemctl enable node_exporter.service

// Node Exporter默认端口 - 9100
$ netstat -lntup | grep "9100"          
tcp6       0      0 :::9100                 :::*                    LISTEN      5122/node_exporter  

3.Node Exporter default fetch addresshttp://IP:9100/metrics

$ curl  http://127.0.0.1:9100/metrics

4.Prometheus profile adds被监控机器

// 默认node-exporter端口为9100
$ vim /usr/local/prometheus/prometheus.yml
  - job_name: 'prometheus'
    static_configs:
    - targets: ['localhost:9100']

  - job_name: 'node_exporter_centos'
    scrape_interval: 5s
    static_configs:
      - targets: ['10.0.0.171:9100']

prometheus.yml in total define the two monitoring: Monitoring prometheus is a self-service, and the other is to monitor Linux servers

5. Restart prometheusService

$ systemctl restart prometheus

6. GrafanaInstallation Node Exporterof dashboards
Download: https://grafana.com/grafana/dashboards/11074/revisions

(1) the end of the downloaded files Grafana .json import, select the corresponding data sources and

(2) through Node Exporterthe dashboards you can see a lot of charts