1. Introduction to Prometheus
Prometheus is an open source system monitoring alarm box, which has the following advantages compared to Nagios or Zabbix
1.1 Ease of management
Prometheus: The core part of Prometheus has only a single binary file, which can work directly locally without relying on distributed storage.
1.2 Business data relevance
Prometheus: Monitor the running status of the service. Based on Prometheus's rich Client library, users can easily add Prometheus support to the application, so that users can get the true running status of the service and the application.
1.3 High efficiency:
A single Prometheus can handle millions of monitoring indicators; hundreds of thousands of data points per second.
1.4 Easy to scale:
Prometheus can be expanded by using sharing + federation to form a logical cluster; Prometheus provides client SDKs in multiple languages. These SDKs can quickly allow applications to be monitored by Prometheus.
1.5 Good visualization:
In addition to Prometheus comes with Prometheus UI, Grafana visualization tool also provides complete Proetheus support
2. Prometheus framework diagram
3. Introduction to Prometheus components
3.1 Prometheus Server：
Prometheus Sever is the core part of the Prometheus component, responsible for obtaining, storing and querying monitoring data. Prometheus Server can manage monitoring targets through static configuration, or dynamically manage monitoring targets in conjunction with Service Discovery, and obtain data from these monitoring targets. Secondly, Prometheus Sever needs to store the collected data. Prometheus Server itself is a real-time database, which stores the collected monitoring data in a local disk in a time series. Prometheus Server provides customized PromQL externally to realize data query and analysis. In addition, the federated cluster capability of Prometheus Server allows it to obtain data from other Prometheus Server instances.
3.2 Exporters：
Exporter exposes the endpoint of monitoring data collection to Prometheus Server in the form of HTTP service. Prometheus Server can obtain the monitoring data that needs to be collected by accessing the Endpoint endpoint provided by the Exporter. Exporter can be divided into two categories:
Direct collection: This type of Exporter directly has built-in support for Prometheus monitoring, such as cAdvisor, Kubernetes, Etcd, Gokit, etc., all have directly built-in endpoints for exposing monitoring data to Prometheus.
Indirect collection: The original monitoring target does not directly support Prometheus, so it is necessary to write the monitoring collection program of the monitoring target through the Client Library provided by Prometheus. For example: Mysql Exporter, JMX Exporter, Consul Exporter, etc.
3.3 AlertManager：
Prometheus Server supports creating alarm rules based on Prom QL. If the rules defined by Prom QL are met, an alarm will be generated. After AlertManager receives the alerts from the Prometheus server, it will remove duplicate data, group it, and route it to the receiving method to send an alarm. Common receiving methods are: email, pagerduty, webhook, etc.
3.4 PushGateway:
Prometheus data collection is based on Prometheus Server pulling data from Exporter, so when the network environment does not allow Prometheus Server and Exporter to communicate, you can use PushGateway to transfer. Actively push the monitoring data of the internal network to the Gateway through PushGateway, and Prometheus Server uses the same method for Exporter to pull the monitoring data from PushGateway to Prometheus Server.