Prometheus service discovery based on Consul

Previously, I added the monitored terminal manually. After adding the monitored terminal in the Prometheus configuration file, I hot-loaded Prometheus and the target terminal can be found in the Prometheus graphical interface.

If the number of monitored terminals is large, service discovery can be used, that is, the targets expected to be monitored are automatically added without manual intervention. For example, zabbix has network segment-based scanning.

Prometheus service discovery

---

Prometheus supports two ways to add the monitored end:

• Static configuration: manual configuration

• Service discovery: Dynamically discover Target instances that need to be monitored

Sources that support service discovery:

• azure_sd_configs

• consul_sd_configs

• dns_sd_configs

• ec2_sd_configs

• openstack_sd_configs

• file_sd_configs

• gce_sd_configs

• kubernetes_sd_configs

• marathon_sd_configs

• nerve_sd_configs

• serverset_sd_configs

• triton_sd_configs

Service discovery based on Consul

---

Docker deploys Consul:

docker run --name consul -d -p 8500:8500 consul

 Consul mainly uses service discovery in Prometheus. In this service discovery, there is a provider and a consumer. The consumer can obtain the provider's registration information from this consul.

Consul can be used on any machine, as long as Prometheus can connect to consul

Here the collector acts as a provider, and Prometheus acts as a consumer. The provider registers with consul and registers the address of the exposed monitoring indicators Ip+Port+Metrics.

Prometheus obtains the registration information from consul, obtains the monitored terminal, automatically joins the monitoring, and finally uses grafana to provide access.

Here we mainly use the service discovery function. There is only one service discovery, only consul itself.

Now the exporter component started by the monitored end will automatically add its own information (ip:port/metrics), because Prometheus wants to monitor the monitored end and only cares about (ip:port/metrics)

Assemble test information into consul and want to call its api

Register the service with Consul:

curl -X PUT -d '{"id": "Linux-1","name": "Linux","address": "192.168.179.100","port": 9100,"tags": ["service"],"checks": [{"http": "http://192.168.179.100:9100","interval": "5s"}]}' http://192.168.179.100:8500/v1/agent/service/register

• id is similar to instance
• name is equivalent to job
• "tags": ["service"] registered tags
• Consul supports health checks, which means checking whether the target is accessible at intervals. 
• http://192.168.179.100:8500/v1/agent/service/register   interface address

[root@www ~]# curl -X PUT -d '{"id": "Linux-1","name": "Linux","address": "192.168.179.100","port":

> 9100,"tags": ["service"],"checks": [{"http": "http://192.168.179.100:9100","interval":

> "5s"}]}' http://192.168.179.100:8500/v1/agent/service/register

Now we need to enable consul service discovery in Prometheus

[root@localhost ~]# vim /usr/local/prometheus/prometheus.yml 
  - job_name: 'consul_sd'
    basic_auth:
      username: prometheus
      password: 123456
    consul_sd_configs:
    - server: 192.168.179.100:8500
      services: ['linux']   #配置服务发现里面consul里面的哪个组

Now as long as you register the information in consul, Prometheus will automatically include it in the monitoring terminal. There is no need to configure it manually. You only need to register the information in consul.

curl -X PUT -d '{"id": "Docker-1","name": "Docker","address": "192.168.179.100","port": 
8080,"tags": ["service"],"checks": [{"http": "http://192.168.179.100:8080","interval": 
"5s"}]}' http://192.168.179.100:8500/v1/agent/service/register


[root@localhost ~]# vim /usr/local/prometheus/prometheus.yml 
  - job_name: 'consul_sd'
    basic_auth:
      username: prometheus
      password: 123456
    consul_sd_configs:
    - server: 192.168.179.100:8500
      services: ['linux','docker']

 

 

 

Consul deletes invalid service nodes 

---

1. As shown in the figure, the directly registered service has been abandoned, but it is still displayed. 

2. Delete invalid service nodes

 Use HTTP request, PUT method, the example is as follows:

http://192.168.80.19:8500/v1/agent/service/deregister/location-server-7101

 If the environment port is not open to the outside world, it can be executed directly on the server. 

curl -X PUT http://127.0.0.1:8500/v1/agent/service/deregister/location-server-7101