Use prometheus to monitor centos7 host

     How to monitor server hardware and software, an excellent monitoring software is necessary, prometheus is such a monitoring software.
    It supports monitoring of a large number of databases and application servers (via ... exporter, which is essentially an agent), and of course prometheus also supports monitoring of nodes, including cpu/mem/disk/network usage.


// ------------------------------------------------ --------------------------------
References
https://www.digitalocean.com/community/tutorials/how- to-use-prometheus-to-monitor-your-ubuntu-14-04-server


// ----------------------------- -------------------------------------------------- -Download
address
https://github.com/prometheus/prometheus/releases/download/0.15.1/prometheus-0.15.1.linux-amd64.tar.gz
https://github.com/prometheus/node_exporter/releases/ download/0.11.0/node_exporter-0.11.0.linux-amd64.tar.gz


Download, extract and install node_exporter, note the version number
cd /opt/linuxsir
mkdir node_exporter
cd node_exporter
tar -xzvf ../node_exporter-0.11.0.linux-amd64.tar.gz


Download, extract and install prometheus, pay attention to the version number
cd /opt/linuxsir
mkdir prometheus
cd prometheus
tar -xzvf ../prometheus-0.15. 1. linux-amd64.tar.gz


configuration
file Create a configuration file
prometheus.yml under /opt/linuxsir/prometheus
as
scrape_configs:


 - job_name: "node"


   scrape_interval: "5s"


   target_groups:


   - targets: ['192.168.31.119 :9100']
 
Please refer to
https://www.digitalocean.com/community/tutorials/how-to-use-prometheus-to-monitor-your-ubuntu-14-04-server


// ------------------------------------------------ --------------------------------
Start node_exporter
cd /opt/linuxsir
cd node_exporter
./node_exporter &
stop node_exporter with the following commands
netstat -ntlp|grep 9100
displays the process number
kill -9 process number


Start prometheus
cd /opt/linuxsir
cd prometheus
./prometheus --config.file=prometheus.yml &
stop prometheus use the following command
netstat -ntlp|grep 9090
display the process number
kill -9 process ID




access prometheus
http://192.168.31.119:9090/


access node information, including cpu/mem/disk/network usage
http://192.168.31.119:9090/consoles/node.html


// ------------------------------------------------ --------------------------------
Reference query
can be found on the http://192.168.31.119:9090/ interface of prometheus , enter the query to display the data


cpu monitored by prometheus
sum(rate(node_cpu{job='node',mode='user'}[5m])) * 100 / count(count by (cpu)(node_cpu{job=' node'}))
sum(rate(node_cpu{job='node',mode='system'}[5m])) * 100 / count(count by (cpu)(node_cpu{job='node'}))
above Add two items


mem
node_memory_MemTotal{job='node'}
node_memory_MemFree{job='node'}
node_memory_MemFree{job='node'}/node_memory_MemTotal{job='node'}


disk
rate(node_disk_sectors_read{job='node', device ='sda'  }[5m]) * 512
rate(node_disk_sectors_written{job='node', device='sda' }[5m]) * 512
Add the above two


network
rate(node_network_receive_bytes{job='node', device!='lo'}[5m])
rate(node_network_transmit_bytes{job='node', device!='lo'}[5m])
// -------------------------------------------------- ------------------------------
Other references


understanding-machine-cpu-usage
https://www.robustperception.io/understanding -machine-cpu-usage/


go and export to csv
https://github.com/ryotarai/prometheus-query
https://github.com/ryotarai/prometheus-query


python and export to csv
https://www.robustperception .io/prometheus-query-results-as-csv/
https://www.robustperception.io/prometheus-query-results-as-csv/