Monitoring deployment---Prometheus+grafana (super detailed)

Prometheus+grafana monitoring tool construction steps--details

build steps

1. Prometheus construction steps

官网下载二进制包：https://prometheus.io/download/

1. Download the binary package

wget https://github.com/prometheus/prometheus/releases/download/v2.36.0/prometheus-2.36.0.linux-amd64.tar.gz

2. After downloading, decompress it in the server

tar -xzvf prometheus-2.36.0.linux-amd64.tar.gz

3. After decompression, add prometheus to systemd management:

Some machines modify /usr/lib/systemd/system/ Here...
I modified it in /etc/systemd/system/

Note, "--web.listen-address=:80" in it, I said to change the default port 9090 to port 80, here you need to pay attention, if you don't need to modify it, just change 80 to 9090. (Also pay attention to whether the path of the yml file inside is consistent)

[root@zishan666 ~]# cat /etc/systemd/system/prometheus.service 
[Unit]
Description=Prometheus
Documentation=https://prometheus.io/
After=network.target

[Service]
Type=simple
User=root
ExecStart=/usr/local/prometheus/prometheus --web.listen-address=:80 --config.file=/usr/local/prometheus/prometheus.yml --web.enable-lifecycle --storage.tsdb.path=/usr/local/prometheus/data --storage.tsdb.retention=60d
Restart=on-failure

[Install]
WantedBy=multi-user.target

4. Modify the prometheus configuration file:

[root@zishan666 prometheus]# pwd
/usr/local/prometheus

[root@zishan666 prometheus]# cat prometheus.yml 
# my global config
global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093     # 开启alertmanager告警，去掉 # 号即可

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"    # prometheus读取监控的数据文件
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "prometheus"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["120.26.90.57:80"].   
  - job_name: "nodes"
    static_configs:
    - targets: ["120.26.90.57:9100"]    # 监控 node_exporter 数据，主要监控node节点数据（内存，cpu，负载等）

The above yml file should pay attention to static_configs: the following ip needs to be modified to the corresponding ip, fill in the ip of the client client under nodes, and remember to modify the port.

5. Load configuration and start

systemctl daemon-reload
systemctl start prometheus.service

6. Successful access to the web page

http://120.26.90.57:9090/graph
insert image description here

2. Deployment of node_exporter

1. Download the binaries

下载网址： https://prometheus.io/download/

wget https://github.com/prometheus/node_exporter/releases/download/v1.3.1/node_exporter-1.3.1.linux-amd64.tar.gz

2. Unzip and move to the specified directory

tar xzvf node_exporter-1.3.1.linux-amd64.tar.gz
mv node_exporter-1.3.1.linux-amd64 node_exporter
cp -r /root/test/node_exporter  /usr/local/

3. Add systemd management

[root@zishan666 html]# cat /etc/systemd/system/node_exporter.service 
[Unit]
Description=node_exportier
After=network.target
[Service]
Type=simple
User=root
ExecStart=/usr/local/node_exporter/node_exporter
Restart=on-failure
[Install]
WantedBy=multi-user.target

4. Load configuration and start

systemctl daemon-reload
systemctl start node_exporter

5. Verify that the node is deployed successfully

insert image description here

3. The deployment process of grafana

Official website download URL: https://grafana.com/grafana/download

1. Download the binary package and decompress it

wget https://dl.grafana.com/enterprise/release/grafana-enterprise-8.5.4.linux-amd64.tar.gz

cd /root/test/

tar -zxvf grafana-enterprise-8.5.4.linux-amd64.tar.gz

After decompression, you can actually open grafana, directly enter the bin directory, /root/test/grafana/bin/ and then
execute ./grafana-server or ./grafana-server &, we can also add system quick start:

2. Add systemd management

[root@zishan666 test]# cat /etc/systemd/system/grafana.service 
[Unit]
Description=grafana
[Service]
ExecStart=/root/test/grafana/bin/grafana-server -homepath=/root/test/grafana
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
[Install]
WantedBy=multi-user.target

3. Load the system configuration and start

systemctl daemon-reload
systemctl start grafana.service

4. Grafana is deployed and started successfully

insert image description here

4. Steps to build mysqld_exporter

Before installing mysqld.exporter, you need to ensure that mysql has been installed in your server

1. Download the binary package

Download URL: https://prometheus.io/download/

[root@Zisson ~]# wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.14.0/mysqld_exporter-0.14.0.linux-amd64.tar.gz

2. Unzip to the specified directory

[root@Zisson ~]# tar xzf mysqld_exporter-0.14.0.linux-amd64.tar.gz -C /root/test/

3. Connect to Mysql and open the permissions.

mysqld_exporter needs to connect to Mysql and open Mysql permissions

[root@Zisson ~]# mysql -u root -p密码
mysql> grant all privileges on *.* to 'root'@'%' identified by 'WU@123456a' with grant option;
mysql> GRANT REPLICATION CLIENT,PROCESS ON *.* TO 'mysql_monitor'@'localhost' identified by 'WU@123456a';
mysql> GRANT SELECT ON *.* TO 'mysql_monitor'@'localhost';
mysql> flush privileges;
mysql> \q

4. Create and edit .my.cnf

[root@Zisson ~]# cat /usr/local/mysqld_exporter/.my.cnf
[client]
user=mysql_monitor
password=WU@123456a

You can choose to start directly in the background: nohup ./mysqld_exporter -config.my-cnf=.my.cnf &
#Start mysqld_exporter and run it
in the background. It is recommended to cd to the directory before running, so that nohup logs will exist in the current directory. If you run it in the system root directory, all nohup.out log files will be confused

5. Configure the systemd shortcut startup method and start it

[root@Zisson ~]# cat /etc/systemd/system/mysqld_exporter.service 
[Unit]

Descripton=mysqls_exporter server

Documentation=https://prometheus.io/docs/introduction/overview/

After=network.target

[Service]

Restart=on-failure

ExecStart=/usr/local/mysqld_exporter/mysqld_exporter --config.my-cnf=/usr/local/mysqld_exporter/.my.cnf --web.listen-address=:9104

[Install]
WantedBy=multi-user.target

The default port of mysqld_exporter is: 9104

[root@Zisson ~]# systemctl enable mysqld_exporter
[root@Zisson ~]# systemctl start mysqld_exporter

6. Configure and modify the prometheus.yml file of Prometheus

Add the following:

  - job_name: "mysqld_exporter"
    static_configs:
      - targets: ['120.26.57.23:9104']

7. Restart Prometheus, the build is complete!

insert image description here

Note: If you are in Alibaba Cloud, remember to open port 9104 of the Alibaba Cloud security group

insert image description here