Postgresql database monitoring experiment 07-PostgreSQL installation and monitoring
surroundings
Machine: 10.176.140.72 plat-ecloud01-mgmt-monitor04 monitor04 of
mobile cloud
Operating system:
CentOS Linux release 7.3.1611 (Core)
InfluxDB version:
influxdb-1.7.9
telegraf version:
telegraf-1.12.6
grafana 版本 ：
grafana-4.3.1
postgresql version:
postgresql12-12.1
Install postgresql
Reference: Official Website Document
Download the latest repo:
[root@plat-ecloud01-mgmt-monitor04 ~]# yum install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm -y
Install the client, server, additional modules, expansion packages:
[root@plat-ecloud01-mgmt-monitor04 ~]# yum install postgresql12 postgresql12-server postgresql12-contrib postgresql12-devel -y
Configure postgresql
1. Create a directory
Create directories to store data files and archive logs:
[root@plat-ecloud01-mgmt-monitor04 ~]# mkdir -p /opt/data/pgdata
[root@plat-ecloud01-mgmt-monitor04 ~]# mkdir -p /opt/data/archlog
[root@plat-ecloud01-mgmt-monitor04 ~]# chown -R postgres:postgres /opt/data
2. Configure environment variables
Modify postgres user environment variables:
[root@plat-ecloud01-mgmt-monitor04 ~]# su - postgres
-bash-4.2$ vim ./.bash_profile
[ -f /etc/profile ] && source /etc/profile
export PGPORT=9999
export PGUSER= postgres
export PGDATA=/opt/data/pgdata
export LANG=en_US.utf8
export PGHOME=/usr/pgsql-12/bin
export LD_LIBRARY_PATH=$PGHOME/lib:/lib64:/usr/lib64:/usr/local/lib64:/lib:/usr/lib:/usr/local/lib
export DATE=`date +"%Y%m%d%H%M"`
export PATH=$PGHOME/bin:$PATH:.
export MANPATH=$PGHOME/share/man:$MANPATH
alias rm='rm -i'
alias ll='ls -lh'
# If you want to customize your settings,
# Use the file below. This is not overridden
# by the RPMS.
[ -f /var/lib/pgsql/.pgsql_profile ] && source /var/lib/pgsql/.pgsql_profile
Make the environment variable changes effective:
-bash-4.2$ source ./.bash_profile
3. Initialize the database
Use the postgres user to execute the initialization command, specify the character set as UTF8, and specify the data directory as / opt / data / pgdata /
-bash-4.2$ /usr/pgsql-12/bin/initdb -E UTF8 --locale=C -D /opt/data/pgdata/
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /opt/data/pgdata ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Asia/Shanghai
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/usr/pgsql-12/bin/pg_ctl -D /opt/data/pgdata/ -l logfile start
4. Modify the database configuration
Modify the config file:
-bash-4.2$ vim /opt/data/pgdata/postgresql.conf
data_directory = '/opt/data/pgdata'
hba_file = '/opt/data/pgdata/pg_hba.conf'
ident_file = '/opt/data/pgdata/pg_ident.conf'
listen_addresses = '*'
port = 9999
max_connections = 1000
superuser_reserved_connections = 5
shared_buffers = 3GB
wal_level = replica
fsync = on
synchronous_commit = on
archive_mode = on
archive_command = 'test ! -f /opt/data/archlog//backup_in_progress && (test ! -f /opt/data/archlog/%f && cp %p /opt/data/archlog/%f)'
'test ! -f /opt/data/archlog//backup_in_progress && (test ! -f /opt/data/archlog/%f && cp %p /opt/data/archlog/%f)'
max_wal_senders = 10
wal_keep_segments = 32
hot_standby = on
hot_standby_feedback = on
Modify the hba file:
-bash-4.2$ vim /opt/data/pgdata/pg_hba.conf
host all all 0.0.0.0/0 md5
host replication all 0.0.0.0/0 trust
5. Start the database
Use the postgres user to start the database
-bash-4.2$ /usr/pgsql-12/bin/pg_ctl start -D /opt/data/pgdata/
waiting for server to start....2019-12-13 10:50:39.277 CST [12881] LOG: starting PostgreSQL 12.1 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
2019-12-13 10:50:39.278 CST [12881] LOG: listening on IPv4 address "0.0.0.0", port 9999
2019-12-13 10:50:39.278 CST [12881] LOG: listening on IPv6 address "::", port 9999
2019-12-13 10:50:39.281 CST [12881] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.9999"
2019-12-13 10:50:39.287 CST [12881] LOG: listening on Unix socket "/tmp/.s.PGSQL.9999"
2019-12-13 10:50:39.464 CST [12881] LOG: redirecting log output to logging collector process
2019-12-13 10:50:39.464 CST [12881] HINT: Future log output will appear in directory "log".
done
server started
6. Configure password
Use the postgres user to enter the database and change the postgres password to postgres:
-bash-4.2$ psql -p 9999
psql (12.1)
Type "help" for help.
postgres=# alter user postgres with password 'postgres';
ALTER ROLE
Monitoring configuration
1. Create test data
Build library test
postgres=# create database test ;
CREATE DATABASE
Create a table and insert data:
postgres=# \c test
You are now connected to database "test" as user "postgres".
test=# select i,'t:'||i as t into t from generate_series(1,10000000) as i;
SELECT 10000000
View table size:
test=# select pg_size_pretty(pg_relation_size('t'));
pg_size_pretty
----------------
422 MB
(1 row)
View library size:
test=# select schemaname,round(sum(pg_total_relation_size(schemaname||'.'||tablename))/1024/1024) "Mb" from pg_tables group by 1;
schemaname | Mb
--------------------+-----
public | 422
pg_catalog | 8
information_schema | 0
2. Configure telegraf
Reference document: Official document
Need to add database related configuration in telegraf:
[root@plat-ecloud01-mgmt-monitor04 ~]# vim /etc/telegraf/telegraf.conf
# Read metrics from one or many postgresql servers
[[inputs.postgresql]]
## specify address via a url matching:
## postgres://[pqgotest[:password]]@localhost[/dbname]\
## ?sslmode=[disable|verify-ca|verify-full]
## or a simple string:
## host=localhost user=pqotest password=... sslmode=... dbname=app_production
##
## All connection parameters are optional.
##
## Without the dbname parameter, the driver will default to a database
## with the same name as the user. This dbname is just for instantiating a
## connection with the server and doesn't restrict the databases we are trying
## to grab metrics for.
##
# address = "host=localhost user=postgres sslmode=disable"
address = "postgres://postgres:postgres@localhost:9999" #数据库地址，格式参见前面介绍
## A custom name for the database that will be used as the "server" tag in the
## measurement output. If not specified, a default one generated from
## the connection address is used.
# outputaddress = "db01"
## connection configuration.
## maxlifetime - specify the maximum lifetime of a connection.
## default is forever (0s)
max_lifetime = "0s"
## A list of databases to explicitly ignore. If not specified, metrics for all
## databases are gathered. Do NOT use with the 'databases' option.
# ignored_databases = ["postgres", "template0", "template1"]
ignored_databases = ["postgres", "template0", "template1"] #忽略括号内数据库
## A list of databases to pull metrics about. If not specified, metrics for all
## databases are gathered. Do NOT use with the 'ignored_databases' option.
# databases = ["app_production", "testing"]
# databases = ["test"] #只监控括号内数据库不能与ignored_databases同用
Restart telegraf:
[root@plat-ecloud01-mgmt-monitor04 ~]# systemctl restart telegraf
3. Verification
Check the telegraf library of influxdb and find that a new measurement named postgresql has been created:
[root@plat-ecloud01-mgmt-monitor04 ~]# influx -precision rfc3339 -database telegraf -execute 'select * from postgresql limit 3'
name: postgresql
time blk_read_time blk_write_time blks_hit blks_read buffers_alloc buffers_backend buffers_backend_fsync buffers_checkpoint buffers_clean checkpoint_sync_time checkpoint_write_time checkpoints_req checkpoints_timed conflicts datid datname db deadlocks host maxwritten_clean numbackends server temp_bytes temp_files tup_deleted tup_fetched tup_inserted tup_returned tup_updated xact_commit xact_rollback
---- ------------- -------------- -------- --------- ------------- --------------- --------------------- ------------------ ------------- -------------------- --------------------- --------------- ----------------- --------- ----- ------- -- --------- ---- ---------------- ----------- ------ ---------- ---------- ----------- ----------- ------------ ------------ ----------- ----------- -------------
2019-12-13T06:46:20Z 0 0 20586 254 3241 135511 0 535 0 12 149643 3 45 0 14185 postgres postgres 0 plat-ecloud01-mgmt-monitor04 0 0 dbname=test host=localhost port=9999 user=postgres 0 0 0 4182 0 290465 0 478 0
2019-12-13T06:46:20Z 0 0 25572 83830 0 16384 test test 0 plat-ecloud01-mgmt-monitor04 1 dbname=test host=localhost port=9999 user=postgres 141400000 2 68 5629 10100113 287196 3 471 1
2019-12-13T06:46:30Z 0 0 20663 254 3333 135511 0 535 0 12 149643 3 45 0 14185 postgres postgres 0 plat-ecloud01-mgmt-monitor04 0 0 dbname=test host=localhost port=9999 user=postgres 0 0 0 4193 0 291704 0 480 0
Confirm that the data has been fetched normally and saved like influxdb
4.Place grafana
Search the available templates directly on the official website: official template address
I use the Postgres Overview-more info template here, the id is 7626. For the
import method, see: postgresql database monitoring experiment 05-Grafana installation
After importing grafana shows as follows:
