postgresql数据库监控实验07-PostgreSQL安装及监控

环境

机器：
移动云的
10.176.140.72 plat-ecloud01-mgmt-monitor04 monitor04

操作系统：
CentOS Linux release 7.3.1611 (Core)

InfluxDB版本：
influxdb-1.7.9

telegraf版本：
telegraf-1.12.6

grafana版本：
grafana-4.3.1

postgresql版本：
postgresql12-12.1

安装postgresql

参考：官网文档

下载最新repo：

[root@plat-ecloud01-mgmt-monitor04 ~]# yum install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm -y

安装客户端，服务端，额外模块，拓展包：

[root@plat-ecloud01-mgmt-monitor04 ~]# yum install postgresql12 postgresql12-server postgresql12-contrib postgresql12-devel -y

配置postgresql

1.创建目录

创建目录，分别用来存放数据文件和归档日志：

[root@plat-ecloud01-mgmt-monitor04 ~]# mkdir -p /opt/data/pgdata
[root@plat-ecloud01-mgmt-monitor04 ~]# mkdir -p /opt/data/archlog
[root@plat-ecloud01-mgmt-monitor04 ~]# chown -R postgres:postgres /opt/data

2.配置环境变量

修改postgres用户环境变量：

[root@plat-ecloud01-mgmt-monitor04 ~]# su - postgres
-bash-4.2$ vim ./.bash_profile

[ -f /etc/profile ] && source /etc/profile
export PGPORT=9999
export PGUSER= postgres
export PGDATA=/opt/data/pgdata
export LANG=en_US.utf8

export PGHOME=/usr/pgsql-12/bin
export LD_LIBRARY_PATH=$PGHOME/lib:/lib64:/usr/lib64:/usr/local/lib64:/lib:/usr/lib:/usr/local/lib
export DATE=`date +"%Y%m%d%H%M"`
export PATH=$PGHOME/bin:$PATH:.
export MANPATH=$PGHOME/share/man:$MANPATH
alias rm='rm -i'
alias ll='ls -lh'

# If you want to customize your settings,
# Use the file below. This is not overridden
# by the RPMS.
[ -f /var/lib/pgsql/.pgsql_profile ] && source /var/lib/pgsql/.pgsql_profile

使环境变量变更生效：

-bash-4.2$ source ./.bash_profile

3.初始化数据库

使用postgres用户执行初始化命令，指定字符集为UTF8，指定数据目录为/opt/data/pgdata/

-bash-4.2$ /usr/pgsql-12/bin/initdb -E UTF8 --locale=C -D /opt/data/pgdata/
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "C".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /opt/data/pgdata ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Asia/Shanghai
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    /usr/pgsql-12/bin/pg_ctl -D /opt/data/pgdata/ -l logfile start

4.修改数据库配置

修改config文件：

-bash-4.2$ vim /opt/data/pgdata/postgresql.conf 

data_directory = '/opt/data/pgdata'
hba_file = '/opt/data/pgdata/pg_hba.conf'
ident_file = '/opt/data/pgdata/pg_ident.conf'
listen_addresses = '*'
port = 9999
max_connections = 1000
superuser_reserved_connections = 5
shared_buffers = 3GB
wal_level = replica
fsync = on
synchronous_commit = on
archive_mode = on
archive_command = 'test ! -f /opt/data/archlog//backup_in_progress && (test ! -f /opt/data/archlog/%f && cp %p /opt/data/archlog/%f)'
'test ! -f /opt/data/archlog//backup_in_progress && (test ! -f /opt/data/archlog/%f && cp %p /opt/data/archlog/%f)'
max_wal_senders = 10
wal_keep_segments = 32
hot_standby = on
hot_standby_feedback = on

修改hba文件：

-bash-4.2$ vim /opt/data/pgdata/pg_hba.conf

host all all 0.0.0.0/0 md5
host replication all 0.0.0.0/0 trust

5.启动数据库

使用postgres用户启动数据库

-bash-4.2$ /usr/pgsql-12/bin/pg_ctl start -D /opt/data/pgdata/
waiting for server to start....2019-12-13 10:50:39.277 CST [12881] LOG:  starting PostgreSQL 12.1 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
2019-12-13 10:50:39.278 CST [12881] LOG:  listening on IPv4 address "0.0.0.0", port 9999
2019-12-13 10:50:39.278 CST [12881] LOG:  listening on IPv6 address "::", port 9999
2019-12-13 10:50:39.281 CST [12881] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.9999"
2019-12-13 10:50:39.287 CST [12881] LOG:  listening on Unix socket "/tmp/.s.PGSQL.9999"
2019-12-13 10:50:39.464 CST [12881] LOG:  redirecting log output to logging collector process
2019-12-13 10:50:39.464 CST [12881] HINT:  Future log output will appear in directory "log".
 done
server started

6.配置密码

使用postgres用户进入数据库，并修改postgres的密码为postgres：

-bash-4.2$ psql -p 9999
psql (12.1)
Type "help" for help.

postgres=# alter user postgres with password 'postgres';
ALTER ROLE

监控配置

1.创建测试数据

建库test

postgres=# create database test ;
CREATE DATABASE

建表，并插入数据：

postgres=# \c test
You are now connected to database "test" as user "postgres".
test=# select i,'t:'||i as t into t from generate_series(1,10000000) as i;
SELECT 10000000

查看表大小：

test=# select pg_size_pretty(pg_relation_size('t'));
 pg_size_pretty 
----------------
 422 MB
(1 row)

查看库大小：

test=# select schemaname,round(sum(pg_total_relation_size(schemaname||'.'||tablename))/1024/1024) "Mb" from pg_tables  group by 1;
     schemaname     | Mb  
--------------------+-----
 public             | 422
 pg_catalog         |   8
 information_schema |   0

2.配置telegraf

参考文档：官方文档
需要在telegraf中添加数据库相关配置：

[root@plat-ecloud01-mgmt-monitor04 ~]# vim /etc/telegraf/telegraf.conf

 # Read metrics from one or many postgresql servers
 [[inputs.postgresql]]
   ## specify address via a url matching:
   ##   postgres://[pqgotest[:password]]@localhost[/dbname]\
   ##       ?sslmode=[disable|verify-ca|verify-full]
   ## or a simple string:
   ##   host=localhost user=pqotest password=... sslmode=... dbname=app_production
   ##
   ## All connection parameters are optional.
   ##
   ## Without the dbname parameter, the driver will default to a database
   ## with the same name as the user. This dbname is just for instantiating a
   ## connection with the server and doesn't restrict the databases we are trying
   ## to grab metrics for.
   ##
   # address = "host=localhost user=postgres sslmode=disable"
   address = "postgres://postgres:postgres@localhost:9999"  #数据库地址，格式参见前面介绍
   ## A custom name for the database that will be used as the "server" tag in the
   ## measurement output. If not specified, a default one generated from
   ## the connection address is used.
   # outputaddress = "db01"

   ## connection configuration.
   ## maxlifetime - specify the maximum lifetime of a connection.
   ## default is forever (0s)
   max_lifetime = "0s"

   ## A  list of databases to explicitly ignore.  If not specified, metrics for all
   ## databases are gathered.  Do NOT use with the 'databases' option.
   # ignored_databases = ["postgres", "template0", "template1"]
   ignored_databases = ["postgres", "template0", "template1"]  #忽略括号内数据库
   ## A list of databases to pull metrics about. If not specified, metrics for all
   ## databases are gathered.  Do NOT use with the 'ignored_databases' option.
   # databases = ["app_production", "testing"]
   # databases = ["test"] #只监控括号内数据库不能与ignored_databases同用

重启telegraf：

[root@plat-ecloud01-mgmt-monitor04 ~]# systemctl restart telegraf

3.验证

查看influxdb的telegraf库，发现已新建了名为postgresql的measurement：

[root@plat-ecloud01-mgmt-monitor04 ~]# influx -precision rfc3339 -database telegraf -execute 'select * from postgresql limit 3'
name: postgresql
time                 blk_read_time blk_write_time blks_hit blks_read buffers_alloc buffers_backend buffers_backend_fsync buffers_checkpoint buffers_clean checkpoint_sync_time checkpoint_write_time checkpoints_req checkpoints_timed conflicts datid datname  db       deadlocks host                         maxwritten_clean numbackends server                                             temp_bytes temp_files tup_deleted tup_fetched tup_inserted tup_returned tup_updated xact_commit xact_rollback
----                 ------------- -------------- -------- --------- ------------- --------------- --------------------- ------------------ ------------- -------------------- --------------------- --------------- ----------------- --------- ----- -------  --       --------- ----                         ---------------- ----------- ------                                             ---------- ---------- ----------- ----------- ------------ ------------ ----------- ----------- -------------
2019-12-13T06:46:20Z 0             0              20586    254       3241          135511          0                     535                0             12                   149643                3               45                0         14185 postgres postgres 0         plat-ecloud01-mgmt-monitor04 0                0           dbname=test host=localhost port=9999 user=postgres 0          0          0           4182        0            290465       0           478         0
2019-12-13T06:46:20Z 0             0              25572    83830                                                                                                                                                                       0         16384 test     test     0         plat-ecloud01-mgmt-monitor04                  1           dbname=test host=localhost port=9999 user=postgres 141400000  2          68          5629        10100113     287196       3           471         1
2019-12-13T06:46:30Z 0             0              20663    254       3333          135511          0                     535                0             12                   149643                3               45                0         14185 postgres postgres 0         plat-ecloud01-mgmt-monitor04 0                0           dbname=test host=localhost port=9999 user=postgres 0          0          0           4193        0            291704       0           480         0