在oracle中如果表或索引的统计信息不准，可能导致oracle走错执行计划。同样在postgres中也同样存在统计信息，如果统计信息不准也会导致走错执行计划。本文中主要测试什么时候会触发postgres做自动统计信息收集，因为postgres数据库是开源的，所以在源码中能看到触发统计信息的公式，可以通过临界值来测试自动统计信息收集。
postgres中统计信息是否准确是依赖于postgresql.conf中是否配置autovacuum该参数为on。数据被删除或更新之后，但是在没有提交的情况下，其他会话是无法看到被修改的数据的，postgres和oracle实现该功能的方式有所不同，oracle数据库是通过undo的方式通过构造CR块来访问被修改前的数据，postgres数据库是通过每次修改数据之前，把之前的数据复制一份，然后修改复制之后的数据，而原来的数据是不会变的，如果事物还没有提交，那么其他会话就通过访问原来的记录。当从表中删除某些记录时，PostgreSQL并不会立即从数据文件中删除这些记录，他们会被标记为删除。当更新某些记录时，相当于把旧的数据复制一份，然后修改新插入的数据，老版本的记录仍然存在。这就是postgres的MVCC（Multiversion Concurrent Control），MVCC的目的是提高并发度，读操作不会阻塞写操作，写操作也不会阻止读操作。但是这样会引入一个新问题，当修改的事物已经提交，原来的旧记录已经不再需要，这个时候旧记录的数据就需要删除，不然会导致表占用的空间越来越大。所以这个时候需要vacuum进程对旧记录的空间进行标识，进而在表内重用。
一、首先测试postgres下的MVCC机制
1、创建一个张测试表TEST
[postgres@postgres pgdata]$ psql
psql (9.3.4)
Type "help" for help.
postgres=#
postgres=# create table test (id int, name text);
CREATE TABLE
postgres=#
postgres=# select pg_relation_filenode('test'::regclass);
pg_relation_filenode
----------------------
16390 <====记录表test对应的filenode
(1 row)
postgres=# select pg_relation_filepath('test'::regclass);
pg_relation_filepath
----------------------
base/12902/16390 <===查看表test对应的数据文件位置，格式是/base/数据库oid/文件filenode, 比如这里的12902就是代表 postgres数据库
(1 row)
postgres=#
postgres=#
postgres=# select oid, datname from pg_database;
oid | datname
-------+-----------
1 | template1
12897 | template0
12902 | postgres <=====12902,postgres数据库的oid，oid（object identified），就是数据库中的一个唯一标识
(3 rows)
postgres=# select current_database();
current_database
------------------
postgres
(1 row)
postgres=#
postgres=# select oid, relname from pg_class where relname = 'test';
oid | relname
-------+---------
16390 | test <====表test本身的filenode和oid一样，都是16390
(1 row)
postgres=#
postgres=# 2、查看文件大小
[postgres@postgres 12902]$ pwd
/home/postgres/pgdata/base/12902
[postgres@postgres 12902]$ ls -lt | more
total 6588
-rw------- 1 postgres postgres 0 Jul 21 13:53 16390 <========16390就是代表test表，表的大小是0字节
-rw------- 1 postgres postgres 344064 Jul 21 13:51 12653
-rw------- 1 postgres postgres 8192 Jul 21 13:46 12688_vm
-rw------- 1 postgres postgres 237568 Jul 21 13:46 12772
-rw------- 1 postgres postgres 229376 Jul 21 13:46 12773
-rw------- 1 postgres postgres 90112 Jul 21 13:46 12656
-rw------- 1 postgres postgres 131072 Jul 21 13:46 12655
postgres=# insert into test select generate_series(1,5),md5(chr((random()*25)::int+20567)); <=====插入5行数据
INSERT 0 5
postgres=# select ctid,xmin,xmax, id,name from test;
ctid | xmin | xmax | id | name
-------+------+------+----+----------------------------------
(0,1) | 1823 | 0 | 1 | a1cce56977a6d0c7c14f1455289e7567
(0,2) | 1823 | 0 | 2 | fd2f5bde60ac062ad3ce027cd533f8c3
(0,3) | 1823 | 0 | 3 | b31314f9e4615d8a90abdcc10e99925e
(0,4) | 1823 | 0 | 4 | 62d053d847ac40d8ac33c4bb94c50b35
(0,5) | 1823 | 0 | 5 | fd2f5bde60ac062ad3ce027cd533f8c3
(5 rows)
postgres=#
其中ctid代表数据的物理位置，比如（0，1）含义就是0号块的第一行。
xmin就是代表数据插入时的事物号(可以通过txid_current()查询),每次使用之后会自动加1
xmax代表事物被修改时的事物号
postgres=# select txid_current();
txid_current
--------------
1824
(1 row)
postgres=#
<================重新查看数据文件的大小
[postgres@postgres 12902]$ pwd
/home/postgres/pgdata/base/12902
[postgres@postgres 12902]$
[postgres@postgres 12902]$ ls -lt | more
total 6596
-rw------- 1 postgres postgres 8192 Jul 21 13:57 16390 <======大小8192字节
-rw------- 1 postgres postgres 237568 Jul 21 13:56 12772
-rw------- 1 postgres postgres 229376 Jul 21 13:56 12773
-rw------- 1 postgres postgres 90112 Jul 21 13:56 12656
数据库最小存储单位是块，块的大小在安装编译postgres中可以指定，安装之后的postgres数据库可以通过pg_config命令查看，如下
postgres=# \q
[postgres@postgres ~]$
[postgres@postgres ~]$ pg_con
pg_config pg_controldata
[postgres@postgres ~]$ pg_config
BINDIR = /opt/pgsql9.3.4/bin
DOCDIR = /opt/pgsql9.3.4/share/doc
HTMLDIR = /opt/pgsql9.3.4/share/doc
INCLUDEDIR = /opt/pgsql9.3.4/include
PKGINCLUDEDIR = /opt/pgsql9.3.4/include
INCLUDEDIR-SERVER = /opt/pgsql9.3.4/include/server
LIBDIR = /opt/pgsql9.3.4/lib
PKGLIBDIR = /opt/pgsql9.3.4/lib
LOCALEDIR = /opt/pgsql9.3.4/share/locale
MANDIR = /opt/pgsql9.3.4/share/man
SHAREDIR = /opt/pgsql9.3.4/share
SYSCONFDIR = /opt/pgsql9.3.4/etc
PGXS = /opt/pgsql9.3.4/lib/pgxs/src/makefiles/pgxs.mk
CONFIGURE = '--prefix=/opt/pgsql9.3.4' '--with-pgport=5432' '--with-perl' '--with-tcl' '--with-python' '--with-openssl' '--with-pam' '--without-ldap' '--with-libxml' '--with-libxslt' '--enable-thread-safety' '--with-wal-blocksize=8' '--with-blocksize=8' '--enable-dtrace' '--enable-debug' '--enable-cassert'
CC = gcc
CPPFLAGS = -D_GNU_SOURCE -I/usr/include/libxml2
CFLAGS = -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -g
CFLAGS_SL = -fpic
LDFLAGS = -L../../../src/common -Wl,--as-needed -Wl,-rpath,'/opt/pgsql9.3.4/lib',--enable-new-dtags
LDFLAGS_EX =
LDFLAGS_SL =
LIBS = -lpgport -lpgcommon -lxslt -lxml2 -lpam -lssl -lcrypto -lz -lreadline -lcrypt -ldl -lm
VERSION = PostgreSQL 9.3.4
[postgres@postgres ~]$ 其中CONFIGURE中的'--with-blocksize=8' 就代表了编译数据库时默认块大小。
3、创建两个会话测试
会话1
更新一行数据不提交
postgres=# select pg_backend_pid(); <=====记录会话pid
pg_backend_pid
----------------
7177
(1 row)
postgres=# begin; <====开始事物
BEGIN
postgres=# select txid_current();
txid_current
--------------
1825
(1 row)
postgres=# update test set id = 6 where id = 1; <=======更新一行数据不提交
UPDATE 1
postgres=# select ctid,xmin,xmax, id,name from test;
ctid | xmin | xmax | id | name
-------+------+------+----+----------------------------------
(0,2) | 1823 | 0 | 2 | fd2f5bde60ac062ad3ce027cd533f8c3
(0,3) | 1823 | 0 | 3 | b31314f9e4615d8a90abdcc10e99925e
(0,4) | 1823 | 0 | 4 | 62d053d847ac40d8ac33c4bb94c50b35
(0,5) | 1823 | 0 | 5 | fd2f5bde60ac062ad3ce027cd533f8c3
(0,6) | 1825 | 0 | 6 | a1cce56977a6d0c7c14f1455289e7567 <======由于更新的id=1的第一行数据，第一行数据本身物理位置为(0,1),现在可以看到被修改的数据的物理位置是(0,6)，也就是说源数据没有改变，现在修改的只是重新复制的一份数据， 因为还没有提交，其他会话查询test表时，看到的记录应该还是(0,1)位置的数据。xmin记录了数据被修改时的事物id(txid)。
(5 rows)
postgres=#
会话2
新开一个会话，查询表test
[postgres@postgres 12902]$ psql
psql (9.3.4)
Type "help" for help.
postgres=#
postgres=# select pg_backend_pid();
pg_backend_pid
----------------
9487
(1 row)
postgres=# begin;
BEGIN
postgres=# select txid_current();
txid_current
--------------
1826
(1 row)
postgres=# select ctid,xmin,xmax,id,name from test;
ctid | xmin | xmax | id | name
-------+------+------+----+----------------------------------
(0,1) | 1823 | 1825 | 1 | a1cce56977a6d0c7c14f1455289e7567 <=====注意新开启的会话查看到的数据的物理位置为(0,1)。并且xmax字段上面的数字为1825，这个事物号正是会话1修改时的事物ID。
(0,2) | 1823 | 0 | 2 | fd2f5bde60ac062ad3ce027cd533f8c3
(0,3) | 1823 | 0 | 3 | b31314f9e4615d8a90abdcc10e99925e
(0,4) | 1823 | 0 | 4 | 62d053d847ac40d8ac33c4bb94c50b35
(0,5) | 1823 | 0 | 5 | fd2f5bde60ac062ad3ce027cd533f8c3
(5 rows)
postgres=#
postgres=# commit;
COMMIT
postgres=# 上面的描述的就是postgres的MVCC功能。
4、查询测试表TEST的统计信息
这个时候还没有对test表做统计信息，现在查询表test的属性记录。
postgres=# \x
Expanded display is on.
postgres=# select * from pg_class where relname = 'test';
-[ RECORD 1 ]--+------
relname | test
relnamespace | 2200
reltype | 16392
reloftype | 0
relowner | 10
relam | 0
relfilenode | 16390
reltablespace | 0
relpages | 0 <========pages代表占用的块数，由于还没有做统计信息，所以这里显示为0
reltuples | 0 <========tuples表示表的行数，由于还没有做统计信息，所以这里显示为0
relallvisible | 0
reltoastrelid | 16393
reltoastidxid | 0
relhasindex | f
relisshared | f
relpersistence | p
relkind | r
relnatts | 2
relchecks | 0
relhasoids | f
relhaspkey | f
relhasrules | f
relhastriggers | f
relhassubclass | f
relispopulated | t
relfrozenxid | 1822
relminmxid | 1
relacl |
reloptions |
postgres=#
postgres=#
postgres=# select * from pg_stat_user_tables where relname = 'test';
-[ RECORD 1 ]-----+-----------------------------
relid | 16390
schemaname | public
relname | test
seq_scan | 7
seq_tup_read | 35
idx_scan |
idx_tup_fetch |
n_tup_ins | 5 <=====表test插入多少行记录
n_tup_upd | 0 <=====表test更新过多少行记录
n_tup_del | 0 <=====表test删除过多少行记录
n_tup_hot_upd | 0
n_live_tup | 5 <=====活的数据为5行
n_dead_tup | 0 <=====死的数据(待清理数据)为0行
last_vacuum | <=====最后一次手动vacuum时间
last_autovacuum | <=====最后一次自动vacuum时间
last_analyze | <=====最后一次手动分析时间
last_autoanalyze | <=====最后一个自动统计信息时间
vacuum_count | 0
autovacuum_count | 0
analyze_count | 0
autoanalyze_count | 0
postgres=#
postgres=# analyze test; <=====对表做统计信息收集
ANALYZE
postgres=# select * from pg_class where relname = 'test';
-[ RECORD 1 ]--+------
relname | test
relnamespace | 2200
reltype | 16392
reloftype | 0
relowner | 10
relam | 0
relfilenode | 16390
reltablespace | 0
relpages | 1 <======占用一个块(8192字节，通过上面的数据文件大小也可以看到)
reltuples | 5 <======一共有5行数据
relallvisible | 0
reltoastrelid | 16393
reltoastidxid | 0
relhasindex | f
relisshared | f
relpersistence | p
relkind | r
relnatts | 2
relchecks | 0
relhasoids | f
relhaspkey | f
relhasrules | f
relhastriggers | f
relhassubclass | f
relispopulated | t
relfrozenxid | 1822
relminmxid | 1
relacl |
reloptions |
postgres=# select * from pg_stat_user_tables where relname = 'test'; <====统计信息查询
-[ RECORD 1 ]-----+-----------------------------
relid | 16390
schemaname | public
relname | test
seq_scan | 7
seq_tup_read | 35
idx_scan |
idx_tup_fetch |
n_tup_ins | 5
n_tup_upd | 1 <====前面测试的时候，更新了第一行数据，所以这里显示为1。
n_tup_del | 0
n_tup_hot_upd | 1
n_live_tup | 5
n_dead_tup | 1 <====由于更新了第一行，所以postgres认为被更新的那行数据已经是dead，因为已经提交，所以第一行旧的数据可以被清理。
last_vacuum |
last_autovacuum |
last_analyze | 2018-07-21 14:13:40.21459-04 <=====手动收集统计信息的时间
last_autoanalyze |
vacuum_count | 0
autovacuum_count | 0
analyze_count | 1 <=====统计信息次数
autoanalyze_count | 0
postgres=# delete from test; <======把数据全部删除
DELETE 5
postgres=# select * from test;
(No rows)
postgres=#
postgres=# analyze;
ANALYZE
postgres=# select *,now() from pg_class where relname = 'test';
-[ RECORD 1 ]--+------------------------------
relname | test
relnamespace | 2200
reltype | 16392
reloftype | 0
relowner | 10
relam | 0
relfilenode | 16390
reltablespace | 0
relpages | 1
reltuples | 0
relallvisible | 0
reltoastrelid | 16393
reltoastidxid | 0
relhasindex | f
relisshared | f
relpersistence | p
relkind | r
relnatts | 2
relchecks | 0
relhasoids | f
relhaspkey | f
relhasrules | f
relhastriggers | f
relhassubclass | f
relispopulated | t
relfrozenxid | 1822
relminmxid | 1
relacl |
reloptions |
now | 2018-07-21 14:26:22.401484-04
postgres=#
postgres=# select *,now() from pg_stat_user_tables where relname = 'test';
-[ RECORD 1 ]-----+------------------------------
relid | 16390
schemaname | public
relname | test
seq_scan | 8
seq_tup_read | 40
idx_scan |
idx_tup_fetch |
n_tup_ins | 5
n_tup_upd | 1
n_tup_del | 5
n_tup_hot_upd | 1
n_live_tup | 0 <====现在已经不存在活的记录
n_dead_tup | 6 <====之前1行存在的一条再加上现在删除的5行，所以dead数据是6行
last_vacuum |
last_autovacuum |
last_analyze | 2018-07-21 14:26:08.614944-04
last_autoanalyze |
vacuum_count | 0
autovacuum_count | 0
analyze_count | 2
autoanalyze_count | 0
now | 2018-07-21 14:26:49.969941-04
postgres=#
由于没有达到自动触发条件，索引这个时候last_autoanalyze，autovacuum_count的记录仍然是空的。
查询表数据占用的大小如下：
postgres=#
postgres=# \x
Expanded display is off.
postgres=# \df *filepath*
List of functions
Schema | Name | Result data type | Argument data types | Type
------------+----------------------+------------------+---------------------+--------
pg_catalog | pg_relation_filepath | text | regclass | normal
(1 row)
postgres=# select pg_relation_filepath('test'::regclass);
pg_relation_filepath
----------------------
base/12902/16390
(1 row)
postgres=#
postgres=#
postgres=# \q
[postgres@postgres pgdata]$ cd base/12902/
[postgres@postgres 12902]$ ls -ltr 16390
-rw------- 1 postgres postgres 8192 Jul 21 14:31 16390 <=====占用大小还是8192字节，也就是说虽然test表已经没有数据，但是仍然占用一个块(postgres中存储的最小单位。具体块的大小，安装编译时可以指定)，也就是说占用的空间没有释放。
下面对dead记录进行清除
[postgres@postgres 12902]$
[postgres@postgres 12902]$
[postgres@postgres 12902]$ psql
psql (9.3.4)
Type "help" for help.
postgres=# select * from test;
id | name
----+------
(0 rows)
postgres=# \x
Expanded display is on.
postgres=# select *, now() from pg_stat_user_tables where relname = 'test';
-[ RECORD 1 ]-----+------------------------------
relid | 16390
schemaname | public
relname | test
seq_scan | 10
seq_tup_read | 40
idx_scan |
idx_tup_fetch |
n_tup_ins | 5
n_tup_upd | 1
n_tup_del | 5
n_tup_hot_upd | 1
n_live_tup | 0
n_dead_tup | 6 <=====清除之前dead数据为6
last_vacuum |
last_autovacuum |
last_analyze | 2018-07-21 14:26:08.614944-04
last_autoanalyze |
vacuum_count | 0
autovacuum_count | 0
analyze_count | 2
autoanalyze_count | 0
now | 2018-07-21 14:34:17.903573-04
postgres=# vacuum test; <=====对死记录进行移除
VACUUM
postgres=# select *, now() from pg_stat_user_tables where relname = 'test';
-[ RECORD 1 ]-----+------------------------------
relid | 16390
schemaname | public
relname | test
seq_scan | 10
seq_tup_read | 40
idx_scan |
idx_tup_fetch |
n_tup_ins | 5
n_tup_upd | 1
n_tup_del | 5
n_tup_hot_upd | 1
n_live_tup | 0
n_dead_tup | 0 <=====清除之后dead数据已经不存在，说明从块中已经清除。
last_vacuum | 2018-07-21 14:34:24.356313-04 <======手动触发的时间
last_autovacuum |
last_analyze | 2018-07-21 14:26:08.614944-04
last_autoanalyze |
vacuum_count | 1
autovacuum_count | 0
analyze_count | 2
autoanalyze_count | 0
now | 2018-07-21 14:34:26.581347-04
postgres=# \x
Expanded display is off.
postgres=# select pg_relation_filepath('test'::regclass);
pg_relation_filepath
----------------------
base/12902/16390
(1 row)
postgres=#
[postgres@postgres 12902]$
[postgres@postgres 12902]$ pwd
/home/postgres/pgdata/base/12902
[postgres@postgres 12902]$ ls -ltr 16390
-rw------- 1 postgres postgres 0 Jul 21 14:34 16390 <======注意，这个时候16390的数据文件大小变成了0
[postgres@postgres 12902]$ 通过pg_stat_user_tables也可以看到包含了这样四个字段last_vacuum、last_analyze、last_autovacuum、last_autoanalyze，这四个字段的单位都是时间， 其中前两个代表手动触发时的时间，后面两个代表自动触发时的条件。下面就介绍last_autovacuum、last_autoanalyze什么情况下会触发这两项功能。
5、postgres统计信息相关的进程和源码文件
[postgres@postgres ~]$ ps f -U postgres
PID TTY STAT TIME COMMAND
7304 pts/1 S 0:00 -bash
9486 pts/1 S+ 0:00 \_ psql
3628 pts/0 S 0:00 -bash
25815 pts/0 R+ 0:00 \_ ps f -U postgres
7129 pts/0 S 0:00 /opt/pgsql9.3.4/bin/postgres -D /home/postgres/pgdata
7131 ? Ss 0:00 \_ postgres: checkpointer process
7132 ? Ss 0:00 \_ postgres: writer process
7133 ? Ss 0:00 \_ postgres: wal writer process
7134 ? Ss 0:00 \_ postgres: autovacuum launcher process <======autovacuum
7135 ? Ss 0:00 \_ postgres: stats collector process <======收集统计信息进程
9487 ? Ss 0:00 \_ postgres: postgres postgres [local] idle
[postgres@postgres ~]$
[postgres@postgres postmaster]$ pwd
/opt/soft_bak/postgresql-9.3.4/src/backend/postmaster <=======对应的源文件在src/backend/postmaster里面
[postgres@postgres postmaster]$ ls -ltr
total 1548
-rw-r--r--. 1 1107 1107 11430 Mar 17 2014 walwriter.c
-rw-r--r--. 1 1107 1107 36911 Mar 17 2014 syslogger.c
-rw-r--r--. 1 1107 1107 6339 Mar 17 2014 startup.c
-rw-r--r--. 1 1107 1107 172454 Mar 17 2014 postmaster.c
-rw-r--r--. 1 1107 1107 135690 Mar 17 2014 pgstat.c <======统计信息
-rw-r--r--. 1 1107 1107 19187 Mar 17 2014 pgarch.c
-rw-r--r--. 1 1107 1107 540 Mar 17 2014 Makefile
-rw-r--r--. 1 1107 1107 3893 Mar 17 2014 fork_process.c
-rw-r--r--. 1 1107 1107 41599 Mar 17 2014 checkpointer.c
-rw-r--r--. 1 1107 1107 11923 Mar 17 2014 bgwriter.c
-rw-r--r--. 1 1107 1107 88783 Mar 17 2014 autovacuum.c <========autovacuum进程源码位置
-rw-r--r--. 1 root root 177000 Jul 8 18:50 autovacuum.o
-rw-r--r--. 1 root root 42984 Jul 8 18:50 bgwriter.o
-rw-r--r--. 1 root root 7096 Jul 8 18:50 fork_process.o
-rw-r--r--. 1 root root 32336 Jul 8 18:50 pgarch.o
-rw-r--r--. 1 root root 287912 Jul 8 18:50 pgstat.o
-rw-r--r--. 1 root root 256176 Jul 8 18:50 postmaster.o
-rw-r--r--. 1 root root 15752 Jul 8 18:50 startup.o
-rw-r--r--. 1 root root 61600 Jul 8 18:50 syslogger.o
-rw-r--r--. 1 root root 44032 Jul 8 18:50 walwriter.o
-rw-r--r--. 1 root root 82872 Jul 8 18:50 checkpointer.o
-rw-r--r--. 1 root root 349 Jul 8 18:50 objfiles.txt
[postgres@postgres postmaster]$统计信息存放位置和格式
----数据库刚启动到关闭之前，会把统计信息存放到临时文件夹里面，在正常关闭数据库之后会把临时文件中的统计信息mv到pg_stat文件夹里面，统计信息都放在临时文件夹中pg_stat_tmp中，在postgres.conf中可以配置临时文件存放位置
[postgres@postgres pgdata]$ cat postgresql.conf | grep 'pg_stat'
#stats_temp_directory = 'pg_stat_tmp'
[postgres@postgres pgdata]$
[postgres@postgres pgdata]$ cd pg_stat_tmp/
[postgres@postgres pg_stat_tmp]$ ls -ltr
total 24
-rw------- 1 postgres postgres 2878 Jul 21 17:05 db_0.stat
-rw------- 1 postgres postgres 471 Jul 21 17:05 global.stat
-rw------- 1 postgres postgres 15215 Jul 21 17:05 db_12902.stat <======12902是数据库的oid，代表postgres数据库，
[postgres@postgres pg_stat_tmp]$ cd ..
[postgres@postgres pgdata]$ cd pg_stat
[postgres@postgres pg_stat]$ ls -ltr
total 0
[postgres@postgres pg_stat]$
[postgres@postgres pg_stat]$ psql
psql (9.3.4)
Type "help" for help.
postgres=# select oid, datname from pg_database where oid = 12902;
oid | datname
-------+----------
12902 | postgres <=====12902就对应postgres数据库
(1 row)
postgres=#
postgres=# \help create database
Command: CREATE DATABASE
Description: create a new database
Syntax:
CREATE DATABASE name
[ [ WITH ] [ OWNER [=] user_name ]
[ TEMPLATE [=] template ]
[ ENCODING [=] encoding ]
[ LC_COLLATE [=] lc_collate ]
[ LC_CTYPE [=] lc_ctype ]
[ TABLESPACE [=] tablespace_name ]
[ CONNECTION LIMIT [=] connlimit ] ]
postgres=#
postgres=# create database qxy TEMPLATE template0; <======创建一个新数据库
CREATE DATABASE
postgres=#
postgres=# select oid, datname from pg_database;
oid | datname
-------+-----------
1 | template1
12897 | template0
12902 | postgres
16406 | qxy <======新数据库对应的oid为16406
(4 rows)
postgres=#
postgres=# \c qxy postgres <======切换到新创建的数据库qxy
You are now connected to database "qxy" as user "postgres".
qxy=#
qxy=# create table t (id int);
CREATE TABLE
qxy=# insert into t select generate_series(1,10); <======插入10行数据
INSERT 0 10
qxy=# analyze t;
ANALYZE
qxy=#
qxy=# \q
[postgres@postgres pg_stat]$ cd ..
[postgres@postgres pgdata]$ cd pg_stat_tmp/
[postgres@postgres pg_stat_tmp]$ ls -ltr
total 32
-rw------- 1 postgres postgres 4230 Jul 21 17:09 db_16406.stat <=====qxy数据库对应的统计信息已经生成
-rw------- 1 postgres postgres 3216 Jul 21 17:09 db_0.stat
-rw------- 1 postgres postgres 656 Jul 21 17:09 global.stat
-rw------- 1 postgres postgres 15215 Jul 21 17:09 db_12902.stat
[postgres@postgres pg_stat_tmp]$ psql
postgres=# \l+
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges | Size | Tablespace | Description
-----------+----------+----------+------------+------------+-----------------------+---------+------------+--------------------------------------------
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | 6706 kB | pg_default | default administrative connection database
qxy | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | 6578 kB | pg_default |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +| 6457 kB | pg_default | unmodifiable empty database
| | | | | postgres=CTc/postgres | | |
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +| 6457 kB | pg_default | default template for new databases
| | | | | postgres=CTc/postgres | | |
(4 rows)
postgres=# \c qxy postgres
qxy=# \x
Expanded display is on.
qxy=# select now(),* from pg_stat_all_tables where relname = 't';
-[ RECORD 1 ]-----+------------------------------
now | 2018-07-21 18:06:04.867794-04
relid | 16407
schemaname | public
relname | t
seq_scan | 0
seq_tup_read | 0
idx_scan |
idx_tup_fetch |
n_tup_ins | 10
n_tup_upd | 0
n_tup_del | 0
n_tup_hot_upd | 0
n_live_tup | 10
n_dead_tup | 0
last_vacuum |
last_autovacuum |
last_analyze | 2018-07-21 17:09:51.300716-04
last_autoanalyze |
vacuum_count | 0
autovacuum_count | 0
analyze_count | 1
autoanalyze_count | 0
qxy=# \q
<==========关闭数据库，然后删除统计信息文件
[postgres@postgres pg_stat_tmp]$ pg_ctl stop
LOG: received smart shutdown request
LOG: autovacuum launcher shutting down
waiting for server to shut down....LOG: shutting down
LOG: database system is shut down
done
server stopped
[postgres@postgres pg_stat_tmp]$cd ..
[postgres@postgres pg_stat]$ ls
db_0.stat db_12902.stat db_16406.stat global.stat <=====统计信息已经mv到pg_stat目录
[postgres@postgres pg_stat]$ rm -rf *
[postgres@postgres pg_stat]$
[postgres@postgres pg_stat]$ pg_ctl start
server starting
[postgres@postgres pg_stat]$ LOG: database system was shut down at 2018-07-21 18:07:17 EDT
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
[postgres@postgres pg_stat]$
[postgres@postgres pg_stat]$ psql -d qxy -U postgres
psql (9.3.4)
Type "help" for help.
qxy=# \x
Expanded display is on.
qxy=# select now(),* from pg_stat_all_tables where relname = 't';
-[ RECORD 1 ]-----+------------------------------
now | 2018-07-21 18:07:53.436379-04
relid | 16407
schemaname | public
relname | t
seq_scan | 0
seq_tup_read | 0
idx_scan |
idx_tup_fetch |
n_tup_ins | 0 <=======由于统计信息已经被删除，所以原来的10变成了0
n_tup_upd | 0
n_tup_del | 0
n_tup_hot_upd | 0
n_live_tup | 0
n_dead_tup | 0
last_vacuum |
last_autovacuum |
last_analyze | <=======时间变为空
last_autoanalyze |
vacuum_count | 0
autovacuum_count | 0
analyze_count | 0
autoanalyze_count | 0
qxy=# 统计信息读写的函数如下
/* ----------
* pgstat_read_statsfiles() -
*
* Reads in the existing statistics collector files and initializes the
* databases' hash table. If the permanent file name is requested (which
* only happens in the stats collector itself), also remove the file after
* reading; the in-memory status is now authoritative, and the permanent file
* would be out of date in case somebody else reads it.
*
* If a deep read is requested, table/function stats are read also, otherwise
* the table/function hash tables remain empty.
* ----------
*/
/* ----------
* pgstat_write_statsfiles() -
* Write the global statistics file, as well as requested DB files.
*
* If writing to the permanent files (happens when the collector is
* shutting down only), remove the temporary files so that backends
* starting up under a new postmaster can't read the old data before
* the new collector is ready.
*
* When 'allDbs' is false, only the requested databases (listed in
* last_statrequests) will be written; otherwise, all databases will be
* written.
* ----------
*/
#define PGSTAT_STAT_PERMANENT_DIRECTORY "pg_stat"
#define PGSTAT_STAT_PERMANENT_FILENAME "pg_stat/global.stat"
#define PGSTAT_STAT_PERMANENT_TMPFILE "pg_stat/global.tmp"
static void
pgstat_write_statsfiles(bool permanent, bool allDbs)
{
HASH_SEQ_STATUS hstat;
PgStat_StatDBEntry *dbentry;
FILE *fpout;
int32 format_id;
const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
int rc;
elog(DEBUG2, "writing statsfile '%s'", statfile);
/*
* Open the statistics temp file to write out the current values.
*/
fpout = AllocateFile(tmpfile, PG_BINARY_W);
if (fpout == NULL)
{
ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open temporary statistics file \"%s\": %m",
tmpfile)));
return;
}
/*
* Set the timestamp of the stats file.
*/
globalStats.stats_timestamp = GetCurrentTimestamp();
/*
* Write the file header --- currently just a format ID.
*/
format_id = PGSTAT_FILE_FORMAT_ID;
rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Write global stats struct
*/
rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Walk through the database table.
*/
hash_seq_init(&hstat, pgStatDBHash);
while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
{
/*
* Write out the tables and functions into the DB stat file, if
* required.
*
* We need to do this before the dbentry write, to ensure the
* timestamps written to both are consistent.
*/
if (allDbs || pgstat_db_requested(dbentry->databaseid))
{
dbentry->stats_timestamp = globalStats.stats_timestamp;
pgstat_write_db_statsfile(dbentry, permanent);
}
/*
* Write out the DB entry. We don't write the tables or functions
* pointers, since they're of no use to any other process.
*/
fputc('D', fpout);
rc = fwrite(dbentry, offsetof(PgStat_StatDBEntry, tables), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
/*
* No more output to be done. Close the temp file and replace the old
* pgstat.stat with it. The ferror() check replaces testing for error
* after each individual fputc or fwrite above.
*/
fputc('E', fpout);
if (ferror(fpout))
{
ereport(LOG,
(errcode_for_file_access(),
errmsg("could not write temporary statistics file \"%s\": %m",
tmpfile)));
FreeFile(fpout);
unlink(tmpfile);
}
else if (FreeFile(fpout) < 0)
{
ereport(LOG,
(errcode_for_file_access(),
errmsg("could not close temporary statistics file \"%s\": %m",
tmpfile)));
unlink(tmpfile);
}
else if (rename(tmpfile, statfile) < 0)
{
ereport(LOG,
(errcode_for_file_access(),
errmsg("could not rename temporary statistics file \"%s\" to \"%s\": %m",
tmpfile, statfile)));
unlink(tmpfile);
}
if (permanent)
unlink(pgstat_stat_filename);
/*
* Now throw away the list of requests. Note that requests sent after we
* started the write are still waiting on the network socket.
*/
if (!slist_is_empty(&last_statrequests))
{
slist_mutable_iter iter;
/*
* Strictly speaking we should do slist_delete_current() before
* freeing each request struct. We skip that and instead
* re-initialize the list header at the end. Nonetheless, we must use
* slist_foreach_modify, not just slist_foreach, since we will free
* the node's storage before advancing.
*/
slist_foreach_modify(iter, &last_statrequests)
{
DBWriteRequest *req;
req = slist_container(DBWriteRequest, next, iter.cur);
pfree(req);
}
slist_init(&last_statrequests);
}
}
/*
* return the filename for a DB stat file; filename is the output buffer,
* of length len.
*/
6、统计信息收集的维度的设置
========track_activities关闭和开启影响,收集sql语句的执行
开始时间以及sql语句的内容
postgres=# show track_activities; <=====该参数默认打开
track_activities
------------------
on
(1 row)
postgres=#
postgres=#
postgres=# select query, query_start from pg_stat_activity ;
query | query_start
---------------------------------------------------+-------------------------------
select query, query_start from pg_stat_activity ; | 2018-07-21 19:20:52.337224-04
(1 row)
postgres=# set track_activities=off; <=======关闭该参数之后，不会记录sql语句的执行情况
SET
postgres=# select query, query_start from pg_stat_activity ;
query | query_start
-------+-------------
|
(1 row)
postgres=#
上面的关闭时会话级别的，如果想在数据库级别关闭，需要修改postgres.conf配置参数中的track_activities
=====track_activity_query_size(integer)
指定统计信息中允许存储的sql长度，超出的长度将会阶段，默认1024
=======把该参数设置成100
# - Query/Index Statistics Collector -
#track_activities = on
#track_counts = on
#track_io_timing = off
#track_functions = none # none, pl, all
#track_activity_query_size = 1024 # (change requires restart)
track_activity_query_size = 100 # (change requires restart) 《=====该参数最小值为100
#stats_temp_directory = 'pg_stat_tmp'
[postgres@postgres pgdata]$ pg_ctl restart -D fast
pg_ctl: PID file "fast/postmaster.pid" does not exist
Is server running?
starting server anyway
pg_ctl: could not read file "fast/postmaster.opts"
[postgres@postgres pgdata]$ pg_ctl restart -m fast
waiting for server to shut down.... done
server stopped
server starting
[postgres@postgres pgdata]$ LOG: 00000: redirecting log output to logging collector process
HINT: Future log output will appear in directory "pg_clog".
LOCATION: SysLogger_Start, syslogger.c:649
[postgres@postgres pgdata]$ psql
psql (9.3.4)
Type "help" for help.
postgres=#
postgres=# \d test
Table "public.test"
Column | Type | Modifiers
--------+---------+-----------
id | integer |
name | text |
postgres=# truncate table test;
TRUNCATE TABLE
postgres=# show track_activity_query_size;
track_activity_query_size
---------------------------
100
(1 row)
postgres=# select pg_backend_pid();
pg_backend_pid
----------------
40050
(1 row)
postgres=# insert into test values (1,'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
postgres'# aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');
INSERT 0 1
postgres=#
<======新开启一个会话查询 40050的query记录
postgres=#
postgres=# select query, query_start from pg_stat_activity where pid = 40050;
query | query_start
-----------------------------------------------------------------------------------------------------+-------------------------------
insert into test values (1,'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa | 2018-07-21 19:27:38.267123-04
(1 row)
postgres=# select length(query), query_start from pg_stat_activity where pid = 40050;
length | query_start
--------+-------------------------------
99 | 2018-07-21 19:27:38.267123-04 <=======query的长度为99
(1 row)
postgres=#
=====track_counts(boolean) ----收集数据库的活动信息（如表的新增行数，删除的行数等）autovacuum进程需要用到这部分的信息
postgres=# create table test (id int, name text);
CREATE TABLE
postgres=# insert into test select generate_series(1,10000),md5(chr((random()*25)::int+20567));
INSERT 0 10000
postgres=# select * from pg_stat_all_tables where relname = 'test';
-[ RECORD 1 ]-----+-------
relid | 16422
schemaname | public
relname | test
seq_scan | 0
seq_tup_read | 0
idx_scan |
idx_tup_fetch |
n_tup_ins | 10000
n_tup_upd | 0
n_tup_del | 0
n_tup_hot_upd | 0
n_live_tup | 10000
n_dead_tup | 0
last_vacuum |
last_autovacuum |
last_analyze |
last_autoanalyze |
vacuum_count | 0
autovacuum_count | 0
analyze_count | 0
autoanalyze_count | 0
postgres=#
<======关闭track_counts
postgres=# set track_counts=off;
SET
postgres=# delete from test;
DELETE 10000
postgres=#
postgres=# select * from pg_stat_all_tables where relname = 'test';
-[ RECORD 1 ]-----+-----------------------------
relid | 16422
schemaname | public
relname | test
seq_scan | 0
seq_tup_read | 0
idx_scan |
idx_tup_fetch |
n_tup_ins | 10000
n_tup_upd | 0
n_tup_del | 0
n_tup_hot_upd | 0
n_live_tup | 10000 <======提示记录还都是有效的
n_dead_tup | 0
last_vacuum |
last_autovacuum |
last_analyze |
last_autoanalyze | 2018-07-21 19:32:38.50718-04
vacuum_count | 0
autovacuum_count | 0
analyze_count | 0
autoanalyze_count | 1
postgres=#
<=====重新打开track_counts=on
postgres=# set track_counts=on
postgres-# ;
SET
postgres=#
postgres=# insert into test select generate_series(1,1000),md5(chr((random()*25)::int+20567));
INSERT 0 1000
postgres=# select * from pg_stat_all_tables where relname = 'test';
-[ RECORD 1 ]-----+-----------------------------
relid | 16422
schemaname | public
relname | test
seq_scan | 0
seq_tup_read | 0
idx_scan |
idx_tup_fetch |
n_tup_ins | 11000
n_tup_upd | 0
n_tup_del | 0
n_tup_hot_upd | 0
n_live_tup | 11000 <======提示记录还有11000行，其实这个时候已经不准了，因为这个时候只有1000数据
n_dead_tup | 0
last_vacuum |
last_autovacuum |
last_analyze |
last_autoanalyze | 2018-07-21 19:32:38.50718-04
vacuum_count | 0
autovacuum_count | 0
analyze_count | 0
autoanalyze_count | 1
postgres=#
postgres=# select count(*) from test;
-[ RECORD 1 ]
count | 1000
postgres=#
也就是说关闭之后，postgres的统计信息就会出现问题7、postgres配置文件中对应的默认参数
#------------------------------------------------------------------------------
# AUTOVACUUM PARAMETERS
#------------------------------------------------------------------------------
#autovacuum = on # Enable autovacuum subprocess? 'on'
# requires track_counts to also be on.
#log_autovacuum_min_duration = -1 # -1 disables, 0 logs all actions and
# their durations, > 0 logs only
# actions running at least this number
# of milliseconds.
#autovacuum_max_workers = 3 # max number of autovacuum subprocesses
# (change requires restart)
#autovacuum_naptime = 1min # time between autovacuum runs
#autovacuum_vacuum_threshold = 50 # min number of row updates before
# vacuum
#autovacuum_analyze_threshold = 50 # min number of row updates before
# analyze
#autovacuum_vacuum_scale_factor = 0.2 # fraction of table size before vacuum
#autovacuum_analyze_scale_factor = 0.1 # fraction of table size before analyze
#autovacuum_freeze_max_age = 200000000 # maximum XID age before forced vacuum
# (change requires restart)
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
#autovacuum_vacuum_cost_delay = 20ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
#autovacuum_vacuum_cost_limit = -1 # default vacuum cost limit for
# autovacuum, -1 means use
# vacuum_cost_limit
#------------------------------------------------------------------------------
8、源码中关于什么时候会触发自动收集统计信息和vacuum
[postgres@QXY1 postmaster]$ ls -ltr *auto*
-rw-r--r--. 1 postgres dba 100357 Feb 27 06:10 autovacuum.c
-rw-r--r--. 1 postgres dba 46248 Mar 21 07:40 autovacuum.o
[postgres@QXY1 postmaster]$ vi autovacuum.c
static void relation_needs_vacanalyze(Oid relid, AutoVacOpts *relopts,
Form_pg_class classForm,
PgStat_StatTabEntry *tabentry,
int effective_multixact_freeze_max_age,
bool *dovacuum, bool *doanalyze, bool *wraparound);
* relation_needs_vacanalyze
*
* Check whether a relation needs to be vacuumed or analyzed; return each into
* "dovacuum" and "doanalyze", respectively. Also return whether the vacuum is
* being forced because of Xid or multixact wraparound.
*
* relopts is a pointer to the AutoVacOpts options (either for itself in the
* case of a plain table, or for either itself or its parent table in the case
* of a TOAST table), NULL if none; tabentry is the pgstats entry, which can be
* NULL.
*
* A table needs to be vacuumed if the number of dead tuples exceeds a
* threshold. This threshold is calculated as
*
* threshold = vac_base_thresh + vac_scale_factor * reltuples
*
* For analyze, the analysis done is that the number of tuples inserted,
* deleted and updated since the last analyze exceeds a threshold calculated
* in the same fashion as above. Note that the collector actually stores
* the number of tuples (both live and dead) that there were as of the last
* analyze. This is asymmetric to the VACUUM case.
*
* We also force vacuum if the table's relfrozenxid is more than freeze_max_age
* transactions back, and if its relminmxid is more than
* multixact_freeze_max_age multixacts back.
*
* A table whose autovacuum_enabled option is false is
* automatically skipped (unless we have to vacuum it due to freeze_max_age).
* Thus autovacuum can be disabled for specific tables. Also, when the stats
* collector does not have data about a table, it will be skipped.
*
* A table whose vac_base_thresh value is < 0 takes the base value from the
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
* value < 0 is substituted with the value of
* autovacuum_vacuum_scale_factor GUC variable. Ditto for analyze.
*/
* threshold = vac_base_thresh + vac_scale_factor * reltuples
if (PointerIsValid(tabentry))
{
reltuples = classForm->reltuples;
vactuples = tabentry->n_dead_tuples;
anltuples = tabentry->changes_since_analyze;
vacthresh = (float4) vac_base_thresh + vac_scale_factor * reltuples;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
/*
* Note that we don't need to take special consideration for stat
* reset, because if that happens, the last vacuum and analyze counts
* will be reset too.
*/
elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
NameStr(classForm->relname),
vactuples, vacthresh, anltuples, anlthresh);
/* Determine if this table needs vacuum or analyze. */
*dovacuum = force_vacuum || (vactuples > vacthresh);
*doanalyze = (anltuples > anlthresh);
}通过源码可以看到，收集的公式为 * threshold = vac_base_thresh + vac_scale_factor * reltuples
需要满足的条件为 ：
*dovacuum = force_vacuum || (vactuples > vacthresh);
*doanalyze = (anltuples > anlthresh);
其中vactuples = tabentry->n_dead_tuples;
vacthresh = (float4) vac_base_thresh + vac_scale_factor * reltuples;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
又根据postgres.conf参数文件中的配置信息：
#autovacuum_vacuum_threshold = 50 # min number of row updates before # vacuum
#autovacuum_analyze_threshold = 50 # min number of row updates before # analyze
#autovacuum_vacuum_scale_factor = 0.2 # fraction of table size before vacuum
#autovacuum_analyze_scale_factor = 0.1 # fraction of table size before analyze
所以： vacthresh = 50+0.2*tuples
anlthresh = 50+0.1*tuples
根据当前的配置参数可以指定，anlthresh是肯定小于vacthresh的，所以也就是说 先触发自动统计信息收集，然后再满足一定条件之后才会触发vacuum的。
假如一张表有1000行记录，那么anlthresh = 50+0.1*1000 = 150，再根据*doanalyze = (anltuples > anlthresh);
所以改变的数据量要大于150才会做自动统计信息收集，也就是说至少删除151行数据才会触发自动收集统计信息。自动收集统计信息发生之后，vacuum对应的tuples的 数据就为1000-151=849，所以 vacthresh = 50+0.2*849 = 219.8。也就是说至少删除220行数据才会触发vacuum。简单的说就是先触发自动统计信息收集，然后用剩余的行数再计算vacthresh的值从而得到最终触发vacuum的行数，下面用实验证明：
reltuples = 代表被修改表的记录数
postgres=# drop table test;
DROP TABLE
postgres=# create table test(id int, name text);
CREATE TABLE
postgres=# insert into test select generate_series(1,1000),md5(chr((random()*25)::int+20567));
INSERT 0 1000
postgres=# select count(*) from test;
count
-------
1000
(1 row)
postgres=# \x
Expanded display is off.
postgres=# select relname, reltuples from pg_class where relname = 'test';
relname | reltuples
---------+-----------
test | 1000 <=====这里的tuple就是1000
(1 row)
所以* threshold = vac_base_thresh + vac_scale_factor * reltuples
threshold = 50+0.1*1000 = 150
postgres=# \x
Expanded display is on.
postgres=# select *,now() from pg_stat_user_tables where relname = 'test';
-[ RECORD 1 ]-----+------------------------------
relid | 16428
schemaname | public
relname | test
seq_scan | 1
seq_tup_read | 1000
idx_scan |
idx_tup_fetch |
n_tup_ins | 1000
n_tup_upd | 0
n_tup_del | 0
n_tup_hot_upd | 0
n_live_tup | 1000
n_dead_tup | 0
last_vacuum |
last_autovacuum |
last_analyze |
last_autoanalyze | 2018-07-21 19:42:38.503033-04
vacuum_count | 0
autovacuum_count | 0
analyze_count | 0
autoanalyze_count | 1
now | 2018-07-21 19:47:32.815724-04
postgres=#
首先删除50行记录
postgres=# show autovacuum_naptime; <=====这个参数是postgres检查数据的频率
autovacuum_naptime
--------------------
3s
(1 row)
postgres=# delete from test where id <=50;
DELETE 50
postgres=# select *,now() from pg_stat_user_tables where relname = 'test';
-[ RECORD 1 ]-----+------------------------------
relid | 16428
schemaname | public
relname | test
seq_scan | 2
seq_tup_read | 2000
idx_scan |
idx_tup_fetch |
n_tup_ins | 1000
n_tup_upd | 0
n_tup_del | 50
n_tup_hot_upd | 0
n_live_tup | 950
n_dead_tup | 50 <=======50行记录已经删除(注意，这个时候dead记录还存在，也就是说虽然数据删除了，但是占用的空间还没有释放)
last_vacuum |
last_autovacuum |
last_analyze |
last_autoanalyze | 2018-07-21 19:42:38.503033-04 <======该时间是插入记录的时间，删除50行数据之后，该时间还没有刷新，当前时间是now()对应的2018-07-21 19:51:37.788074-04
vacuum_count | 0
autovacuum_count | 0
analyze_count | 0
autoanalyze_count | 1
now | 2018-07-21 19:51:37.788074-04
postgres=#
<====再删除100行
postgres=# delete from test where id <=150;
DELETE 100
postgres=# select *,now() from pg_stat_user_tables where relname = 'test';
-[ RECORD 1 ]-----+------------------------------
relid | 16428
schemaname | public
relname | test
seq_scan | 3
seq_tup_read | 2950
idx_scan |
idx_tup_fetch |
n_tup_ins | 1000
n_tup_upd | 0
n_tup_del | 150
n_tup_hot_upd | 0
n_live_tup | 850
n_dead_tup | 150 <=====一共删除了150行 （dead记录还在，占用空间还没有释放）
last_vacuum |
last_autovacuum |
last_analyze |
last_autoanalyze | 2018-07-21 19:42:38.503033-04 <=====时间还是没有变（还是插入数据时的时间）
vacuum_count | 0
autovacuum_count | 0
analyze_count | 0
autoanalyze_count | 1 <=====自动分析的个数为1
now | 2018-07-21 19:55:17.436719-04
postgres=#
<=====再多删除一行
postgres=# delete from test where id <=151;
DELETE 1
postgres=# select *,now() from pg_stat_user_tables where relname = 'test';
-[ RECORD 1 ]-----+------------------------------
relid | 16428
schemaname | public
relname | test
seq_scan | 4
seq_tup_read | 3800
idx_scan |
idx_tup_fetch |
n_tup_ins | 1000
n_tup_upd | 0
n_tup_del | 151
n_tup_hot_upd | 0
n_live_tup | 849
n_dead_tup | 151 <=====一共删除了151行，根据公式已经大于了150，所以这个时候应该做自动收集统计信息了
last_vacuum |
last_autovacuum |
last_analyze |
last_autoanalyze | 2018-07-21 19:56:18.836931-04 <=====时间已经变成了，说明做了自动收集统计信息
vacuum_count | 0
autovacuum_count | 0
analyze_count | 0
autoanalyze_count | 2 <======收集统计信息的个数也加了1
now | 2018-07-21 19:56:21.797522-04
postgres=#
什么时候触发autovacuum呢
只有做过一次auto分析，tuples就要重新计算，所以这个时候的tuples个数就是849
if (PointerIsValid(tabentry))
{
reltuples = classForm->reltuples;
vactuples = tabentry->n_dead_tuples;
anltuples = tabentry->changes_since_analyze;
vacthresh = (float4) vac_base_thresh + vac_scale_factor * reltuples;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
/*
* Note that we don't need to take special consideration for stat
* reset, because if that happens, the last vacuum and analyze counts
* will be reset too.
*/
elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
NameStr(classForm->relname),
vactuples, vacthresh, anltuples, anlthresh);
/* Determine if this table needs vacuum or analyze. */
*dovacuum = force_vacuum || (vactuples > vacthresh);
*doanalyze = (anltuples > anlthresh);
}
if (PointerIsValid(tabentry))
{
reltuples = classForm->reltuples;
vactuples = tabentry->n_dead_tuples;
anltuples = tabentry->changes_since_analyze;
vacthresh = (float4) vac_base_thresh + vac_scale_factor * reltuples;
anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
/*
* Note that we don't need to take special consideration for stat
* reset, because if that happens, the last vacuum and analyze counts
* will be reset too.
*/
elog(DEBUG3, "%s: vac: %.0f (threshold %.0f), anl: %.0f (threshold %.0f)",
NameStr(classForm->relname),
vactuples, vacthresh, anltuples, anlthresh);
/* Determine if this table needs vacuum or analyze. */
*dovacuum = force_vacuum || (vactuples > vacthresh);
*doanalyze = (anltuples > anlthresh);
}
vacthresh = (float4) vac_base_thresh + vac_scale_factor * reltuples;
vacthresh = 50+0.2*849 = 219.8 也就是说需要删除大于219才会做autovacuum，下面看测试
删除219行数据
postgres=#
postgres=# delete from test where id <=219;
DELETE 68
postgres=# select * from pg_stat_user_tables where relname = 'test';
-[ RECORD 1 ]-----+------------------------------
relid | 16467
schemaname | public
relname | test
seq_scan | 2
seq_tup_read | 1849
idx_scan |
idx_tup_fetch |
n_tup_ins | 1000
n_tup_upd | 0
n_tup_del | 219
n_tup_hot_upd | 0
n_live_tup | 781
n_dead_tup | 219 <=====dead记录已经存在219，并且这个时候删除的记录还没有被清理
last_vacuum |
last_autovacuum |
last_analyze |
last_autoanalyze | 2018-07-21 19:56:18.836931-04
vacuum_count | 0
autovacuum_count | 0
analyze_count | 0
autoanalyze_count | 2
postgres=#
<=======在删除一行，220大于219.8，这个时候会触发自动vacuum。
postgres=#
postgres=# delete from test where id <=220;
DELETE 1
postgres=# select * from pg_stat_user_tables where relname = 'test';
-[ RECORD 1 ]-----+------------------------------
relid | 16467
schemaname | public
relname | test
seq_scan | 3
seq_tup_read | 2630
idx_scan |
idx_tup_fetch |
n_tup_ins | 1000
n_tup_upd | 0
n_tup_del | 220
n_tup_hot_upd | 0
n_live_tup | 780
n_dead_tup | 0 <======dead记录被清除，占用的空间也会自动释放
last_vacuum |
last_autovacuum | 2018-07-21 19:59:37.376797-04 <=======已经做了autovacuum
last_analyze |
last_autoanalyze | 2018-07-21 19:56:18.836931-04
vacuum_count | 0
autovacuum_count | 1 <=====autovacuum记录个数加1
analyze_count | 0
autoanalyze_count | 2
postgres=#
上面简单测试了什么情况下会自动触发统计信息收集和vacuum，自动收集的参数一般都是采用默认值，也可以根据数据库的具体情况修改指定的参数。