PostgreSQL configuration optimization

PostgreSQL configuration optimization

• PostgreSQL configuration optimization
  • • Hardware and system configuration
    • test tools
    • Configuration file
    • Main options
    • Test Data
    • to sum up

Hardware and system configuration

operating system	Ubuntu13.04
System bits	64
CPU	Intel(R) Core(TM)2 Duo CPU
RAM	4G
hard disk	Seagate ST2000DM001-1CH164
test tools	PostgreSQL-9.1.11

test tools

Tool name	pgbench
The amount of data	200W (the size of the entire database is about 300M)
Number of simulated clients	4
Threads	4
testing time	60 seconds

• Preparation command: pgbench -i -s 20 pgbenchdb
• Test command: pgbench -r -j4 -c4 -T60 testdb

Configuration file

The default configuration configuration file is the postgresql.conf file stored in the /etc/postgresql/VERSION/main directory

• If you want to check whether the parameter modification takes effect, you can use psql to connect to the database and use <show option name> to check.
• If you want to modify shared_buffers, you may need to execute the command <sysctl -w>Managing Kernel Resources under ubuntu

Main options

Options	Defaults	Description	Whether to optimize	the reason
max_connections	100	Maximum number of client connections allowed	no	Because during the test, 100 connections are enough
fsync	on	Force the data to be updated to disk synchronously	Yes	Because the IO pressure of the system is very high, in order to better test the impact of other configurations, change the parameter to off
shared_buffers	24MB	Decide how much memory can be used by PostgreSQL to cache data (1/4 of recommended memory)	Yes	In the case of high IO pressure, increasing this value can reduce IO
work_mem	1MB	Make internal sorting and some complex queries completed in this buffer	Yes	Helps increase the speed of operations such as sorting and reduce IO
effective_cache_size	128MB	The optimizer assumes that the maximum memory that a query can use has nothing to do with shared_buffers (1/2 of the recommended memory)	Yes	The setting is slightly larger, the optimizer prefers to use index scan instead of sequential scan
maintenance_work_mem	16MB	The memory defined here is only used when called by VACUUM and other resource-intensive commands	Yes	Increase the value to speed up the execution of the command
wal_buffer	768kB	The size of the log buffer	Yes	Can reduce IO, if there are more concurrent short transactions, it should be used together with commit_delay
checkpoint_segments	3	Set the maximum number of wal logs (the size of a log is 16M)	Yes	The default 48M cache is a serious bottleneck, basically it must be set to 10 or more
checkpoint_completion_target	0.5	Indicates that the completion time of checkpoint should be completed within N% of the interval between two checkpoints	Yes	Can reduce average write overhead
commit_delay	0	After the transaction is committed, the time interval from when the log is written to the wal log to when the wal_buffer is written to the disk. Need to cooperate with commit_sibling	Yes	Ability to write multiple transactions at once, reduce IO and improve performance
commit_siblings	5	Set the number of concurrent transactions that trigger commit_delay, configure according to the number of concurrent transactions	Yes	Reduce IO, improve performance

Test Data

• The test data is run 3 times and averaged.
• Turning off fsync is to better reflect the impact of other parameters on PostgreSQL.

parameter	Modify value	Total number of transactions	tps (including connection establishment)	tps (not including connection establishment)
default setting		8464	140.999792	141.016182
fsync	off	92571	1479.969755	1480.163355
shared_buffers	1GB	100055	1635.759275	1635.977823
work_mem	10MB	101209	1665.804812	1666.04082
effective_cache_size	2GB	98209	1636.733152	1636.970271
maintenance_work_mem	512MB	92930	1548.029233	1548.223108
checkpoint_segments	32	195982	3265.995	3266.471064
checkpoint_completion_target	0.9	194390	3239.406493	3239.842596
wal_buffer	8MB	198639	3310.241458	3310.724067
Restore fsync	off	11157	185.883542	185.909849
commit_delay && commit_siblings	10 && 4	11229	187.103538	187.131747

总结

	事务总数	tps(包括建立连接)	tps(不包括建立连接)
优化前	8464	140.999792	141.016182
优化后(fsync=on)	11229	187.103538	187.131747
优化后(fsync=off)	198639	3310.241458	3310.724067

在fsync打开的情况下，优化后性能能够提升30%左右。因为有部分优化选项在默认的SQL测试语句中没有体现出它的优势，如果到实际测试中，提升应该不止30%。
测试的过程中，主要的瓶颈就在系统的IO，如果需要减少IO的负荷，最直接的方法就是把fsync关闭，但是这样就会在掉电的情况下，可能会丢失部分数据。