Background Description: Friends OA system unit recently upgraded makeover, MySQL backend data storage used, the line ran a month, began to complain about the phone call after another, is not open here, there is no response, it was a metaphor upgrade after the classic cars become more and more slowly, the problem is urgent, it must immediately wanted to countermeasures. Since the deployment of the use of specification documents, before the line also did various tests, and thus, the online investigation, unsuccessful, pulls out the implementation of the document, one by one reading, unsuccessful, then remembered once a business system, also encountered a similar situation, and later through the kind of optimization to ease, there was the following, "MySQL optimization core theory and practice."
Description: This theoretical part of the source of the teacher leaves Bowen, the practical part of the source of work accumulation and numerous love MySQL technology sharing friends, organize original intention was to more fully understand the MySQL optimization, master the technology for more MySQL optimization, enhance their own, back love technology shared by all users.
Optimized hardware layer of
newly purchased servers running in the default energy-saving mode, the concurrent access to a large amount of business scenarios will lead to database performance can not keep up, resulting in a large number of delays, will eventually wear down business systems. At the same time, select the disk array card set incorrectly also make database performance becomes a bottleneck entire business system.
A goal: a comprehensive closed-saving mode, so MySQL running in high performance mode
1. Turn off the CPU power-saving mode
Find OPI Link Speed Select Options, select Performance Max
2. Close the memory-saving mode
Find Memory Speed option, select Max Performance
Found Power C-States option, select Disable
to find C1 Enhanced Mode option, select Disable
goal two: Turn off NUMA, CPU can always make efficient use of memory
closed NUMA
Find Socket Interleave option, select Non-NUMA
Goal Three: to enhance IOPS performance, so that disk I / O is not back again and again
when sufficient funds 1. Procurement even SSD PCIe-SSD
SSD and PCIe-SSD brings not only a surprise, more at ease, this disk I / O is no longer a demon
2. Mechanical disk arrays with a card, Cache strategy, BBU battery, RAID-10,15KRPM
Array card easily face a plurality of mechanical disc, the BBU Cache Strategies battery does not lose data protection mode High Performance
Cache Strategies Write Back Write Back even Always
the Read-ahead array of the Policy option, choose Normal
use RAID-10, higher performance RAID-5
using 15KRPM high-speed disk, disk performance than 7.2KRPM
Note: server hardware setup parameters are derived from IBM X3650M3
Optimizing system-level
operating system, there are also many places worth optimization, the same can significantly enhance IOPS performance. In addition, SWAP to use less, not only can not help, but will make business systems verge of collapse.
A goal: to enhance IOPS performance, so that the database no longer scapegoat
1. reasonable allocation of I / O scheduler
Mechanical disk with deadline, execute the command echo deadline> / sys / block / sda / queue / scheduler
SSD with noop, execute the command echo noop> / sys / block / sda / queue / scheduler
Note that sda is a data file where the partition
2. File system to make use of XFS, if still using ext4, just hope the transition phase
3.mount parameter increases noatime, nodiratime, nobarrier
VI / etc / fstab
/ dev / sda1 / Data XFS Defaults, the noatime, nodiratime, nobarrier 0 0
/ dev / sda2 / XFS Defaults, the noatime, nodiratime, nobarrier 0 0
/ dev / sda3 the swap the swap Defaults, the noatime, nodiratime, nobarrier 0 0
Mount -o remount / the Data
Mount
goal II: the tendency to reduce or even ban the use of sWAP, stable disk I / O and network to reduce the waiting time for the MySQL performance is more stable
1.vm.swappiness set to 0 or even 5, if you do not care occur OOM
echo 'vm.swappiness = 5' >> / etc / the sysctl.conf
/ sbin / sysctl -p
2.vm.dirty_background_ratio set to 5, vm.dirty_ratio set to 10, let the dirty pages to disk brush continued to avoid disk I / O write instantly generate TIME_WAIT
echo '= vm.dirty_background_ration. 5' >> / etc / the sysctl.conf
echo 'vm.dirty_ratio = 10' >> / etc / the sysctl.conf
/ sbin / sysctl -p
3.net.ipv4.tcp_tw_recycle and net.ipv4. tcp_tw_reuse set to 1 double, reduce network latency and improve efficiency
echo '= net.ipv4.tcp_tw_recycle. 1' >> / etc / the sysctl.conf
echo '= net.ipv4.tcp_tw_reuse. 1' >> / etc / the sysctl.conf
/ sbin / sysctl -p
optimized MySQL layer
selected from MySQL version of particularly important, find the version for business systems in order to play a greater performance. Operating parameters is also true, as appropriate, and the need to repeatedly adjust. Sql schema design and specification writing, as well as operation and maintenance management process after the line specifications, the same will bring no small gains.
Goal One: choose the right version, let MySQL start unashamedly
1. The first priority Oracle MySQL, more and more new systems embracing the official version 5.7.x
2. Second recommend Percona branch version Here you can enjoy free thread pool and audit plugin
3. Finally, MariaDB branch version, in addition to the thread pool and audit plug-in, where to enjoy a free black &
Goal II: adjust the appropriate parameters, so MySQL performance more stable
1. If you choose to use Percona or MariaDB branch version, it is strongly recommended to open thread pool
2. Set the default-storage-engine = innodb, innodb meet more than 99% of the business scenario
3. Set innodb_buffer_pool_size suitable size, mostly single instance innodb table, set a 50% -70% of physical memory
4. Set the appropriate value innodb_flush_log_at_trx_commit and sync_binlog
A double set, data is not lost, lower performance
settings 2 and 10, that data is lost, the performance of the general
set-bis 0, data is not ××× full, highest performance
5. Set innodb_file_per_table = 1, using a separate tablespaces
6. Set innodb_data_file_path = ibdata1: 1G: autoextend, good performance at high concurrent transaction
7.设置innodb_log_file_size=256M，innodb_log_files_in_group=2
8. Set long_query_time = 0.05, recording more than 50 milliseconds slow SQL
9. appropriate transfer large max_connection, max_connection_error recommended set to 100,000 or more, provided open_files_limit, innodb_open_files, table_open_cache, table_definition_cache about 10 times max_connection
10.不宜设置过大的参数tmp_table_size、max_heap_table_size、sort_buffer_size、join_buffer_size、read_buffer_size、read_rnd_buffer_size
11. Set key_buffer_size = 32M, query cache off function
Close QC needs to be configured before starting MySQL
query_cache_type = 0
query_cache_size = 0
Goal Three: Schema design and SQL prepared in accordance with specifications set reference, help to improve the efficiency of MySQL
1. All innodb tables are designed to a non-business use of auto-increment do primary key
2. The field type is satisfied enough, small length as much as possible; field attributes are plus possible constraint NOT NULL
3. far as possible without TEXT and BLOB field types, certain sub-list if necessary to split
4. query, try to fill in the required columns, all the columns do not inquire, to avoid serious problems random read
5. General varchar (n) indexing the column, the top 50% of the length can be taken
6. low performance sub-query processing, suggested a change rewritten using SQL JOIN
7. multi-table join queries, keyword type as consistent as possible, and must have an index
8. When the multi-table join query, the results filtered set smaller table as the driving table
Advantages: no data does not appear, small SQL query range, high efficiency
9. multi-table join query and when there is, the sort field must be driven table, otherwise do not take the sort column index
The multi-composite index, a plurality of separate index less, especially small column base is not created index recommendations
11. When you use SQL paging function, choose the keywords and do comply with the primary key index, again executed, efficiency would be much higher
Goal Four: management and maintenance optimization, operation and maintenance more efficient
1.online DDL too costly, time enough machine performance, we recommend no more than a single physical table 10G, the number of single-table row is not more than 100 million, the average length of the line does not exceed 8KB
2. OOM KILL and extensive use of SWAP does not appear, do not worry MySQL process consumes too much memory
3. single instance running in hardware resources is still relatively tight, do not run multiple instance
4. Periodically pt-duplicate-key-checker to check and remove duplicate index, the index periodically pt-index-usage with checking and removing the less
The slow query log collected periodically, and analyzed by pt-query-digest tool, combined Anemometer slow query management systems and the like, for analysis and optimization
6. You can use pt-kill to kill the super-long SQL requests, Percona version there is an option innodb_kill_idle_transaction can achieve this function
7 may be used pt-online-schema-change table to accomplish the needs ONLINE DDL
8. Regular use pt-table-checksum, pt-table-sync mysql to investigate and fix the main data copied from the difference
Core Program: Before the line, change any of the parameters, stress tests are done to avoid slip through the net led to the emergence of various MySQL CRASH.
At the end of write: late at plans every detail theoretical analysis and stress testing, finishing writing for the first time, there may be imperfections, welcome messages and communication.
Strength is strongly recommended that two teachers: Ye Jinrong and Wubing Xi
Original link:
more comprehensive MySQL Optimization Reference (Part I)
more comprehensive MySQL Optimization Reference (Part II)
Published: July 17, 2019