What we are doing performance testing aims to, is where you want to measure a system bottleneck, in the end is where affect the performance of our system, find the problem and then solve it. Of course, a system consists of a combination of many things together together, applications, databases, servers, middleware, etc. in a lot. That time we tested above these things there might be any link problems, may affect the performance of our system. This blog is mainly about the next mysql database performance testing we are doing what should be monitored when, where and what needs to be optimized.
What things will affect the performance of mysql?
1. Hardware
2. System configuration
3. Database table structure
4.SQL and index
hardware
It refers to the hardware configuration of the database server, the server that white is only one computer, if the computer's configuration is high, strong cpu processing power, large memory, the hard disk is ssd, and that is certainly a good performance. Of course, this is also the highest cost way to spend money thing.
System Configuration
Configuring a system refers to the operating system's configuration, there are some operating system configurations can affect the performance of mysql, linux server now most of our servers are used, linux above everything is file-based, data mysql table inside , data and so are the file exists on disk.
linux system has a system configuration is the number of open files, the default is 1024, which is a maximum of 1024 open file , that there are more tables in a database, large concurrent case, the 1024 is not enough, in order to obtain after the data have to open the file, but the number of open files on up to 1024, will lead to some data acquisition less, you have to wait for another file is closed, to open. It would have to modify the configuration of the system, in /etc/security/limits.conf can modify the maximum number of open files inside.
There are some mysql configuration parameters can affect the performance of mysql.
sleep timeout
number of connections mysql is configured in advance, if the code is badly written program which, under some database operations did not close the database, and that this link will not be released, will always occupy a link like this concurrent large, it It causes database connections are not enough , you can not connect the database. mysql default eight hours does not operate the database will automatically close links, so the sleep timeout will affect the performance of mysql.
Set up a separate table space
The number of read / write process configuration
After reading mysql5.5, the number of write process is configurable. The default number of read and write processes are four .
Of course, we all know that many hands make work thing. The process is more than many people work, the specific configuration according to the configuration of these two values and the number of cpu core business logic.
If the cpu core is 32, then that while there are 32 processes running, you can these two values to transfer large.
If the system is said to be a type of content sites, most of the operations are read, then you can read the number of processes is set larger, the process of writing the number set smaller.
How to modify it, to find the mysql configuration file, in the [mysqld] node add the following parameters to the
Cache configuration
Before saying we should first understand clearly cache configuration, the computer at the time of processing tasks is how to deal with, and read data on disk to start, and then put inside memory, cpu memory to go inside to get the data, and then processed .
Just the opposite in writing, after cpu processed, the data into memory inside, then put the disk in memory .
That from the above, we find that, if the data directly from memory, then take the inside, and that speed is much faster, and we look at the chart below, read data 1M, and how much memory there faster than from disk.
From the above we find that this figure which is read from the memory N times faster than the data fetch data from the disk inside.
That mysql to which, if the time of access to data, mysql put some data into the cache memory inside, take data directly from memory which does not take on a lot faster .
Let's say before mysql cache, at first said the implementation of a mysql query when they are done.
From the above chart we find, mysql there are two places to check the memory. If the memory inside to find the data we want, then do not go on to query the data of the disk . So what are these two caches, how to configure it.
qcache Configuration
Cache complete SELECT statements and query results, when a query cache hit, MySQL will immediately return the results, skip parsing, optimization and implementation phases.
Query caching system keeps track of each table, if the table changes, all cached queries related to this table and all fail.
When checking the query cache, MySQL will not be any SQL processing, it uses the exact query sent by the client (select), as long as the character case, or comment a little bit different, the query cache is considered to be a different query .
Any query that contains undefined function (such as now (), current_date ()) will not be cached.
MySQL query cache can improve performance, but when in use, there are some issues that need attention:
open query cache for read and write all adds additional overhead. For reading, before the beginning of the query you need to check the cache; for writing, after writing needs to update the cache.
Typically these costs are relatively small, so the query cache is generally still good. But also according to business characteristics weigh whether to enable the query cache.
How to configure it, to find the mysql configuration file, under the [mysqld] node add the following parameters can be
innodb_buffer_pool Configuration
mysql there is also a cache configuration is innodb_buffer_pool configuration, innodb mysql is now the default storage engine, the storage engine mysql when it plainly on the data stored in the end is how stored.
How it is placed inside a warehouse of goods.
buffer pool cache is a pool with innodb storage engine, query data when it first query from memory, if memory exists, return directly to improve query response time.
qcache difference innodb buffer pool and are: qcacche sql statement cache is set corresponding to the result, buffer pool is cached data in the table. buffer pool server is generally set to 70% of physical memory.
How to configure it, to find the mysql configuration file, in the [mysqld] node add the following parameters to the
Optimization of mysql architecture
Separate read and write
Multi-written
Database table structure optimization
Of course, in the design of the system when the table structure, usually a bunch of architects and developers have put a good table structure design , something we did not reach the level of architecture that we do not know, in the design table structure when the need to pay attention something.
mysql Index Tuning
What is it indexes, dictionaries and directories on the same. There directory, and then we check the data on the fast.
Most suitable for building the index column that appears in the back row where clause.
The only index the best effect, because it is unique.
Using the most left-prefix.
The index is not better.
There are four types of indexes mysql
1, the general index
The most common index, all columns can be added
create index index_name on table_name (col);
2, the primary key index
When construction of the table plus the primary key
3, the combination index
create index index_name on table_name (col,col2);
4, a unique index
CREATE UNIQUE INDEX index_name ON table_name (column_name);
De-duplication, redundancy index
Because the level of development of each is different, inevitably there will be some problems repeat index. How to find that we have some redundancy in the index yet.
Percona-toolkit is necessary to use this tool, it contains pt-duplicate-key-checker This tool can help find out what we have inside the index table redundancy, and gives modify the index statement.
Pt - Duplicate - Key - Checker - -uroot - Pxxx - DXX # - U refers to the user - P is the password - D database
The help we find out duplicate index, there are some that simply do not need to use the index, although indexing is not repeated, but in fact did not use it with the query, how to do it, percona-toolkit this there is a tool in the tool pt-index-usage, it can read the slow query log, we help find those useless index.
pt-index-usage /opt/data/slow.log #后面是慢查询日志
慢查询日志
什么是慢查询日志呢，它这个就是个神器了，对咱们测试特别有帮助，它会记录执行时间长的sql语句，这样咱们找问题的时候就比较方便了。
SET Global slow_query_log = ON ; # open slow query log SET Global long_query_time = . 1 ; # Set record how long the query than SQL SET Global slow_query_log_file = ' /tmp/slow_query.log ' ; Set # mysql slow query log path, which need to have write access to the sET , Ltd. Free Join log_queries_not_using_indexes = ON ; # setting does not use the index sql recorded SHOW VARIABLES the LIKE ' % sLOW% ' ; # View slow query configuration
Mysql log records inside, we looked less clear, we use pt-query-digest this tool to help resolve our slow query log, it will perform all the time and specific sql sql, how many times have helped execute our statistics come out.
Here is the usage of pt-query-digest
Pt - Query - Digest - filter = '$ Event -> Fingerprint {m} = ~ / ^ select / I' slow.log # view slow query select statements Pt - Query - Digest - Operating since slow.log = 12h # last 12 hours Pt - Query - Digest - Operating since '2017-12-01 09:30:00' --until '2017-12-02 10:00:00' --filter = '$ Event -> { fingerprint} = ~ m / ^ select / i 'slow.log # specified time period
If you want to get real-time has not been implemented a long time sql, use the following sql statement
select id,`user`,`host`,DB,command,`time`,state,info from information_schema.PROCESSLIST where TIME>=60;
explain
By slow query log we can find the sql statement in question, that how we look at how this sql problem then, we should explain the use, as long as you want to perform in front of the sql statement with explain to
all <index <range <ref <eq_ref <const, system sql execution type column in the worst to best
Note that when sql optimization
Query by using indexed columns, sort columns using the index
Avoid select *, select * usually will result in a full table scan
Avoid sub-queries, MySQL sub-query execution plan there has been a major problem, although this problem has existed for years, but so far all have been released stable version is prevalent, has not improved much. Although the official also acknowledged the problem early on, and commitment to resolve as soon as possible, but at least so far we have not seen what a version better solution to this problem.
thing
Examples of the bank to save money.
lock
Table-level locking, row-level locking.
SELECT * FROM information_schema.INNODB_TRX\G
mysql performance testing tool
mysqlslap is mysql comes with a performance testing tool. It can simulate a variety of concurrency, and which use sql, how much data, generate long run, produce reports.
Common options
- Concurrency number of concurrent, the plurality may be separated by commas - Engine engines to be tested, there can be multiple, separated by delimiters, such as MyISAM = --engines, InnoDB - Auto-Generate-SQL with the system itself the generated SQL script to test - Auto-the generate SQL-to-the Load-testing of the type of read or write, or a combination of both (the read, the write, Update, mixed) - number the total-of-Queries how many times you want to run Inquire. The number of queries each customer can run with the total number of queries / concurrent calculated - Debug-info extra output from the CPU and memory of relevant information - the number of type int field number-int-cols to create a test table - Number The-char- the number of chat-type field cols to create a test table - the create Schema-test Database - Query own SQL script execution test - only-Print If you want to print to see what SQL statements, you can use this option
The following are examples of the use of
100 concurrent operation 1000, the write and read operations are automatically generated sql, int type field 2, char type 10, mysqlslap - . H127 0.0 . . 1 - -uroot - p123456 - Concurrency = 100 --auto- generate-sql --auto-generate-sql -load-type = mixed --engine = innodb --auto-generate-sql-add-autoincrement --number-int-cols = 2 --number-char-cols = 10 Queries =-of---number 10 100 concurrent operation 5000, besttest performed on this database SQL mysqlslap - H127. 0.0 . . 1 - -uroot - p123456 - concurrency --query = 100 = 'SELECT * from STU;' - create-schema = besttest --engine = innodb --number-of-queries = 5000 --debug-info 100 concurrent operation 5000, besttest this database to perform the specified file sql mysqlslap - . H127 0.0 . . 1 - -uroot - p123456 - Concurrency --query = 100 = / tmp / besttest.sql -create-Schema = besttest - engine = innodb --number-of-queries = 5000 --debug-info