A very shallow understanding of mysql tuning steps

mpstat, iostat, sar and vmstat to view the performance status of the system

Open the slow query log
show variables like'slow_query%';
±--------------------±---------------- ------------------+
| Variable_name | Value |
±--------------------±---- ------------------------------+
| slow_query_log | OFF | Check whether the slow query log is turned on
| slow_query_log_file | /var/lib/ mysql/localhost-slow.log | log location
show variables like'long %';
±----------------±----------+
| Variable_name | Value |
±----------------±----------+
| long_query_time | 10.000000 | sql timeout time
±---------- ------±----------+
set global slow_query_log = ON;
set long_query_time = 5;

more localhost-slow.log View log query timeout sql

Explain query sql execution plan for timeout sql | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |filtered | Extra
±—±---------- --±-----------±-----------±-----±--------------±-- ---±--------±-----±-----±---------±------±-------- ---------
| 1 | SIMPLE | stock_info | NULL | ALL | NULL | NULL | NULL | NULL | 143 | 100.00 | NULL |100.00 | NULL
3. profiling analysis query query sql resource utilization
select @@ profiling;
profiling is off by default
show profiles\G

Ø Ordinary index: This is the most basic index type, there are no restrictions such as uniqueness.

Ø Unique index: Basically the same as a normal index, but all index column values remain unique.

Ø Primary key: The primary key is a unique index, but it must be designated as "PRIMARY KEY".

Ø Full-text indexing: MYSQL supports full-text indexing and full-text retrieval starting from 3.23.23. In MYSQL, the index type of full-text index is FULLTEXT. Full-text indexes can be created on VARCHAR or TEXT type columns

This part is about some trivial suggestions and precautions that should be paid attention to when writing SQL statements.

Use LIMIT 1 when the result set has only one row of data
Avoid SELECT *, always specify the columns you need

The more data read from the table, the slower the query will become. He increased the time required to operate the disk, or in the case where the database server and the WEB server are independent. You will experience very long network delays simply because the data is transferred between servers unnecessarily.

Use join (JOIN) instead of sub-queries (Sub-Queries)

连接（JOIN）.. 之所以更有效率一些，是因为MySQL不需要在内存中创建临时表来完成这个逻辑上的需要两个步骤的查询工作。

Use ENUM, CHAR instead of VARCHAR, use reasonable field attribute length
Use NOT NULL as much as possible
Fixed-length tables will be faster
Split large DELETE or INSERT statements
The smaller the query column, the faster

Three, configuration optimization

After installing MySQL, the configuration file my.cnf is in the /MySQL installation directory/share/mysql directory. The directory also contains multiple configuration files for reference, including my-large.cnf, my-huge.cnf, and my-medium .cnf, my-small.cnf, respectively correspond to the configuration of large, medium and small database applications. The .ini file that exists in the MySQL installation directory under the win environment.

The main variables that have a greater impact on performance optimization are listed below, which are mainly divided into connection request variables and buffer variables.

Variables for connection request:

```
max_connections
```

The maximum number of MySQL connections. Increasing this value increases the number of file descriptors required by mysqld. If the number of concurrent connection requests of the server is relatively large, it is recommended to increase this value to increase the number of parallel connections. Of course, this is based on the situation that the machine can support, because if the number of connections is larger, MySQL will provide a connection for each connection The buffer, the more memory will be spent, so you must adjust the value appropriately, and you cannot blindly increase the setting.

If the value is too small, ERROR 1040: Too many connections will often occur. You can use the'conn%' wildcard to check the number of connections in the current state to determine the value of this value.

show variables like'max_connections' maximum number of connections

show status like'max_used_connections' response number of connections

as follows:

mysql> show variables like ‘max_connections‘;

+———————–+——-+

| Variable_name　| Value |

+———————–+——-+

| max_connections | 256　　|

+———————–+——-+

mysql> show status like ‘max%connections‘;

+———————–+——-+

| Variable_name　 | Value |

+—————————-+——-+

| max_used_connections | 256|

+—————————-+——-+

max_used_connections / max_connections * 100% (ideal value ≈ 85%)

If max_used_connections is the same as max_connections, then max_connections is set too low or exceeds the upper limit of server load. If it is less than 10%, it is set too large.

```
back_log
```

The number of connections that MySQL can temporarily store. This works when the main MySQL thread gets very many connection requests in a short period of time. If the MySQL connection data reaches max_connections, new requests will be stored in the stack to wait for a connection to release resources. The number of the stack is back_log. If the number of waiting connections exceeds back_log, connection resources will not be granted.

The back_log value indicates how many requests can be stored on the stack in a short period of time before MySQL temporarily stops answering new requests. Only if you expect to have many connections in a short period of time, you need to increase it, in other words, this value is the size of the listening queue for incoming TCP/IP connections.

The default value is 50, and the adjustable value is 128. For Linux systems, the setting range is an integer less than 512.

```
interactive_timeout
```

The number of seconds an interactive connection waits for action before being closed by the server. An interactive client is defined as a client that uses the CLIENT_INTERACTIVE option to mysql_real_connect().

The default value is 28800, and the adjustable value is 7200.

Buffer variable

Global buffer:

```
key_buffer_size
```

key_buffer_size specifies the size of the index buffer, which determines the speed of index processing, especially the speed of index reading. By checking the status values Key_read_requests and Key_reads, you can know whether the key_buffer_size setting is reasonable. The ratio of key_reads / key_read_requests should be as low as possible, at least 1:100, 1:1000 is better (the above status value can be obtained using SHOW STATUS LIKE'key_read%').

key_buffer_size only works for MyISAM tables. Even if you do not use MyISAM tables, but the internal temporary disk tables are MyISAM tables, you must use this value. You can use the check status value created_tmp_disk_tables to know the details.

Examples are as follows:

mysql> show variables like ‘key_buffer_size‘;

+——————-+————+

| Variable_name | Value |

+———————+————+

| key_buffer_size | 536870912 |

+———— ———-+————+

The key_buffer_size is 512MB, let's take a look at the usage of key_buffer_size:

mysql> show global status like ‘key_read%‘;

+————————+————-+

| Variable_name　 | Value |

+————————+————-+

| Key_read_requests| 27813678764 |

| Key_reads　　　| 6798830 |

+————————+————-+

There are a total of 27813678764 index read requests, and 6,798,830 requests are not found in the memory to read the index directly from the hard disk, and calculate the probability of index misses in the cache:

key_cache_miss_rate = Key_reads / Key_read_requests * 100%, it is better to set around 1/1000

The default configuration value is 8388600 (8M), the host has 4GB of memory, and the adjustable value is 268435456 (256MB).

```
query_cache_size
```

Using query buffering, MySQL stores the query results in a buffer. In the future, for the same SELECT statement (case sensitive), the results will be read directly from the buffer.

By checking the status value Qcache_*, you can know whether the query_cache_size setting is reasonable (the above status value can be obtained using SHOW STATUS LIKE'Qcache%'). If the value of Qcache_lowmem_prunes is very large, it indicates that there is often insufficient buffering. If the value of Qcache_hits is also very large, it indicates that the query buffer is used very frequently, and the buffer size needs to be increased at this time; if the value of Qcache_hits is not large, it indicates that your The query repetition rate is very low. In this case, the use of query buffering will affect efficiency, so you can consider not using query buffering. In addition, adding SQL_NO_CACHE to the SELECT statement can clearly indicate that the query buffer is not used.

Parameters related to query buffering include query_cache_type, query_cache_limit, and query_cache_min_res_unit.

query_cache_type specifies whether to use query buffer, which can be set to 0, 1, 2. This variable is a SESSION-level variable.

query_cache_limit specifies the buffer size that can be used by a single query, the default is 1M.

query_cache_min_res_unit was introduced after version 4.1. It specifies the smallest unit of buffer space allocated, and the default is 4K. Check the status value Qcache_free_blocks. If the value is very large, it indicates that there are many fragments in the buffer, which indicates that the query results are relatively small. At this time, query_cache_min_res_unit needs to be reduced.

Examples are as follows:

mysql> show global status like ‘qcache%‘;

+——————————-+—————–+

| Variable_name | Value　 |

+——————————-+—————–+

| Qcache_free_blocks　 | 22756　 |

| Qcache_free_memory　 | 76764704 |

| Qcache_hits　　　　　 | 213028692 |

| Qcache_inserts　　　　 | 208894227 |

| Qcache_lowmem_prunes | 4010916 |

| Qcache_not_cached　| 13385031 |

| Qcache_queries_in_cache | 43560　|

| Qcache_total_blocks | 111212　 |

+——————————-+—————–+

mysql> show variables like ‘query_cache%‘;

+————————————–+————–+

| Variable_name　　　　　 | Value　 |

+————————————–+———–+

| query_cache_limit　　　　　 | 2097152 |

| query_cache_min_res_unit　 | 4096　　 |

| query_cache_size　　　　　 | 203423744 |

| query_cache_type　　　　　 | ON　 |

| query_cache_wlock_invalidate | OFF　 |

+————————————–+—————+

Query cache fragmentation rate = Qcache_free_blocks / Qcache_total_blocks * 100%

If the query cache fragmentation rate exceeds 20%, you can use FLUSH QUERY CACHE to defragment the cache fragments, or try to reduce query_cache_min_res_unit, if your queries are all small data volumes.

Query cache utilization = (query_cache_size – Qcache_free_memory) / query_cache_size * 100%

If the query cache utilization rate is below 25%, it means that the query_cache_size setting is too large and can be appropriately reduced; if the query cache utilization rate is above 80% and Qcache_lowmem_prunes> 50, it means that the query_cache_size may be a bit small, or there are too many fragments.

Query cache hit rate = (Qcache_hits – Qcache_inserts) / Qcache_hits * 100%

Example server query cache fragmentation rate = 20.46%, query cache utilization rate = 62.26%, query cache hit rate = 1.94%, the hit rate is very poor, maybe write operations are frequent, and there may be some fragmentation.

Buffer per connection

record_buffer_size

Each thread that performs a sequential scan allocates a buffer of this size for each table it scans. If you do a lot of sequential scans, you may want to increase this value.

The default value is 131072 (128K), it can be changed to 16773120 (16M)

```
read_rnd_buffer_size
```

Random read buffer size. When the rows are read in any order (for example, in the sort order), a random read buffer will be allocated. When performing a sorting query, MySQL will scan the buffer first to avoid disk search and increase the query speed. If you need to sort a large amount of data, you can increase the value appropriately. But MySQL will allocate this buffer space for each client connection, so you should try to set this value appropriately to avoid excessive memory overhead.

Generally can be set to 16M

```
sort_buffer_size
```

Each thread that needs to be sorted allocates a buffer of this size. Increasing this value speeds up ORDER BY or GROUP BY operations.

The default value is 2097144 (2M), which can be changed to 16777208 (16M).

```
join_buffer_size
```

The buffer size that can be used by the joint query operation

record_buffer_size, read_rnd_buffer_size, sort_buffer_size, join_buffer_size are exclusive to each thread, that is to say, if there are 100 threads connected, the occupation is 16M*100

table_cache

The size of the table cache. Whenever MySQL accesses a table, if there is room in the table buffer, the table is opened and put into it, so that the table contents can be accessed faster. By checking the state values Open_tables and Opened_tables of the peak time, you can determine whether you need to increase the value of table_cache. If you find that open_tables is equal to table_cache, and opened_tables is increasing, then you need to increase the value of table_cache (the above status value can be obtained using SHOW STATUS LIKE'Open%tables'). Note that you cannot blindly set table_cache to a large value. If it is set too high, it may cause insufficient file descriptors, resulting in unstable performance or connection failure.

For 1G memory machines, the recommended value is 128-256. For servers with a memory of about 4GB, this parameter can be set to 256M or 384M.

max_heap_table_size

The size of the memory table that the user can create. This value is used to calculate the maximum row value of the memory table. This variable supports dynamic changes, namely set @max_heap_table_size=#

This variable and tmp_table_size together limit the size of the internal memory table. If the size of an internal heap (stacked) table exceeds tmp_table_size, MySQL can automatically change the heap table in memory to a hard disk-based MyISAM table as needed.

tmp_table_size

Increase the size of a temporary table by setting the tmp_table_size option, such as a temporary table generated by an advanced GROUP BY operation. If you increase the value, MySQL will increase the size of the heap table at the same time, which can achieve the effect of improving the connection query speed. It is recommended to optimize the query as much as possible. Make sure that the temporary table generated during the query process is in memory to avoid excessive temporary table generation. The MyISAM table of the hard disk.

mysql> show global status like ‘created_tmp%‘;

+——————————–+———+

| Variable_name　　 | Value　|

+———————————-+———+

| Created_tmp_disk_tables | 21197 |

| Created_tmp_files　　　| 58　　|

| Created_tmp_tables　　| 1771587 |

+——————————–+———–+

Every time a temporary table is created, Created_tmp_tables increases. If the size of the temporary table exceeds tmp_table_size, a temporary table is created on the disk. Created_tmp_disk_tables also increases. Created_tmp_files represents the number of temporary files created by the MySQL service. The ideal configuration is:

Created_tmp_disk_tables / Created_tmp_tables * 100% <= 25% For example, the above server Created_tmp_disk_tables / Created_tmp_tables * 100% =1.20%, it should be quite good

The default is 16M, and it is best to adjust to 64-256. The thread is exclusively used. If it is too large, the memory may not be enough.

thread_cache_size

The number of threads stored in that can be reused. If so, the new thread is obtained from the cache, and if there is space when the connection is disconnected, the client's line is placed in the cache. If there are many new threads, you can use this variable to improve performance.

By comparing the variables of Connections and Threads_created states, you can see the effect of this variable.

The default value is 110, and the adjustable value is 80.

thread_concurrency

It is recommended to set it to twice the number of server CPU cores. For example, for a dual-core CPU, the value of thread_concurrency should be 4; for 2 dual-core CPUs, the value of thread_concurrency should be 8. The default is 8

wait_timeout

Specify the maximum connection time for a request, which can be set to 5-10 for a server with about 4GB of memory.

Configure several variables of InnoDB

innodb_buffer_pool_size

For InnoDB tables, innodb_buffer_pool_size has the same effect as key_buffer_size for MyISAM tables. InnoDB uses this parameter to specify the size of memory to buffer data and indexes. For a single MySQL database server, the maximum value can be set to 80% of the physical memory.

According to the MySQL manual, for a machine with 2G memory, the recommended value is 1G (50%).

innodb_flush_log_at_trx_commit

It mainly controls the time point at which InnoDB writes the data in the log buffer into the log file and flushes the disk. The values are 0, 1, and 2 respectively. 0, means that when the transaction is committed, no log write operation is performed, but the data in the log buffer is written to the log file every second and the disk is flushed; 1, then every second or every transaction commit will be The operations that cause the log file to be written and flush the disk ensure the ACID of the transaction; set to 2, each transaction commit causes the write to the log file, but the flush disk operation is completed every second.

The actual test found that this value has a great influence on the speed of inserting data. When it is set to 2, it only takes 2 seconds to insert 10000 records, when it is set to 0, it only takes 1 second, and when it is set to 1, it takes 229 seconds. Therefore, the MySQL manual also recommends combining insert operations into one transaction as much as possible, which can greatly increase the speed.

According to the MySQL manual, this value can be set to 0 or 2 under the premise of allowing the risk of losing the most recent transaction.

innodb_log_buffer_size

The log cache size is generally 1-8M, and the default is 1M. For larger transactions, the cache size can be increased.

Can be set to 4M or 8M.

innodb_additional_mem_pool_size

This parameter specifies the size of the memory pool used by InnoDB to store the data dictionary and other internal data structures. The default value is 1M. Usually it is not too big, as long as it is enough, it should be related to the complexity of the table structure. If it is not enough, MySQL will write a warning message in the error log.

According to the MySQL manual, for a machine with 2G memory, the recommended value is 20M, which can be increased appropriately.

innodb_thread_concurrency=8

The recommended setting is 2*(NumCPUs+NumDisks), the default is generally 8

A very shallow understanding of mysql tuning steps

A very shallow understanding of mysql tuning steps

Guess you like