Server parameters
16G memory, 4-core CPU
vim /etc/my.cnf
Original:
back_log=170
max_connections=600
max_user_connections=0
thread_concurrency=10
#skip-name-resolve does not change parameters, it is commented by default
#skip-networking is commented out by default. This parameter is not present.
default_storage_engine=innodb
key_buffer_size=8M
innodb_buffer_pool_size=128M
innodb_additional_mem_pool_size=8M
innodb_log_buffer_size=8M
innodb_log_file_size=48M
innodb_flush_log_at_trx_commit=1
query_cache_size=1M
query_cache_type=OFF
read_buffer_size=128K
sort_buffer_size=256K
read_rnd_buffer_size=256K
read_rnd_buffer_size=256K
mp_table_size=16M
thread_cache_size=14
optimization:
back_log=300 (256kb per connection, occupancy: 75M)
indicates how many requests can be stored in the stack for a short time before MySQL temporarily stops answering new requests. That is to say, if the connection data of MySql reaches max_connections, the new request will be stored in the stack to wait for a connection to release resources. The number of the stack is the back_log. If the number of waiting connections exceeds the back_log, it will not be processed Grant connection resources. Will report: unauthenticated user | xxx.xxx.xxx.xxx | NULL | Connect | NULL | login | NULL pending connection process.
The back_log value cannot exceed the size of the listening queue for TCP/IP connections. If it exceeds, it will be invalid. Check the size of the listening queue of the current system's TCP/IP connection. Command: cat /proc/sys/net/ipv4/tcp_max_syn_backlog The current system is 1024. For Linux systems, it is recommended to set it to an integer less than 512.
max_connections=800
The maximum number of connections of MySql. If the number of concurrent connection requests of the server is relatively large, it is recommended to increase this value to increase the number of parallel connections. Of course, this is based on the situation that the machine can support. Since MySql provides connection buffers for each connection, it will cost more memory, so this value should be adjusted appropriately, and the setting value should not be increased blindly. You can use the 'conn%' wildcard to view the number of connections in the current state to determine the size of this value.
The maximum number of connections allowed by the MySQL server is 16384;
max_user_connections=800
The maximum connection of each database user, the maximum number of parallel connections for all clients of a certain account to connect to the MYSQL service in parallel. Simply put, it refers to the maximum number of connections that the same account can connect to the MySQL service at the same time. Set to 0 for no limit.
Here is an introduction to Max_used_connections: it refers to the maximum number of parallel connections at the same time since the start of the MySQL service this time. It does not refer to the current connection status, but a comparison value. If at a certain time in the past, 1000 requests were connected to the MYSQL service at the same time, and there were no such large concurrent requests after that, Max_used_connections=1000. Please note the difference from max_user_connections in show variables. Defaults to 0 for infinite.
thread_concurrency=8
Whether the value of thread_concurrency is correct or not has a great impact on the performance of mysql. In the case of multiple cpus (or multi-core), incorrectly setting the value of thread_concurrency will cause mysql to not make full use of multiple cpus (or multi-cores). , there is a situation where only one cpu (or core) is working at the same time.
thread_concurrency should be set to twice the number of CPU cores. For example, if there is a dual-core CPU, the thread_concurrency should be 4; if there are 2 dual-core CPUs, the thread_concurrency value should be 8. For
example, according to the configuration of our current system described above, according to The above calculation rules, here should be: 4*2=8
#skip-name-resolve (comment unchanged)
disables MySQL's DNS resolution for external connections. Using this option can eliminate MySQL's DNS resolution time. However, it should be noted that if this option is enabled, all remote host connection authorizations must use the IP address method, otherwise MySQL will not be able to process connection requests normally!
#skip-networking (recommended to be commented out, do not turn on)
Turning this option on can completely turn off MySQL's TCP/IP connection mode. If the WEB server accesses the MySQL database server through a remote connection, do not turn on this option! Otherwise, it will not be able to connect normally!
default_storage_engine=innodb (default unchanged)
set the default storage type for creating databases and tables
show table status like 'tablename' shows the current storage status value of the table to
see which storage status and default storage status MySQL has
show engines;
create a table and specify the storage type
CREATE TABLE mytable (id int, title char(20)) ENGINE = INNODB;
innodb_lock_wait_timeout = 50 (default)
key_buffer_size=100M
key_buffer_size is the buffer size used for the index block, increasing it can get a better handle index (for all reads and multiple rewrites), for MyISAM (a type of MySQL table storage, you can see details on Baidu, etc.) table The one parameter that has the greatest impact on performance. If you make it too big, the system will start wrapping pages and get really slow. Strictly speaking, it determines the speed of database index processing, especially the speed of index reading. For servers with about 4GB of memory, this parameter can be set to 256M or 384M.
How to know whether the setting of key_buffer_size is reasonable? Generally, you can check the status values of Key_read_requests and Key_reads. The ratio of key_reads / key_read_requests should be as low as possible, such as 1:100, 1:1000, 1:10000. Its value can be found with the following command:
show status like 'key_read%'; For
example, to view the current key_read and key_read_request values of the system:
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| Key_read_requests | 28535 |
| Key_reads | 269 |
+-------------------+-------+
It can be known that there are 28535 requests, and 269 requests are not found in memory to directly read the index from the hard disk.
The probability of missing the cache is: 0.94%=269/28535*100%. Generally, it is better to miss the probability below 0.1 . At present, it is far greater than 0.1, which proves that the effect is not good. If the hit rate is below 0.01, it is recommended to modify the key_buffer_size value appropriately.
innodb_buffer_pool_size=1024M (1G) - important, has a greater impact on performance
innodb_buffer_pool_size: mainly for a parameter that has the greatest impact on InnoDB table performance. The function is the same as Key_buffer_size. The memory occupied by InnoDB, in addition to innodb_buffer_pool_size used to store page cache data, also has an overhead of about 8% under normal circumstances, mainly used in the description of each cached page frame, adaptive hash and other data structures, if it is not safely closed, start If it needs to be restored at any time, about 12% of the memory needs to be opened for recovery, and the two together have an overhead of about 21%. Assuming: 12G innodb_buffer_pool_size, InnoDB may occupy 14.5G memory at most. If the system has only 16G, and only runs MySQL, and MySQL only uses InnoDB, then opening 12G for MySQL will maximize the use of memory.
In addition, the InnoDB and MyISAM storage engines are different. MyISAM's key_buffer_size can only cache index keys, while innodb_buffer_pool_size can cache data blocks and index keys. Properly increasing the size of this parameter can effectively reduce the disk I/O of InnoDB type tables.
When we operate an InnoDB table, all the returned data or any index block used in the process of data removal will go through this memory area.
You can calculate the cache hit rate by (Innodb_buffer_pool_read_requests – Innodb_buffer_pool_reads) / Innodb_buffer_pool_read_requests * 100%, and adjust the innodb_buffer_pool_size parameter size according to the hit rate for optimization. The value can be checked with the following command: show status like 'Innodb_buffer_pool_read%';
For example, to view the current system in the system
| Innodb_buffer_pool_read_requests | 1283826 |
| Innodb_buffer_pool_reads | 519 |
+---------------------------------------+---------+
Its hit rate is 99.959%=(1283826-519)/1283826*100% The higher the hit rate, the better.
If there is only a database on the server, consider setting it to 50 - 80 % of the total memory
innodb_additional_mem_pool_size=40M
innodb_additional_mem_pool_size sets the memory space used by the InnoDB storage engine to store data dictionary information and some internal data structures, so when we have a large number of database objects in a MySQL Instance, we need to adjust the size of this parameter appropriately to ensure that All data can be stored in memory to improve access efficiency.
It is relatively easy to know whether the size of this parameter is sufficient, because when it is too small, MySQL will record Warning information in the error log of the database, and then you will know that it is time to adjust the size of this parameter.
Check the error log cat /var/lib/mysql/machine name.error of the current system mysql and find that there are many warning warnings. Therefore, it should be adjusted to 20M.
According to the MySQL manual, for a machine with 2G memory, the recommended value is 20M.
100M of 32G memory
innodb_log_buffer_size=10M (can be unchanged)
innodb_flush_log_at_trx_commit=2
complains that Innodb is 100 times slower than MyISAM? Then you probably forgot to adjust this value. The default value of 1 means that every transaction commit or out-of-transaction command needs to flush the log to disk, which is time-consuming. Especially when using battery backed up cache. Setting it to 2 is fine for many applications, especially those transferred from MyISAM tables, which means that they are not written to the hard disk but written to the system cache. Logs will still be flushed to disk every second, so you generally won't lose more than 1-2 seconds of updates. Setting it to 0 will be faster, but the security aspect is relatively poor. Even if MySQL hangs, the transaction data may be lost. A value of 2 is only possible to lose data when the entire operating system hangs.
The actual test found that this value has a great impact on the speed of inserting data. When it is set to 2, it only takes 2 seconds to insert 10,000 records, when it is set to 0, it only takes 1 second, and when it is set to 1, it takes 229 seconds.
query_cache_type=ON
query_cache_size=20M (consider whether to enable it according to the database situation, it can be enabled for rarely changed data),
which greatly improves the query efficiency. According to the MySQL user manual, using query buffering can achieve a maximum efficiency of 238%.
Of course, Query Cache also has a fatal flaw, that is, when there is any change in the data of a certain table, the cached data in Query Cache will be invalidated by all select statements that reference the table. Therefore, when our data changes very frequently, using Query Cache may not be worth the harm.
read_buffer_size=2M
table sequential scan buffer, if the sequential scan request is very frequent, it can be appropriately increased by 1M—2M
sort_buffer_size=2M
The buffer size used for sorting. If you want to increase the speed of ORDER BY, first see if you can get MySQL to use an index instead of an extra sort stage. If you can't, try increasing the size of the sort_buffer_size variable
read_rnd_buffer_size=256K (can be unchanged, be careful not to set too large)
random read buffer size. When rows are read in any order (for example, in sorted order), a random read buffer is allocated. When performing a sorting query, MySql will first scan the buffer to avoid disk search and improve query speed. If you need to sort a large amount of data, you can appropriately increase the value. However, MySql will release the buffer space for each client connection, so you should try to set this value appropriately to avoid excessive memory overhead.
mp_table_size=16M (unchanged)
heap (heaped) table buffer size. All joins are done within one DML instruction, and most joins can even be done without temporary tables. Most temporary tables are memory-based (HEAP) tables. Temporary tables with large record lengths (the sum of the lengths of all columns) or tables containing BLOB columns are stored on disk. If the size of an internal heap table exceeds tmp_table_size, MySQL can automatically change the in-memory heap table to a hard disk-based MyISAM table as needed. It is also possible to increase the size of temporary tables by setting the tmp_table_size option. That is to say, if you increase this value, MySql will increase the size of the heap table at the same time, which can improve the speed of join query.
thread_cache_size=32 (server thread cache)
This value indicates the number of threads that can be reused in the cache. If there is space in the cache when disconnecting, the client's thread will be placed in the cache, if the thread is re-requested , then the request will be read from the cache, if the cache is empty or a new request, then the thread will be recreated, if there are many new threads, increasing this value can improve system performance. By comparing Connections and Threads_created The variable of the state, you can see the role of this variable. (–> indicates the value to be adjusted) According to the physical memory setting rules are as follows:
1G —> 8
2G —> 16
3G —> 32
Above 3G —> 64
mysql> show status like 'thread%';
+——————-+——-+
| Variable_name | Value |
+——————-+——-+
| Threads_cached | 9 | Number of cached idle threads
| Threads_connected | 19 | <— Threads that are in use (connected)
| Threads_created | 1706 | <— How many threads have been created since the service started
| Threads_running | 2 | <— Threads that are busy ( querying data, transferring data, etc.)
+——————-+——-+
Check how many times the database has been connected since booting?
mysql> show status like '%connection%';
+————————-+——-+
| Variable_name | Value |
+——————--+——-+
| Connections | 54645 | – >Number of historical connections since the service was started
| Max_used_connections | 131 |
+——————-+——-+
Is the setting value appropriate based on the hit rate of the connection thread pool? The hit rate is more than 90%, and the setting is reasonable.
(Connections - Threads_created) / Connections * 100 %
（54645-1706）/54645*100%
统计修改参数项
back_log=300
max_connections=800
max_user_connections=800
thread_concurrency=8
key_buffer_size=100M
innodb_buffer_pool_size=1024M
innodb_additional_mem_pool_size=40M
innodb_log_file_size = 100M
innodb_log_buffer_size = 40M
innodb_flush_log_at_trx_commit=2
read_buffer_size=2M
sort_buffer_size=2M
thread_cache_size=32