MySQL database performance optimization
Personal blog website: http://01time.cn
Optimization of mysql database, similar to other databases
1. The positioning of database optimization
Find and locate slow query
2. Database optimization methods
- Create an index: Create a suitable index, we can query in the index first, and then directly find the corresponding record after querying
- Sub-table: When a table has a lot of data or some fields of a table have many values and are rarely used, use horizontal sub-table or vertical sub-table to optimize
- Read-write separation: When a server cannot meet the requirements, the cluster is clustered by read-write separation
- Caching: Redis is used for caching
3. The way of database optimization to locate slow queries
Before the project self-inspection project is transferred to the test, start the slow query when starting the mysql database, and write the statement that executes the slow query to the log.
After running for a certain period of time, find the slow query statement by viewing the log
Enable slow query method 1: Modify the configuration file
Windows: The configuration file for Windows is my.ini, usually in the MySQL installation directory or c:\Windows.
Linux: The Linux configuration file is my.cnf, usually under /etc
Add a few lines to my.ini:
long_query_time : Set the threshold of slow query, the SQL exceeding the second set value will be recorded in the slow query log, the default value is 10s slow_query_log : Specifies whether to enable the slow query log log_slow_queries : Specifies whether to enable the slow query log (this parameter should be replaced by slow_query_log for compatibility preservation) slow_query_log_file : Specify the storage location of the slow log file, it can be empty, the system will give a default file host_name-slow.log min_examined_row_limit: Query checks that return less than the rows specified by this parameter will not be logged to the slow query log log_queries_not_using_indexes: Whether the slow query log that does not use the index is recorded to the index
4. Analysis of database optimization slow query
Use the explain slow query statement to analyze the slow query statement in detail
mysql>explain select * from dept where loc='aaa' \G
*********************1.row ********************
id:1 //Query serial number
select_type:SIMPLE //Query type
table:dept //Query table name
In the way described by type:ALL, all indicates that the column of full table scan is very important, and it shows which type of connection is used and whether or not an index is used.
The join types from best to worst are const, eq_reg, ref, range, indexhe, and ALL
possible_keys:NULL//Index that may be used
key:NULL//The actual index used
key_len:NULL//
rows: 10//How many rows are scanned by the sql statement and the records that may be obtained
Extra: Using where //additional information of the sql statement, such as the sorting method filesort
5. Follow the paradigm of database optimization
Database table design needs to follow the paradigm
The normal form of the table is to meet 1NF first, then meet 2NF, and further meet 3NF
1NF: That is, the columns of the table are atomic and cannot be decomposed, that is, the information of the columns cannot be decomposed. As long as the database is a relational database, 1NF is automatically satisfied. In relational databases, splitting columns is not allowed.
2NF: The records in the table are unique, usually we design a primary key to achieve
3NF: that is, there should be no redundant data in the table, that is, if the information of the table can be deduced, a separate field should not be designed to store (foreign key)
Anti-3NF: A database without redundancy is not necessarily the best database. Sometimes, in order to improve operating efficiency, it is necessary to lower the paradigm standard and properly retain redundant data. The standard work is put into consideration when designing the physical data model, reducing the paradigm is adding fields, allowing redundancy, orders and line items, album views and photo views, etc.
6. Choose the appropriate storage engine for database optimization
In development, we often use the storage engine myisam/innodb/memory
MyISAM storage engine
If the table does not have high requirements for things, and is mainly based on query and addition, we consider using the myisam storage engine, such as the post table and reply table in bbs.
INNODB storage engine
The transaction requirements are high, and the data saved are important data. We recommend using the INNODB storage engine, such as the order table and the account table.
Memory storage engine
Our data changes frequently and does not need to be put into storage. At the same time, we are frequently queried and modified. We consider using the Memory storage engine, which is extremely fast.
The main difference between the MyISAM storage engine and the INNODB storage engine:
1. Transaction security Myisam does not support transactions but innodb does
2. Query and add speed Myisam does not need to consider synchronization locks without supporting transactions, and the query and add speed is fast
3. Lock mechanism Myisam only supports table locks, innodb supports row locks
4. Foreign keys myisam does not support foreign keys, innodb supports foreign keys
5. Support full-text index Myisam supports full-text index, innodb does not support full-text index
7. Database optimization to create appropriate indexes
Index (Index) is a data structure that helps DBMS obtain data efficiently
Classification: common index/unique index/primary key index/full-text index
Normal index: allow duplicate values to appear
Unique index: same as ordinary index except that it cannot have duplicate values
Primary key index: It is created with the setting of the primary key, that is, when a column is set as the primary key, the database will automatically create an index for the column. This is the primary key index, which is unique and has no null value.
Full-text index: used to index the text fields (char, varchar, text) in the table, the full-text index is for myisam
8. Index usage tips for database optimization
Indexing drawbacks
1. Take up disk space
2. Affects dml (insert, modify, delete) operations and slows down
scenes to be used
a: must be often used in where conditions, if you don't do a query, it doesn't make sense
b: The contents of this field are not unique values
c: The content of the field does not change frequently and does not appear in the where statement. The field should not be indexed
specific skills
1. For the created multi-column index (composite index), the index will not be used unless the first part is used
alert table dept add index my_index(dname,loc);//The column on the left of dname, loc is the column on the right explain select * from dept where dname='aaa' \G will use the index explain select * from dept where loc='aaa' \G will not use the index
2. For the use of like query, if the query is '%aaa', the index will not be used, and 'aaa%' will use the index
explain select * from dept where dname like '%aaa' \G will not use the index explain select * from dept wehre dname like 'aaa%' \G will use the index
Therefore, when querying like, characters such as % or _ cannot be used at the front of the keyword. If there must be a changed value in front, consider using the full-text index -> sphinx
3. If there is an or in the condition, and the index is not used in the condition, even if there is a conditional index, it will not be used. In other words, all fields that are required to be used must be able to use the index when they are used alone.
4. If the column type is a string, be sure to enclose the database in quotation marks in the condition, otherwise do not use the index
explain select * from dept where dname='111'//Use index explain select * from dept where dname =111//Do not use index
5. If myslq estimates that using a full table scan is faster than using an index, do not use an index
For example: there is only one record in the table
9. Sub-tables for database optimization
The sub-tables are divided into horizontal (by row) sub-tables and vertical (by columns) sub-tables
Horizontal sub-table:
According to experience, MySQL table data generally reaches millions of levels, the query efficiency will be very low, it is easy to cause table locks, and even a lot of connections will be accumulated, which will directly hang up; horizontal table division can greatly reduce these pressures, and divide tables by row data.
Vertical split table:
If there are a lot of fields in a table (long text, binary), and it is only queried in rare cases, then you can put multiple fields into one table and associate them with foreign keys. table by column
For example: examination details table, generally we only pay attention to the score, not the details
Horizontal split strategy
1. Schedule by time
This method of sub-tables has certain limitations. When the data has strong effectiveness, such as Weibo sending records, WeChat message records, etc., few users will query the data from several months ago. by month
2. Divide the table according to the interval range
Generally, there are strict auto-increment id requirements, such as dividing the table according to the user_id level:
table_1 user_id 从1~100w
table_2 user_id 从101~200w
table_3 user_id 从201~300w
3.hash sub-table****** (used most)
Calculate the table name of the data storage table through the ID or name of an original target through a certain hash algorithm, and then access the corresponding table
10. Database optimization for read-write separation
When the maximum number of concurrent connections supported by a database is limited, if there are too many concurrent accesses by users and one server cannot meet the requirements, it can be clustered. The most commonly used cluster processing technology of MySQL is read-write separation.
1. Master-slave synchronization
The database will eventually persist the data to the disk. If the cluster must ensure that the data of each database server is consistent, operations that can change the database are written to the main database, and other databases synchronize data from the main database.
2. Separation of read and write
Use load balancing to implement write operations to the primary database, and read operations to the slave server
11. Cache for database optimization
Add a cache layer between the persistence layer (dao) and the data (db). If the data accessed by the user has been cached, the user accesses the database directly from the cache without accessing the database, and the cache is at the memory level. Access is fast.
Function: reduce the pressure on the database server and reduce the access time.
Commonly used caches in java are reids, memcache
Can use redis as central cache
