The mysql database is a widely used relational database. Its small size, multi-processor support, open source and free features make it especially used in small and medium-sized websites on the Internet. In the process of using mysql, non-standard SQL writing and non-optimal strategy selection may lead to system performance and even functional defects.
Just a few days ago, the cloud business department of my company held a technical exchange meeting about mysql. One of the parts focused on the common problems in the design and use of the mysql database during the development process, and proposed relevant optimization plans. Based on the content of the meeting and reviewing relevant materials, I made a summary of this part, combined with my own work experience and understanding to form this article for sharing, hoping to help colleagues solve related problems in their work.
This article will focus on the following three questions:
library table design
slow SQL problem
What to do when misoperation or program bug
1. Library table design
1.1 Engine selection
In mysql 5.1, a new plug-in storage engine architecture was introduced, allowing the storage engine to be loaded into a running new mysql server. Using the mysql plug-in storage engine architecture, allows database professionals or software developers designing library tables to select specialized storage engines for specific application requirements, without having to manage any special application coding requirements, and without having to consider all the underlying implementations detail. Therefore, although different storage engines have different capabilities, applications are separated from them. In addition, users can store engines in three levels of server, database and table, providing great flexibility.
Commonly used storage engines for mysql include MYISAM, Innodb and Memory, and their characteristics are as follows:
MYISAM: full table lock, with high execution speed, one write request please block all read and write requests of the same table, poor concurrency performance, relatively small footprint, MySQL 5.5 and below only supports full-text indexing, not transactions .
Innodb: Row-level locks (SQL all use index queries), relatively strong concurrency, 2.5 times the space occupied by MYISAM, does not support full-text indexing (supported since 5.6), and supports transactions.
Memory : Full table lock, stored in memory, fast, but will occupy memory space proportional to the amount of data and the data will be lost when mysql restarts.
Based on the above features, it is recommended that most of them be set to innodb engine, and MYISAM or Memory should be considered for special services, such as full-text index support or extremely high execution efficiency.
1.2 Sub-table method
In the process of using database tables, in order to reduce the burden on the database server and shorten the query time, the design of sub-tables is often considered. There are two types of sub-tables, one is vertical sub-table (the content that could have been in the same table is artificially divided and stored in multiple tables with different structures) and horizontal sub-table (large table structure is cut horizontally into the same table) different tables of structure).
Among them, the common methods of vertical classification include classification according to activity, classification according to importance, etc. Its main problems are as follows:
Resource contention between tables;
The probability of lock contention is small;
Realize the hierarchical storage of core and non-core, such as the UDB login library is divided into first-level, second-level and third-level libraries;
Solved the database synchronization pressure problem.
Horizontal sub-scale refers to the division of big data scales according to some specific rules, such as time sub-scales. Its main problems are as follows:
Performance problems caused by too large a single table;
The single server space problem caused by the single table is too large.
1.3 Indexing Problems
An index is a structure that sorts the values of one or more columns in a database table, and building an index helps get information faster. mysql has four different index types:
Primary key index ( PRIMARY )
Unique Index ( UNIQUE )
Ordinary index ( INDEX )
Full-text indexing (FULLTEXT, MYISAM and Innodb above mysql 5.6)
The purpose of creating an index is to speed up the search or sorting of records in the table, and the more indexes the better, because creating an index comes at a cost: one is to increase the storage space of the database, and the other is to insert and modify data. Spend more time maintaining the index.
When designing a table or index, the following problems often occur:
Build fewer or no indexes. This problem is the most prominent. It is recommended that the DBA can assist in checking when building the table.
Index abuse. Abusing indexes will slow down write requests and slow down the response speed of the overall database (mysql under 5.5 can only use one index).
Joint indexes are never considered. In fact, the efficiency of a joint index is often higher than that of a single-column index.
Suboptimal column selection. Fields with low selectivity are not suitable for building single-column indexes, such as fields of type status.
2. Slow SQL problem
2.1 Reasons for slow SQL
When encountering slow SQL, you cannot simply attribute the cause to SQL writing problems (although this is the most common factor). In fact, there are many factors that cause slow SQL, including bugs in hardware and mysql itself. According to the probability of occurrence, from large to small, they are listed as follows:
SQL writing problem
Lock
Business instances interfere with each other and compete for IO/CPU resources
server hardware
MYSQL BUG
2.2 Slow SQL optimization caused by SQL writing
It is relatively convenient to optimize the slow SQL caused by SQL writing. As mentioned in the previous section, the correct use of indexes can speed up the query speed, so we need to pay attention to the rules related to indexes when writing SQL:
Field type conversion leads to no index, such as string type without quotation marks, number type with quotation marks, etc., which may lead to full table scan without index;
MySQL does not support function conversion, so functions cannot be added in front of fields, otherwise the index will not be used;
Do not add or subtract in front of fields;
If the string is relatively long, you can consider part of the index to reduce the size of the index file and improve the writing efficiency;
like % does not use the index in front;
According to the second and subsequent fields of the joint index, the index is not used for a separate query;
don't use select *;
Please try to use ascending order for sorting;
The query of or should be replaced by union as much as possible (Innodb);
The highly selective fields of the composite index are listed first;
The order by / group by field is included in the index to reduce sorting, which will be more efficient.
In addition to the above index usage rules, you need to pay special attention to the following points when writing SQL:
Try to avoid the SQL of large transactions, the SQL of large transactions will affect the concurrent performance of the database and master-slave synchronization;
The problem of paging statement limit;
To delete all records in the table, please use truncate instead of delete;
Do not let mysql do redundant things, such as calculations;
Write SQL with fields to prevent problems caused by subsequent table changes, and the performance is also relatively good (involving data dictionary parsing, please query the information yourself);
Use select count(*) on Innodb, because Innodb stores statistics;
Use Oder by rand() with caution.
3. Analysis and diagnostic tools
In daily development work, we can do some work to prevent slow SQL problems, such as using diagnostic tools to analyze SQL before going online. Commonly used tools are:
mysqldumpslow
mysql profile
mysql explain
The specific use and analysis methods are not repeated here, and there are abundant resources on the Internet for reference.
4. What to do when misoperation or program bug
This question is obviously mainly aimed at young colleagues who are just starting to work... In fact, it is not uncommon for data deletion or confusion caused by misoperation and program bugs, but developers who have just entered the industry will be more nervous. A mature enterprise often has complete data management specifications and rich data recovery solutions (except for start-up companies), and will perform data backup and data disaster recovery. When you find that the online data has been deleted or changed by mistake due to misoperation or program bug, you must not panic. You should contact the DBA in time to restore the data as soon as possible (stop the service directly in serious cases) to minimize the impact and loss as much as possible. . For the operation of important data (such as funds), it must be repeatedly tested during development to ensure that there are no problems before going online.