MySQL eight kinds of common errors SQL Usage
Foreword
MySQL database remains strong popularity growth trend in 2016. More and more customers to their own applications built on the MySQL database, and even migrate from Oracle to MySQL up. But there are some customers encountered some such as slow response time, CPU, etc. played in the process of using the MySQL database.
Ali cloud cloud RDS expert service team to help customers solve a lot of the urgent problems. Now "ApsaraDB expert diagnostic report" appears in some common SQL problems are summarized as follows, for your reference.
1, LIMIT statement
Paging query is one of the most frequently used scene, but also usually the most problematic areas.
For simple statement such as the following, the general approach is to think of DBA on the type, name, create_time field plus a composite index. Such conditions can be effectively used to sort the index, to elevate performance.
SELECT *
FROM operation
WHERE type = 'SQLStats'
AND name = 'SlowLog'
ORDER BY create_time
LIMIT 1000, 10;复制代码
Okay, maybe more than 90% of the DBA to resolve the problem so far.
But when the LIMIT clause becomes "LIMIT 1000000,10", programmers will still complain: I only get 10 records Why or slow?
To know the database does not know the article 1 million records from where to start, even if there is also need to calculate the index from scratch again. This performance problems, in most cases is a lazy programmer.
Browse the page data in the front end, or a large batch of data to export other scenes, the maximum value may be as a previous parameter as a query. SQL redesigned as follows:
SELECT *
FROM operation
WHERE type = 'SQLStats'
AND name = 'SlowLog'
AND create_time > '2017-03-16 14:00:00'
ORDER BY create_time limit 10;复制代码
The basic query time fixed in the new design, not with the amount of data grows and changes.
2, implicit conversion
SQL statement query variables and fields defined type mismatch is another common mistake. For example, the following statement:
mysql> explain extended SELECT *
> FROM my_balance b
> WHERE b.bpn = 14000000123
> AND b.isverified IS NULL ;
mysql> show warnings;
| Warning | 1739 | Cannot use ref access on index 'bpn' due to type or collation conversion on field 'bpn'复制代码
Wherein bpn defined fields as varchar (20), MySQL, then the strategy comparison string after converting to digital. Function applied to the table fields, indices failure.
The above application framework may be filled automatically parameter rather than the programmer's intent. Now a lot of very complicated application framework, easy to use but also be careful it might give yourself digging.
3, the association update, delete
Although MySQL5.6 the introduction of physical and chemical properties, but it currently only requires special attention for optimizing the query statement. For the need to manually update or delete rewritten JOIN.
For example, the following UPDATE statement, MySQL is the actual implementation cycles / nested subqueries (DEPENDENT SUBQUERY), the execution time can be imagined.
UPDATE operation o
SET status = 'applying'
WHERE o.id IN (SELECT id
FROM (SELECT o.id,
o.status
FROM operation o
WHERE o.group = 123
AND o.status NOT IN ( 'done' )
ORDER BY o.parent,
o.id
LIMIT 1) t);复制代码
Implementation plan:
+----+--------------------+-------+-------+---------------+---------+---------+-------+------+-----------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+-------+-------+---------------+---------+---------+-------+------+-----------------------------------------------------+
| 1 | PRIMARY | o | index | | PRIMARY | 8 | | 24 | Using where; Using temporary |
| 2 | DEPENDENT SUBQUERY | | | | | | | | Impossible WHERE noticed after reading const tables |
| 3 | DERIVED | o | ref | idx_2,idx_5 | idx_5 | 8 | const | 1 | Using where; Using filesort |
+----+--------------------+-------+-------+---------------+---------+---------+-------+------+-----------------------------------------------------+复制代码
After overwriting is JOIN, the selection mode is changed from the subquery DEPENDENT SUBQUERY DERIVED, significantly quicker, decreased from 7 seconds to 2 milliseconds.
UPDATE operation o
JOIN (SELECT o.id,
o.status
FROM operation o
WHERE o.group = 123
AND o.status NOT IN ( 'done' )
ORDER BY o.parent,
o.id
LIMIT 1) t
ON o.id = t.id
SET status = 'applying' 复制代码
Implementation plan reduces to:
+----+-------------+-------+------+---------------+-------+---------+-------+------+-----------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+-------+---------+-------+------+-----------------------------------------------------+
| 1 | PRIMARY | | | | | | | | Impossible WHERE noticed after reading const tables |
| 2 | DERIVED | o | ref | idx_2,idx_5 | idx_5 | 8 | const | 1 | Using where; Using filesort |
+----+-------------+-------+------+---------------+-------+---------+-------+------+-----------------------------------------------------+复制代码
4, mixed sorting
MySQL can not use sort mix index. However, in some scenarios, there are opportunities to use a special method to enhance performance.
SELECT *
FROM my_order o
INNER JOIN my_appraise a ON a.orderid = o.id
ORDER BY a.is_reply ASC,
a.appraise_time DESC
LIMIT 0, 20 复制代码
Show Execution Plan for the full table scan:
+----+-------------+-------+--------+-------------+---------+---------+---------------+---------+-+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra
+----+-------------+-------+--------+-------------+---------+---------+---------------+---------+-+
| 1 | SIMPLE | a | ALL | idx_orderid | NULL | NULL | NULL | 1967647 | Using filesort |
| 1 | SIMPLE | o | eq_ref | PRIMARY | PRIMARY | 122 | a.orderid | 1 | NULL |
+----+-------------+-------+--------+---------+---------+---------+-----------------+---------+-+复制代码
Since is_reply only two states 0 and 1, we follow the following method after rewriting, the execution time is reduced from 1.58 seconds to 2 milliseconds.
SELECT *
FROM ((SELECT *
FROM my_order o
INNER JOIN my_appraise a
ON a.orderid = o.id
AND is_reply = 0
ORDER BY appraise_time DESC
LIMIT 0, 20)
UNION ALL
(SELECT *
FROM my_order o
INNER JOIN my_appraise a
ON a.orderid = o.id
AND is_reply = 1
ORDER BY appraise_time DESC
LIMIT 0, 20)) t
ORDER BY is_reply ASC,
appraisetime DESC
LIMIT 20;复制代码
5, EXISTS statement
When MySQL treat EXISTS clause, the implementation still use nested subqueries. SQL statement such as the following:
SELECT *
FROM my_neighbor n
LEFT JOIN my_neighbor_apply sra
ON n.id = sra.neighbor_id
AND sra.user_id = 'xxx'
WHERE n.topic_status < 4
AND EXISTS(SELECT 1
FROM message_info m
WHERE n.id = m.neighbor_id
AND m.inuser = 'xxx')
AND n.topic_type <> 5 复制代码
Implementation plan are:
+----+--------------------+-------+------+-----+------------------------------------------+---------+-------+---------+ -----+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+-------+------+ -----+------------------------------------------+---------+-------+---------+ -----+
| 1 | PRIMARY | n | ALL | | NULL | NULL | NULL | 1086041 | Using where |
| 1 | PRIMARY | sra | ref | | idx_user_id | 123 | const | 1 | Using where |
| 2 | DEPENDENT SUBQUERY | m | ref | | idx_message_info | 122 | const | 1 | Using index condition; Using where |
+----+--------------------+-------+------+ -----+------------------------------------------+---------+-------+---------+ -----+复制代码
Change removed exists to join, can be avoided nested subquery, execution time is reduced from 1.93 seconds to 1 millisecond.
SELECT *
FROM my_neighbor n
INNER JOIN message_info m
ON n.id = m.neighbor_id
AND m.inuser = 'xxx'
LEFT JOIN my_neighbor_apply sra
ON n.id = sra.neighbor_id
AND sra.user_id = 'xxx'
WHERE n.topic_status < 4
AND n.topic_type <> 5 复制代码
The new execution plan:
+----+-------------+-------+--------+ -----+------------------------------------------+---------+ -----+------+ -----+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+ -----+------------------------------------------+---------+ -----+------+ -----+
| 1 | SIMPLE | m | ref | | idx_message_info | 122 | const | 1 | Using index condition |
| 1 | SIMPLE | n | eq_ref | | PRIMARY | 122 | ighbor_id | 1 | Using where |
| 1 | SIMPLE | sra | ref | | idx_user_id | 123 | const | 1 | Using where |
+----+-------------+-------+--------+ -----+------------------------------------------+---------+ -----+------+ -----+复制代码
6, condition pushdown
External query can not be pushed down to the complex views or subqueries case are:
-
Aggregation subqueries;
-
Containing the subquery LIMIT;
-
UNION or UNION ALL subqueries;
-
Output fields in the sub-queries;
The following statement, it can be seen from the plan execution after its conditioning subquery in the polymerization:
SELECT *
FROM (SELECT target,
Count(*)
FROM operation
GROUP BY target) t
WHERE target = 'rm-xxxx'复制代码
+----+-------------+------------+-------+---------------+-------------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+-------+---------------+-------------+---------+-------+------+-------------+
|
1
| PRIMARY |
<derived2>
| ref |
<auto_key
0
>
| <auto_key0> |
514
| const |
2
| Using where |
| 2 | DERIVED | operation | index | idx_4 | idx_4 | 519 | NULL | 20 | Using index |
+----+-------------+------------+-------+---------------+-------------+---------+-------+------+-------------+复制代码
After determining a query from the semantic element can be directly pushed down, rewritten as follows:
SELECT target,
Count(*)
FROM operation
WHERE target = 'rm-xxxx'
GROUP BY target复制代码
Implementation plan becomes:
+----+-------------+-----------+------+---------------+-------+---------+-------+------+--------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------+------+---------------+-------+---------+-------+------+--------------------+
| 1 | SIMPLE | operation | ref | idx_4 | idx_4 | 514 | const | 1 | Using where; Using index |
+----+-------------+-----------+------+---------------+-------+---------+-------+------+--------------------+复制代码
For details on the MySQL external conditions can not be pushed down instructions please refer to the article:
http://mysql.taobao.org/monthly/2016/07/08
7, advance narrow range
First on the original SQL statement:
SELECT *
FROM my_order o
LEFT JOIN my_userinfo u
ON o.uid = u.uid
LEFT JOIN my_productinfo p
ON o.pid = p.pid
WHERE ( o.display = 0 )
AND ( o.ostaus = 1 )
ORDER BY o.selltime DESC
LIMIT 0, 15 复制代码
The SQL statement is intent: first series connection of the left, and then take 15 records before ordering. It can also be seen from the implementation plan, the final step in sorting the estimated number of 900,000 records, time consuming 12 seconds.
+----+-------------+-------+--------+---------------+---------+---------+-----------------+--------+----------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+-----------------+--------+----------------------------------------------------+
| 1 | SIMPLE | o | ALL | NULL | NULL | NULL | NULL | 909119 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | u | eq_ref | PRIMARY | PRIMARY | 4 | o.uid | 1 | NULL |
| 1 | SIMPLE | p | ALL | PRIMARY | NULL | NULL | NULL | 6 | Using where; Using join buffer (Block Nested Loop) |
+----+-------------+-------+--------+---------------+---------+---------+-----------------+--------+----------------------------------------------------+复制代码
Since the last WHERE conditions and ordering are for the most left main table, so you can first sort of my_order reduce the amount of data in advance to do a left join. After rewriting the following SQL execution time is reduced to about 1 millisecond.
SELECT *
FROM (
SELECT *
FROM my_order o
WHERE ( o.display = 0 )
AND ( o.ostaus = 1 )
ORDER BY o.selltime DESC
LIMIT 0, 15
) o
LEFT JOIN my_userinfo u
ON o.uid = u.uid
LEFT JOIN my_productinfo p
ON o.pid = p.pid
ORDER BY o.selltime DESC
limit 0, 15复制代码
Re-examine the execution plan: after (select_type = DERIVED) subquery materialized participate JOIN. Although the estimated 900,000 still scanning line, but use of the index and LIMIT clause, the actual execution time becomes very small.
+----+-------------+------------+--------+---------------+---------+---------+-------+--------+----------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+--------+---------------+---------+---------+-------+--------+----------------------------------------------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 15 | Using temporary; Using filesort |
| 1 | PRIMARY | u | eq_ref | PRIMARY | PRIMARY | 4 | o.uid | 1 | NULL |
| 1 | PRIMARY | p | ALL | PRIMARY | NULL | NULL | NULL | 6 | Using where; Using join buffer (Block Nested Loop) |
| 2 | DERIVED | o | index | NULL | idx_1 | 5 | NULL | 909112 | Using where |
+----+-------------+------------+--------+---------------+---------+---------+-------+--------+----------------------------------------------------+复制代码
8, the intermediate result sets pushdown
Examples (left main table join in the priority action query) look at the following has been initially optimized:
SELECT a.*,
c.allocated
FROM (
SELECT resourceid
FROM my_distribute d
WHERE isdelete = 0
AND cusmanagercode = '1234567'
ORDER BY salecode limit 20) a
LEFT JOIN
(
SELECT resourcesid， sum(ifnull(allocation, 0) * 12345) allocated
FROM my_resources
GROUP BY resourcesid) c
ON a.resourceid = c.resourcesid复制代码
So the statement that there are other problems? C is not difficult to see the whole table subqueries aggregate queries, in a particularly large number of table situation will lead to decreased performance of the entire statement.
In fact, for subqueries c, left the final outcome of the connection data can be set only interested in the main table and resourceid can match. Therefore, we can rewrite the following statements, execution time decreased from the original 2 milliseconds to 2 seconds.
SELECT a.*,
c.allocated
FROM (
SELECT resourceid
FROM my_distribute d
WHERE isdelete = 0
AND cusmanagercode = '1234567'
ORDER BY salecode limit 20) a
LEFT JOIN
(
SELECT resourcesid， sum(ifnull(allocation, 0) * 12345) allocated
FROM my_resources r,
(
SELECT resourceid
FROM my_distribute d
WHERE isdelete = 0
AND cusmanagercode = '1234567'
ORDER BY salecode limit 20) a
WHERE r.resourcesid = a.resourcesid
GROUP BY resourcesid) c
ON a.resourceid = c.resourcesid复制代码
But a sub-query appears more than once in our SQL statement. Such an approach not only additional costs, but also makes the entire statement of significant complexity. Rewrite again using the WITH statement:
WITH a AS
(
SELECT resourceid
FROM my_distribute d
WHERE isdelete = 0
AND cusmanagercode = '1234567'
ORDER BY salecode limit 20)
SELECT a.*,
c.allocated
FROM a
LEFT JOIN
(
SELECT resourcesid， sum(ifnull(allocation, 0) * 12345) allocated
FROM my_resources r,
a
WHERE r.resourcesid = a.resourcesid
GROUP BY resourcesid) c
ON a.resourceid = c.resourcesid复制代码
to sum up
Database compiler generates the execution plan, determine the actual implementation of SQL. But the compiler just try the service, the compiler of all the databases are not perfect.
Most of the above-mentioned scene, there are performance problems in other databases. Learn database compiler features, its weaknesses in order to avoid regulation, to write high-performance SQL statement.
Programmers in the design of the data model and write SQL statements, thought or consciousness algorithm should be brought in.
Write complex SQL statements to develop the habit of using the WITH statement. Concise and clear thinking of SQL statements can also reduce the burden on the database.