SQL judgment
It can be judged by the appearance of two SQL determining whether there is a problem:
System-level representation
CPU consumption severe
IO wait serious
Page response time is too long
Applications such as time-out error logs
You can use the sar command, top command to view the current system status.
Status may Prometheus, Grafana observation system and other monitoring tools. (Article interested you can look at me before) 640? Wx_fmt = png
SQL statement representation
lengthy
Execution time is too long
Get data from full table scan
Implementation plan rows, cost a lot
Have a good understanding of the lengthy SQL, SQL is too long a period of reading will certainly be poor, and the frequency of problems will certainly be higher. Further questions have to start from the judgment SQL execution plan, as follows:? 640 wx_fmt = png
The implementation plan tells us to go full table scan query Type = ALL, rows great (9,950,400) Basic may determine this is a "taste" of SQL.
SQL access issues
Different databases have different acquisition method, the following tools to obtain the current mainstream slow query SQL databases
MySQL
Slow query log
Testing tools loadrunner
Percona tools such as the company's ptquery
Oracle
AWR report
Testing tools such as loadrunner
The relevant internal view v $, $ session_wait etc.
GRID CONTROL monitoring tools
Dream up a database
AWR report
Testing tools such as loadrunner
DaMeng performance monitoring tools (dem)
The relevant internal view v $, $ session_wait etc.
SQL writing skills
The SQL writer has the following general tips:
• rational use of the index
Index less query slow; the index more large space, perform CRUD statements when necessary to maintain dynamic indexes, affect the performance of select high (duplicate values less) and is frequently cited the need to establish where B-tree index;
General join column needs to be indexed; more complex document types using full-text index query efficiency; indexed to strike a balance between the query and DML performance; pay attention to non-leading column-based query when the composite index creation
• Use alternative UNION ALL UNION
The efficiency is higher than UNION ALL UNION, UNION required when performing duplication; UNION need to sort data
• Avoid writing select *
When you need to execute SQL optimizer * turn into concrete columns; each query to be back to the table, you can not go covering index.
• JOIN field index recommendations
JOIN fields plus the general index advance
• Avoid complex SQL statements
Enhance readability; avoidance probability slow query; can be converted into a plurality of short queries, processing operations end
• Avoid writing where 1 = 1
• Avoid order by rand () Similar wording
RAND () causes data columns are scanned a plurality of times
SQL optimization
Implementation plan
Complete SQL optimization must first read the execution plan, the execution plan will tell you which places low efficiency, where can be optimized. We MYSQL example, to see what the implementation plan yes. (Implementation plan for each database are not the same, you need to inform themselves about) explain sql640? Wx_fmt = png
Fields explained
Each operation identifier id is performed independently, sequentially identifies the object to be operated, the larger the value of id, is performed first, if identical, the execution order from top to bottom
select_type each type of query select clause
Object name table is operated, usually a table name, but other formats
Partitions partition information match (for the non-partitioned table is NULL)
Type type connection operation
possible_keys might use index
Index (most significant column) Key optimizer actually used from best to worst the connection type is const, eq_reg, ref, range, index and ALL. Represents the current SQL appeared in "bad taste" occurs when ALL
key_len optimizer selected index key length in bytes
ref represents a reference bank operation target object, the reference object is a non-NULL
Query number of rows (for innodb, this value is an estimated value) tuple executed scanned
The percentage of the number of tuples filtered condition table data filtered
Additional information extra important implementation plan, when this column appears Using filesort, we must be careful when Using temporary character, probably need to optimize SQL statements
Next we use some actual cases to illustrate SQL optimization process optimization and optimization techniques.
Optimization Case
Table Structure
CREATE TABLE `a`
(
`Id` you (11) NOT NULLAUTO_INCREMENT,
`seller_id` bigint(20) DEFAULT NULL,
`seller_name` varchar(100) CHARACTER SET utf8 COLLATE utf8_bin DEFAULT NULL,
`gmt_create` varchar(30) DEFAULT NULL,
PRIMARY KEY (`id`)
);
CREATE TABLE `b`
(
`Id` you (11) NOT NULLAUTO_INCREMENT,
`seller_name` varchar(100) DEFAULT NULL,
`user_id` varchar(50) DEFAULT NULL,
`user_name` varchar(100) DEFAULT NULL,
`sales` bigint(20) DEFAULT NULL,
`gmt_create` varchar(30) DEFAULT NULL,
PRIMARY KEY (`id`)
);
CREATE TABLE `c`
(
`Id` you (11) NOT NULLAUTO_INCREMENT,
`user_id` varchar(50) DEFAULT NULL,
`order_id` varchar(100) DEFAULT NULL,
`state` bigint(20) DEFAULT NULL,
`gmt_create` varchar(30) DEFAULT NULL,
PRIMARY KEY (`id`)
);
Three correlation tables, query the user orders 10 hours before and after the current time, and time of creation in ascending order based on the order, the following specific SQL
select a.seller_id,
a.seller_name,
b.user_name,
c.state
from a,
b,
c
where a.seller_name = b.seller_name
and b.user_id = c.user_id
and c.user_id = 17
and a.gmt_create
BETWEEN DATE_ADD(NOW(), INTERVAL – 600 MINUTE)
AND DATE_ADD(NOW(), INTERVAL 600 MINUTE)
order by a.gmt_create；
Check the amount of data
640?wx_fmt=png
Original execution time of 640? Wx_fmt = png
Original execution plan 640? Wx_fmt = png
Preliminary optimization ideas
SQL, where the conditions consistent with the type field to keep the table structure, the table is user_id varchar (50) type, int type with actual SQL, there is implicit conversion, also without adding an index. The b and c into the table user_id field int.
Due to the presence of b and c associated table table, b and c create an index table user_id
By the presence of a correlation table and table b, a and b are the fields to create index tables seller_name
The use of a composite index eliminate temporary tables and sort
Preliminary optimization SQL
alter table b modify `user_id` int(10) DEFAULT NULL;
alter table c modify `user_id` int(10) DEFAULT NULL;
alter table c add index `idx_user_id`(`user_id`);
alter table b add index `idx_user_id_sell_name`(`user_id`,`seller_name`);
alter table a add index `idx_sellname_gmt_sellid`(`gmt_create`,`seller_name`,`seller_id`);
After a review of the execution time optimization
640?wx_fmt=png
After a review of the implementation of the optimization program 640? Wx_fmt = png
View warnings information 640? Wx_fmt = png
Continue to optimize the alter table a modify "gmt_create" datetime DEFAULT NULL;
View Execution Time
640?wx_fmt=png
View the execution plan 640? Wx_fmt = png
to sum up
View the execution plan explain
If there are alarms, view alarm information show warnings;
View table structure and index information related to SQL
According to the implementation plan, thinking of possible optimization points
According to the table structure changes may perform optimization points, adding indexes, SQL rewriting operation
After viewing the execution plan and optimize the execution time
If the optimization effect is not obvious,