Common strategies for query optimization
1. Optimize data access: applications should reduce data access to the database, and the database should reduce the number of records actually scanned
For example, Redis cache, avoid "select * from table"
2. Rewrite SQL
For operations that require a large amount of data, they can be performed in batches to reduce the impact on the production system, thereby mitigating replication timeouts
MySQL join seriously reduces concurrency, and you should connect as many tables as possible. It is recommended to implement some connection functions at the application layer.
3. Redesign the library table
In the absence of other optimization methods, you can consider changing the table structure design, adding cache tables, temporarily storing statistical data, or adding redundant columns to reduce connections
4. Index
Indexes solve 80% of the problems
Introduction to the Optimizer
Weaknesses of the Optimizer
1. The statistics of the data may be wrong
2. Whether the CPU, memory, and data are in the cache will affect the optimizer
3. The optimizer will not consider the concurrency situation, and Zheng Yong of resources may cause performance problems
hint:
1. Use the index
select * from table1 use index(col1_index,col2_index) where col1=1 and col2=2 and col3=3;
2. Do not use indexes
select * from table1 ignore index(col3_index) where col1=1 and col2=2 and col3=3;;
3. Force the use of indexes
select * from table1 force index(col3_index) where col1=1 and col2=2 and col3=3;;
Note: use index, ignore index, force index will only affect the index to be used for retrieving records and connecting in MySQL tables, not order by or group by or group by
4. Do not use query cache
SQL_NO_CACHE
5. Use the query cache explicit_mode, query_cache_type=2 to indicate that the SQL needs to be cached before it is cached
SQL_CACHE
6 、Straight_join
Join in the order of the tables described by the FROM clause
MySQL connection mechanism
Nested Loop join
The MySQL optimizer generally chooses a small table as the driver table (external table)
Optimization of various statements
connection optimization
1. Do not connect more than 4 tables
2. ON, the column of the using clause must have an index
3. It is best to convert it to inner join. The cost of left join is much higher than that of inner join.
4. Explain to check the connection. If the output rows column is too high, consider whether the order of the index or the connection table is inappropriate
5. Anti-paradigm design
Group by, distinct, order by statement optimization
1. Sort as few rows as possible
2. Multiple tables are connected, and the column of order by should belong to the first table in the connection order
3. Sort by index
4. The columns of group by and order by should be the columns in the first table as much as possible. If not, consider redundant columns
5. Ensure that the index column and the order by column are the same, and they are sorted in the same direction
6. Increase sort_rnd_buffer_size
7. Change the tempdir variable to point to a memory-based filesystem or other faster disk
8. Specify Order by null
By default, MySQL will sort all Group by queries. If you want to avoid sorting the results, you can specify Order by null;
9、优化Group by with rollup
Consider implementing at the application layer
10. Use non-group by columns instead of group by columns
For example, group by x, y, if group by z can achieve the same result, try to minimize group by
11. Consider using Sphinx instead of the group by statement
Optimize subqueries
In most cases, joins will be faster than subqueries, and subqueries generate temporary tables without indexes
select distinct col1 from t1 where col1 in (select col1 from t2);
Rewritten as:
select distinct t1.col1 from t1,t2 where t1.col1=t2.col1;
select * from t1 where id not in (select id from t2);
rewrite:
select * from t1 where not exists (select id from t2 where t1.id=t2.id)
It can also be rewritten as:
select table1.* from table1 left join table2 on table1.id=table2.id where table2.id is null;
Move clauses from outside to inside of subquery
select * from t1 where s1 in (select s1 from t1) or s1 in (select s1 from t2);
rewrite
select * from t1 where s1 in (select s1 from t1 union all select s1 from s2);
select (select column1 from t1) +5 from t2;
rewrite
select (select column1+5 from t1) from t2;
Optimize the limit clause
Optimize IN
Optimize Union
Optimize queries with BLOB, Text type fields
Optimization of filesort
Optimize SQL_CALC_FOUND_ROWS
Optimize temporary tables
OLAP business optimization