Foreword:
An essential skill for excellent development: performance optimization, including: JVM tuning , caching, Sql performance optimization, etc. This article mainly talks about index optimization based on Mysql.
First of all, we need to understand the processing process of Mysql when executing a query SQL:
Secondly, we need to know, what is the execution order of the SQL we write in Mysql? The execution order of sql is very helpful and important for sql performance optimization. This point needs to be taken into consideration when building a composite index.
Example:
Create a composite index idx_parent_id_code in tb_dept:
Then look at the results of the two SQL interpretations:
1) Under the current index, which SQL index has a high utilization rate?
With the help of the execution order of query SQL above, WHERE is executed first and then GROUP BY is executed, namely:
The order of execution of the first SQL is to execute parent_id after where and then dept_code after group by. The order is the same as the order of the index, the type level is ref, and the number of scanned rows is 4;
The second SQL is to execute dept_code after where and then parent_id after group by. The order is inconsistent with the order of the index. The type level is index, and the number of scanned rows is 19;
From the interpretation results, the first SQL index utilization rate is higher than the second. (Later I will talk about: Index type from good to bad: System--> const-->eq_ref-->ref -->ref_or_null-->index_merge-->unique_subquery-->index_subquery--> range-->index -->all .)
Or comparing the number of rows scanned by the data source can also be intuitively seen, the performance of the two statements:
2) How to optimize?
If the second SQL is used in the business, you need to adjust the order of the indexes to be consistent with the SQL execution order.
Or both sql are used, then create a composite index (idx_code_parent_id)
Then look at the second execution plan:
Execution plan analysis (the following is the focus of this article):
Through explain, you can know how mysql processes statements, and analyze the performance bottleneck of the query or table structure. In fact, it is doing the query optimizer. You can get through expalin:
1. Table reading order
2. Table read operation operation type
3. Which indexes can be used
4. Which indexes are actually used
5. References between tables
6. How many rows in each table are queried by the optimizer
From the above example, we can see that when the explain is executed, there will be a table as a result. This table is the analysis result. Let's explain the header of this table one by one:
Id: The sequence number of the query in the execution plan selected by MySQL QueryOptimizer. Indicates the order in which select clauses or operation tables are executed in the query. The larger the id value, the higher the priority, and the earlier it will be executed. The id is the same, and the execution order is from top to bottom.
Select_type: There are 9 types in total, only 4 commonly used types are introduced:
SIMPLE: simple select query, no union and subquery
PRIMARY: the outermost select query
UNION: The second or subsequent select query in UNION does not depend on the result set of the external query
DERIVED: Used when there is a subquery in the from clause. MySQL will execute these subqueries recursively, putting the results in a temporary table.
Table: The table referenced by the output row
Type: The order from excellent to poor is as follows: (The red mark is the common level.)
system-->const-->eq_ref-->ref-->ref_or_null-->index_merge-->unique_subquery-->index_subquery-->range-->index-->all.
The meaning of each is as follows:
system: The table has only one row. This is a special case of the const connection type.
const: const is used when comparing PRIMARY KEY with a constant value.
eq_ref: Used when the query uses all of the primary key or unique key of the index. That is: it is possible to find a row that meets the conditions through the index key.
ref: It is possible to find multiple qualified rows by index key.
ref_or_null: Same as ref, but MySQL must find the null entry in the result of the first search, and then perform a second search.
index_merge: indicates that index merge optimization is used.
unique_subquery: Use this type in some IN queries instead of regular ref:valueIN (SELECT primary_key FROM single_table WHERE some_expr)
index_subquery: Use this type in some IN queries, similar to unique_subquery, but the query is a non-unique index
range: Retrieve rows in a given range. When using <>, >, >=, <, <=, BETWEEN or IN operator, range will be used.
index: Full table scan, but scan the table according to the index order instead of rows. The main advantage is to avoid sorting, but the overhead is still very large.
all: In the worst case, scan the entire table from beginning to end.
possible_keys: Which indexes might help the query. If it is empty, there is no index available.
key: Actually select the index to be used from possible_key. If it is NULL, the index is not used. In rare cases, MYSQL will choose an under-optimized index. In this case, you can use USE INDEX (indexname) in the SELECT statement to force the use of an index or IGNORE INDEX (indexname) to force MYSQL to ignore the index
key_len: The length of the index used. Without loss of accuracy, the shorter the length, the better.
ref: show which column of the index is used
rows: The approximate number of rows returned by the requested data
extra: Other information. The presence of Using filesort and Using temporary means that the index cannot be used, and the efficiency will be greatly affected. This should be optimized as much as possible.
Using filesort: There is no way to use the existing index for sorting. Additional sorting is required. Suggestion: Create a corresponding appropriate index according to the sorting needs
Using temporary: Need to use a temporary table to store the result set, usually because there is no index on the group by column. It may also be because
there are group by and order by at the same time, but the columns of group by and order by are differentUsing index: Using the covering index, the result data can be obtained without returning to the table (that is, the data is directly read from the index file). This result is good.
The important ones are key, type, rows, and extra . When the key is null, all, or index, the index needs to be adjusted and optimized. Generally, it needs to reach the ref and eq_ref level, and the range search needs to reach the range. Extra has Using filesort and Using temporary which must be optimized. According to rows, the optimization result can be seen directly.
Optimization methods:
① SQL optimization
- Avoid SELECT * and only query the required fields.
- Small tables drive large tables, that is, small data sets drive large data sets:
when the data set of table B is smaller than table A, use in to optimize the execution order of the two tables by first checking table B and then table A. Query statement: SELECT * FROM tb_dept WHERE id in (SELECT id FROM tb_dept);
When the data set of table A is smaller than table B, use exist to optimize in. The order of execution of the two tables is to check table A first, then table B, query statement: SELECT * FROM A WHERE EXISTS (SELECT id FROM B WHERE A.id = B.ID);- Try to use joins instead of subqueries, because when using joins, MySQL will not create temporary tables in memory.
② Optimize the use of indexes
- Try to use primary key queries instead of other indexes, because primary key queries will not trigger back table queries.
- Do not do column calculations, put calculations into each business system to achieve
- The query statement is as simple as possible, and the large statement is split into the small statement to reduce the lock time
- or query is rewritten into union query
- No functions and triggers
- To avoid %xx query, you can use: select * from t where reverse(f) like reverse('%abc');
- Less use of join queries
- Use the same type of comparison, such as '123' and '123', 123 and 123
- Try to avoid using the != or <> operator in the where clause, the query reference will abandon the index and perform a full table scan
- List data use paging query, the amount of data per page should not be too large
- Avoid using is null and is not null on index columns
③ Table structure design optimization
- Use the smallest data type that can store data.
- Try to use tinyint, smallint, mediumint as integer types instead of int.
- Use not null to define fields as much as possible, because null occupies 4 bytes. The number can default to 0, and the string defaults to ""
- Use text type as little as possible, and it is better to create a separate table when it is absolutely necessary.
- Try to use timestamp instead of datetime.
- There should not be too many fields in a single table, and it is recommended to be within 20 fields.
The storage size and scope of common data types in Mysql: https://blog.csdn.net/HXNLYW/article/details/100104768
3. If there are still problems with the above optimization, you can use show profiles to analyze SQL performance
show profiles
show profile for query [queryId]
For details, please check: https://blog.csdn.net/aeolus_pu/article/details/7818498
end:
This article is a summary and record of recent learning about Mysql index optimization. If there is something wrong, please comment on it.
Attach:
Index related knowledge:
———— View table index:
show index from [table]———— Create index directly
CREATE INDEX indexName ON table(column(length))———— Modify the table structure to add an index
ALTER tableADD INDEX indexName ON (column(length))
---Primary key index
ALTER TABLE `table_name` ADD PRIMARY KEY (`column`)
---Unique index
ALTER TABLE `table_name` ADD UNIQUE (`column`)
---Ordinary index
ALTER TABLE `table_name` ADD INDEX index_name (`column`(length))
---Composite index
ALTER TABLE `table_name` ADD INDEX index_name (`column1`, `column2`, `column3`)Determination of length:
If the length of the index column is too long, this kind of column index will produce a large index file, which is inconvenient to operate. You can use the prefix index method for indexing. The prefix index should be controlled at a suitable point, controlled at 0.31 Gold value is sufficient (more than this value can be created).
SELECT COUNT(DISTINCT(LEFT(`title`,10)))/COUNT(*) FROM Arctic; - If this value is greater than 0.31, you can create a prefix index and Distinct to repeat———— 删除索引：
1）ALTER TABLE table_name DROP INDEX index_name
2）DROP INDEX index_name ON table_name;
The difference between MyISAM and InnoBD:
| |
MyISAM |
InnoDB |
| Primary key | Allow tables without any indexes and primary keys to exist, The index of myisam is the address of the saved row. |
If the primary key is not set or a non-empty unique index is not set, a 6-byte primary key will be automatically generated (not visible to the user) Innodb data is part of the main index, and other indexes store the value of the main index. |
| Transaction processing aspects: |
The MyISAM type table emphasizes performance, and its execution is faster than InnoDB type, but it does not provide transaction support or foreign keys. | InnoDB provides transaction support transaction, foreign key (foreign key) and other advanced database functions |
| DML operation |
If you perform a large number of SELECTs, MyISAM is a better choice |
1. If your data performs a large number of INSERT or UPDATE , for performance considerations, you should use InnoDB tables. 2. When DELETE FROM table , InnoDB will not re-create the table, but delete row by row. |
| Automatic growth |
The auto-growth column of the myisam engine must be an index. If it is a composite index, the auto-growth may not be the first column. It can be sorted and incremented according to the previous columns. |
The automatic growth of the innodb engine must be an index. If it is a composite index, it must also be the first column of the composite index. |
| count() function | myisam saves the total number of rows of the table, if select count(*) from table; will directly take out the value | Innodb does not save the total number of rows in the table. If you use select count(*) from table; it will traverse the entire table, which consumes a lot of money, but after adding the wehre condition, myisam and innodb handle the same way. |
| lock |
Table lock |
Provide row locks. In addition, the row locks of InnoDB tables are not absolute. If MySQL cannot determine the range to be scanned when executing a SQL statement, InnoDB tables will also lock the entire table, such as update table set num=1 where name like " %aaa%" |
Optimization of mysql related configuration parameters:
• sort-buffer-size/join-buffer-size / read-rnd-buffer-size，4～8MB为宜
• optimizer_switch=“index_condition_pushdown=on,mrr=on,mrr_cost
_based=off,batched_key_access=on”
• tmp-table-size = max-heap-table-size，100MB左右为宜
• log-queries-not-using-indexes & log_throttle_queries_not_using_indexes