Mysql advanced - index optimization and query optimization (1)

Index optimization

1. Data preparation

500,000 entries are inserted into student lists and 10,000 entries are inserted into class lists.

Create table

CREATE TABLE `class` (
	`id` INT ( 11 ) NOT NULL AUTO_INCREMENT,
	`className` VARCHAR ( 30 ) DEFAULT NULL,
	`address` VARCHAR ( 40 ) DEFAULT NULL,
	`monitor` INT NULL,
	PRIMARY KEY ( `id` ) 
) ENGINE = INNODB AUTO_INCREMENT = 1 DEFAULT CHARSET = utf8;
CREATE TABLE `student` (
	`id` INT ( 11 ) NOT NULL AUTO_INCREMENT,
	`stuno` INT NOT NULL,
	`name` VARCHAR ( 20 ) DEFAULT NULL,
	`age` INT ( 3 ) DEFAULT NULL,
	`classId` INT ( 11 ) DEFAULT NULL,
	PRIMARY KEY ( `id` ) #CONSTRAINT `fk_class_id` FOREIGN KEY (`classId`) REFERENCES `t_class` (`id`)

) ENGINE = INNODB AUTO_INCREMENT = 1 DEFAULT CHARSET = utf8;

Setting parameters

Command on: Allow creation of function settings:

set global log_bin_trust_function_creators=1; # 不加global只是当前窗口有效。

Create function

Ensure that each piece of data is different.

#随机产生字符串

DELIMITER //
CREATE FUNCTION rand_string ( n INT ) RETURNS VARCHAR ( 255 ) BEGIN
	DECLARE
		chars_str VARCHAR ( 100 ) DEFAULT 'abcdefghijklmnopqrstuvwxyzABCDEFJHIJKLMNOPQRSTUVWXYZ';
	DECLARE
		return_str VARCHAR ( 255 ) DEFAULT '';
	DECLARE
		i INT DEFAULT 0;
	WHILE
			i < n DO
			
			SET return_str = CONCAT(
				return_str,
			SUBSTRING( chars_str, FLOOR( 1+RAND ()* 52 ), 1 ));
		
		SET i = i + 1;
		
	END WHILE;
	RETURN return_str;
	
END // 
DELIMITER;#假如要删除
#drop function rand_string;

Randomly generate class numbers

#用于随机产生多少到多少的编号

DELIMITER //
CREATE FUNCTION rand_num ( from_num INT, to_num INT ) RETURNS INT ( 11 ) BEGIN
	DECLARE
		i INT DEFAULT 0;
	
	SET i = FLOOR(
		from_num + RAND()*(
			to_num - from_num + 1 
		));
	RETURN i;
	
END // 
DELIMITER;#假如要删除
#drop function rand_num;

Create stored procedure

#创建往stu表中插入数据的存储过程

DELIMITER //
CREATE PROCEDURE insert_stu ( START INT, max_num INT ) BEGIN
	DECLARE
		i INT DEFAULT 0;
	
	SET autocommit = 0;#设置手动提交事务
	REPEAT#循环
		
		SET i = i + 1;#赋值
		INSERT INTO student ( stuno, NAME, age, classId )
		VALUES
			((
					START + i 
					),
				rand_string ( 6 ),
				rand_num ( 1, 50 ),
			rand_num ( 1, 1000 ));
		UNTIL i = max_num 
	END REPEAT;
	COMMIT;#提交事务
	
END // 
DELIMITER;#假如要删除
#drop PROCEDURE insert_stu;

Create a stored procedure to insert data into the class table

#执行存储过程，往class表添加随机数据

DELIMITER //
CREATE PROCEDURE `insert_class` ( max_num INT ) BEGIN
	DECLARE
		i INT DEFAULT 0;
	
	SET autocommit = 0;
	REPEAT
			
			SET i = i + 1;
		INSERT INTO class ( classname, address, monitor )
		VALUES
			(
				rand_string ( 8 ),
				rand_string ( 10 ),
			rand_num ( 1, 100000 ));
		UNTIL i = max_num 
	END REPEAT;
	COMMIT;
	
END // 
DELIMITER;#假如要删除
#drop PROCEDURE insert_class;

Call stored procedure

class

#执行存储过程，往class表添加1万条数据
CALL insert_class(10000);

stu

#执行存储过程，往stu表添加50万条数据
CALL insert_stu(100000,500000);

Delete an index on a table

Create stored procedure

DELIMITER //
CREATE PROCEDURE `proc_drop_index` (
	dbname VARCHAR ( 200 ),
	tablename VARCHAR ( 200 )) BEGIN
	DECLARE
		done INT DEFAULT 0;
	DECLARE
		ct INT DEFAULT 0;
	DECLARE
		_index VARCHAR ( 200 ) DEFAULT '';
	DECLARE
		_cur CURSOR FOR SELECT
		index_name 
	FROM
		information_schema.STATISTICS 
	WHERE
		table_schema = dbname 
		AND table_name = tablename 
		AND seq_in_index = 1 
		AND index_name <> 'PRIMARY';#每个游标必须使用不同的declare continue handler for not found set done=1来控制游标的结束
	DECLARE
		CONTINUE HANDLER FOR NOT FOUND 
		SET done = 2;#若没有数据返回,程序继续,并将变量done设为2
	OPEN _cur;
	FETCH _cur INTO _index;
	WHILE
			_index <> '' DO
			
			SET @str = CONCAT( "drop index ", _index, " on ", tablename );
		PREPARE sql_str 
		FROM
			@str;
		EXECUTE sql_str;
		DEALLOCATE PREPARE sql_str;
		
		SET _index = '';
		FETCH _cur INTO _index;
		
	END WHILE;
	CLOSE _cur;
	
END // 
DELIMITER;

Execute stored procedure

CALL proc_drop_index("dbname","tablename");

2. Index failure case

2.1 Full Value Match My Favorite

2.2 Best left prefix rule

Extension: Alibaba "Java Development Manual"
index file has the leftmost prefix matching feature of B-Tree. If the value on the left is not determined, then this index cannot be used.

2.3 Primary key insertion order

Insert image description here

If you insert another record with a primary key value of 9 at this time, its insertion position will be as shown below:

Insert image description here

But this data page is already full, what should I do if I insert it again? We need to split the current page into two pages and move some records in this page to the newly created page. What do page splits and record shifts mean? Meaning: Performance loss! So if we want to avoid such unnecessary performance loss as much as possible, it is best to increase the primary key value of the inserted record in sequence, so that such performance loss will not occur. So we suggest: let the primary key have AUTO_INCREMENT, and let the storage engine generate the primary key for the table by itself instead of manually inserting it, for example: person_info table:

CREATE TABLE person_info (
	id INT UNSIGNED NOT NULL AUTO_INCREMENT,
	NAME VARCHAR ( 100 ) NOT NULL,
	birthday DATE NOT NULL,
	phone_number CHAR ( 11 ) NOT NULL,
	country VARCHAR ( 100 ) NOT NULL,
	PRIMARY KEY ( id ),
KEY idx_name_birthday_phone_number ( NAME ( 10 ), birthday, phone_number ) 
);

Our custom primary key column ID has AUTO_INCREMENT attributes, and the storage engine will automatically fill in the auto-incremented primary key value for us when inserting records. Such primary keys take up little space, are written sequentially, and reduce page splits.

2.4 Index failure caused by calculation, function, type conversion (automatic or manual)

EXPLAIN SELECT SQL_NO_CACHE * FROM student WHERE student.name LIKE 'abc%';

EXPLAIN SELECT SQL_NO_CACHE * FROM student WHERE LEFT(student.name,3) = 'abc';

Create index

CREATE INDEX idx_name ON student(NAME);

Type 1: Index optimization takes effect

mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM student WHERE student.name LIKE 'abc%';

Second type: index optimization failure

mysql>  EXPLAIN SELECT SQL_NO_CACHE * FROM student WHERE LEFT(student.name,3) = 'abc';
+----+-------------+---------+------------+------+---------------+------+---------+------+--------+----------+-------------+
| id | select_type | table   | partitions | type | possible_keys | key  | key_len | ref  | rows   | filtered | Extra       |
+----+-------------+---------+------------+------+---------------+------+---------+------+--------+----------+-------------+
|  1 | SIMPLE      | student | NULL       | ALL  | NULL          | NULL | NULL    | NULL | 498917 |   100.00 | Using where |
+----+-------------+---------+------------+------+---------------+------+---------+------+--------+----------+-------------+
1 row in set, 2 warnings (0.00 sec)

mysql> SELECT SQL_NO_CACHE * FROM student WHERE LEFT(student.name,3) = 'abc';
+--------+--------+--------+------+---------+
| id     | stuno  | name   | age  | classId |
+--------+--------+--------+------+---------+
|    399 | 100399 | ABcKtL |   24 |     198 |
|  16470 | 116470 | ABcJlg |   47 |     251 |
|  27952 | 127952 | ABcJmj |   10 |     397 |
|  54809 | 154809 | aBClvu |   37 |     495 |
|  61540 | 161540 | abclUS |   30 |     374 |
|  83160 | 183160 | aBCjpV |   34 |     593 |
|  89664 | 189664 | aBCjmJ |   34 |     350 |
| 240498 | 340498 | aBCksj |   41 |     491 |
| 245214 | 345214 | abciJU |   23 |     568 |
| 258459 | 358459 | aBClxC |   23 |     566 |
| 300169 | 400169 | aBClxC |   21 |     412 |
| 300328 | 400328 | ABcJnn |   27 |     870 |
| 324684 | 424684 | aBCkrg |   30 |     566 |
| 416907 | 516907 | ABcHgI |   46 |     607 |
| 424459 | 524459 | abclVU |   39 |     192 |
| 445547 | 545547 | ABcJpw |   16 |     180 |
| 454772 | 554772 | AbCHFf |   37 |     313 |
| 466466 | 566466 | abckRF |   26 |     725 |
| 475708 | 575708 | abclWY |    4 |     415 |
| 486611 | 586611 | ABcLwb |   41 |     948 |
| 490152 | 590152 | ABcHfC |   24 |     717 |
+--------+--------+--------+------+---------+
21 rows in set, 1 warning (0.12 sec)

type is "ALL", indicating that no index is used

2.5 Type conversion causes index failure

Which of the following sql statements can use indexes. (Assume there is an index set on the name field)

# 未使用到索引
mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM student WHERE name=123;
+----+-------------+---------+------------+------+---------------+------+---------+------+--------+----------+-------------+
| id | select_type | table   | partitions | type | possible_keys | key  | key_len | ref  | rows   | filtered | Extra       |
+----+-------------+---------+------------+------+---------------+------+---------+------+--------+----------+-------------+
|  1 | SIMPLE      | student | NULL       | ALL  | idx_name      | NULL | NULL    | NULL | 498917 |    10.00 | Using where |
+----+-------------+---------+------------+------+---------------+------+---------+------+--------+----------+-------------+
1 row in set, 4 warnings (0.00 sec)

# 使用到索引
mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM student WHERE name='123';
+----+-------------+---------+------------+------+---------------+----------+---------+-------+------+----------+-------+
| id | select_type | table   | partitions | type | possible_keys | key      | key_len | ref   | rows | filtered | Extra |
+----+-------------+---------+------------+------+---------------+----------+---------+-------+------+----------+-------+
|  1 | SIMPLE      | student | NULL       | ref  | idx_name      | idx_name | 63      | const |    1 |   100.00 | NULL  |
+----+-------------+---------+------------+------+---------------+----------+---------+-------+------+----------+-------+
1 row in set, 2 warnings (0.00 sec)

Type conversion occurs in name=123 and the index is invalid.

2.6 The column index on the right side of the range condition is invalid

ALTER TABLE student DROP INDEX idx_name;
ALTER TABLE student DROP INDEX idx_age;
ALTER TABLE student DROP INDEX idx_age_classid;
EXPLAIN SELECT SQL_NO_CACHE * FROM student
WHERE student.age=30 AND student.classId>20 AND student.name = 'abc' ;

create index idx_age_name_classid on student(age,name,classid);

Place the range query condition at the end of the statement:

mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM student WHERE student.age=30 AND student.name =
    -> 'abc' AND student.classId>20 ;
+----+-------------+---------+------------+-------+----------------------+----------------------+---------+------+------+----------+-----------------------+
| id | select_type | table   | partitions | type  | possible_keys        | key                  | key_len | ref  | rows | filtered | Extra                 |
+----+-------------+---------+------------+-------+----------------------+----------------------+---------+------+------+----------+-----------------------+
|  1 | SIMPLE      | student | NULL       | range | idx_age_name_classid | idx_age_name_classid | 73      | NULL |    1 |   100.00 | Using index condition |
+----+-------------+---------+------------+-------+----------------------+----------------------+---------+------+------+----------+-----------------------+
1 row in set, 2 warnings (0.00 sec)

2.7 Not equal to (!= or <>) index invalid

2.8 The index can be used for is null, but the index cannot be used for is not null.

EXPLAIN SELECT SQL_NO_CACHE * FROM student WHERE age IS NULL;

EXPLAIN SELECT SQL_NO_CACHE * FROM student WHERE age IS NOT NULL;

2.9 like index starting with wildcard character % is invalid

Expansion: Alibaba "Java Development Manual"
[Mandatory] Left-fuzzy or full-fuzzy page search is strictly prohibited. If necessary, please use a search engine to solve it.

2.10 There are non-indexed columns before and after OR, and the index is invalid.

# 未使用到索引
mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM student WHERE age = 10 OR classid = 100;
+----+-------------+---------+------------+------+----------------------+------+---------+------+--------+----------+-------------+
| id | select_type | table   | partitions | type | possible_keys        | key  | key_len | ref  | rows   | filtered | Extra       |
+----+-------------+---------+------------+------+----------------------+------+---------+------+--------+----------+-------------+
|  1 | SIMPLE      | student | NULL       | ALL  | idx_age_name_classid | NULL | NULL    | NULL | 498917 |    11.88 | Using where |
+----+-------------+---------+------------+------+----------------------+------+---------+------+--------+----------+-------------+
1 row in set, 2 warnings (0.00 sec)

# 使用到索引
mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM student WHERE age = 10 OR name = 'Abel';
+----+-------------+---------+------------+------+----------------------+------+---------+------+--------+----------+-------------+
| id | select_type | table   | partitions | type | possible_keys        | key  | key_len | ref  | rows   | filtered | Extra       |
+----+-------------+---------+------------+------+----------------------+------+---------+------+--------+----------+-------------+
|  1 | SIMPLE      | student | NULL       | ALL  | idx_age_name_classid | NULL | NULL    | NULL | 498917 |    11.88 | Using where |
+----+-------------+---------+------------+------+----------------------+------+---------+------+--------+----------+-------------+
1 row in set, 2 warnings (0.00 sec)

2.11 The character sets of databases and tables use utf8mb4 uniformly.

The unified use of utf8mb4 (supported by version 5.5.3 or above) has better compatibility, and the unified character set can avoid garbled characters caused by character set conversion. Different character sets need to be converted before comparison, which will cause index failure.

3. Related query optimization

3.1 Data preparation

CREATE TABLE IF NOT EXISTS `type` (
`id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`card` INT(10) UNSIGNED NOT NULL,
PRIMARY KEY (`id`)
);
#图书
CREATE TABLE IF NOT EXISTS `book` (
`bookid` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`card` INT(10) UNSIGNED NOT NULL,
PRIMARY KEY (`bookid`)
);

#向分类表中添加20条记录
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO TYPE(card) VALUES(FLOOR(1 + (RAND() * 20)));

#向图书表中添加20条记录
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));
INSERT INTO book(card) VALUES(FLOOR(1 + (RAND() * 20)));

3.2 Use left outer join

mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM `type` LEFT JOIN book ON type.card = book.card;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+----------------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra                                              |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+----------------------------------------------------+
|  1 | SIMPLE      | type  | NULL       | ALL  | NULL          | NULL | NULL    | NULL |   20 |   100.00 | NULL                                               |
|  1 | SIMPLE      | book  | NULL       | ALL  | NULL          | NULL | NULL    | NULL |   20 |   100.00 | Using where; Using join buffer (Block Nested Loop) |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+----------------------------------------------------+
2 rows in set, 2 warnings (0.00 sec)

Conclusion: type has All

Add index optimization

ALTER TABLE book ADD INDEX Y ( card); #【被驱动表】，可以避免全表扫描

mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM `type` LEFT JOIN book ON type.card = book.card;
+----+-------------+-------+------------+------+---------------+------+---------+----------------------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref                  | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+----------------------+------+----------+-------------+
|  1 | SIMPLE      | type  | NULL       | ALL  | NULL          | NULL | NULL    | NULL                 |   20 |   100.00 | NULL        |
|  1 | SIMPLE      | book  | NULL       | ref  | Y             | Y    | 4       | atguigudb2.type.card |    1 |   100.00 | Using index |
+----+-------------+-------+------------+------+---------------+------+---------+----------------------+------+----------+-------------+
2 rows in set, 2 warnings (0.00 sec)

You can see that the type in the second line has changed to ref, and the rows have also changed to an obvious optimization. This is determined by the left join property. The LEFT JOIN
condition is used to determine how to search for rows from the right table. There must be all rows on the left side, so the right side is our key point and must be indexed.

ALTER TABLE `type` ADD INDEX X (card); #【驱动表】，无法避免全表扫描

mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM `type` LEFT JOIN book ON type.card = book.card;
+----+-------------+-------+------------+-------+---------------+------+---------+----------------------+------+----------+-------------+
| id | select_type | table | partitions | type  | possible_keys | key  | key_len | ref                  | rows | filtered | Extra       |
+----+-------------+-------+------------+-------+---------------+------+---------+----------------------+------+----------+-------------+
|  1 | SIMPLE      | type  | NULL       | index | NULL          | X    | 4       | NULL                 |   20 |   100.00 | Using index |
|  1 | SIMPLE      | book  | NULL       | ref   | Y             | Y    | 4       | atguigudb2.type.card |    1 |   100.00 | Using index |
+----+-------------+-------+------------+-------+---------------+------+---------+----------------------+------+----------+-------------+
2 rows in set, 2 warnings (0.00 sec)

DROP INDEX Y ON book;

mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM `type` LEFT JOIN book ON type.card = book.card;
+----+-------------+-------+------------+-------+---------------+------+---------+------+------+----------+----------------------------------------------------+
| id | select_type | table | partitions | type  | possible_keys | key  | key_len | ref  | rows | filtered | Extra                                              |
+----+-------------+-------+------------+-------+---------------+------+---------+------+------+----------+----------------------------------------------------+
|  1 | SIMPLE      | type  | NULL       | index | NULL          | X    | 4       | NULL |   20 |   100.00 | Using index                                        |
|  1 | SIMPLE      | book  | NULL       | ALL   | NULL          | NULL | NULL    | NULL |   20 |   100.00 | Using where; Using join buffer (Block Nested Loop) |
+----+-------------+-------+------------+-------+---------------+------+---------+------+------+----------+----------------------------------------------------+
2 rows in set, 2 warnings (0.00 sec)

3.3 Use inner joins

drop index X on type;
drop index Y on book;

Replace with inner join (MySQL automatically selects the driver table)

mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM type INNER JOIN book ON type.card=book.card;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+----------------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra                                              |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+----------------------------------------------------+
|  1 | SIMPLE      | type  | NULL       | ALL  | NULL          | NULL | NULL    | NULL |   20 |   100.00 | NULL                                               |
|  1 | SIMPLE      | book  | NULL       | ALL  | NULL          | NULL | NULL    | NULL |   20 |    10.00 | Using where; Using join buffer (Block Nested Loop) |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+----------------------------------------------------+
2 rows in set, 2 warnings (0.00 sec)

Add index optimization

ALTER TABLE book ADD INDEX Y ( card);

mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM type INNER JOIN book ON type.card=book.card;
+----+-------------+-------+------------+------+---------------+------+---------+----------------------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref                  | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+----------------------+------+----------+-------------+
|  1 | SIMPLE      | type  | NULL       | ALL  | NULL          | NULL | NULL    | NULL                 |   20 |   100.00 | NULL        |
|  1 | SIMPLE      | book  | NULL       | ref  | Y             | Y    | 4       | atguigudb2.type.card |    1 |   100.00 | Using index |
+----+-------------+-------+------------+------+---------------+------+---------+----------------------+------+----------+-------------+
2 rows in set, 2 warnings (0.00 sec)

ALTER TABLE type ADD INDEX X (card);

mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM type INNER JOIN book ON type.card=book.card;
+----+-------------+-------+------------+-------+---------------+------+---------+----------------------+------+----------+-------------+
| id | select_type | table | partitions | type  | possible_keys | key  | key_len | ref                  | rows | filtered | Extra       |
+----+-------------+-------+------------+-------+---------------+------+---------+----------------------+------+----------+-------------+
|  1 | SIMPLE      | type  | NULL       | index | X             | X    | 4       | NULL                 |   20 |   100.00 | Using index |
|  1 | SIMPLE      | book  | NULL       | ref   | Y             | Y    | 4       | atguigudb2.type.card |    1 |   100.00 | Using index |
+----+-------------+-------+------------+-------+---------------+------+---------+----------------------+------+----------+-------------+
2 rows in set, 2 warnings (0.00 sec)

then:

DROP INDEX X ON `type`;

mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM TYPE INNER JOIN book ON type.card=book.card;
+----+-------------+-------+------------+------+---------------+------+---------+----------------------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref                  | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+----------------------+------+----------+-------------+
|  1 | SIMPLE      | TYPE  | NULL       | ALL  | NULL          | NULL | NULL    | NULL                 |   20 |   100.00 | NULL        |
|  1 | SIMPLE      | book  | NULL       | ref  | Y             | Y    | 4       | atguigudb2.TYPE.card |    1 |   100.00 | Using index |
+----+-------------+-------+------------+------+---------------+------+---------+----------------------+------+----------+-------------+
2 rows in set, 2 warnings (0.00 sec)

then:

ALTER TABLE `type` ADD INDEX X (card);

mysql> EXPLAIN SELECT SQL_NO_CACHE * FROM `type` INNER JOIN book ON type.card=book.card;
+----+-------------+-------+------------+-------+---------------+------+---------+----------------------+------+----------+-------------+
| id | select_type | table | partitions | type  | possible_keys | key  | key_len | ref                  | rows | filtered | Extra       |
+----+-------------+-------+------------+-------+---------------+------+---------+----------------------+------+----------+-------------+
|  1 | SIMPLE      | type  | NULL       | index | X             | X    | 4       | NULL                 |   20 |   100.00 | Using index |
|  1 | SIMPLE      | book  | NULL       | ref   | Y             | Y    | 4       | atguigudb2.type.card |    1 |   100.00 | Using index |
+----+-------------+-------+------------+-------+---------------+------+---------+----------------------+------+----------+-------------+
2 rows in set, 2 warnings (0.00 sec)

3.4 Principle of join statement

Index Nested-Loop Join

EXPLAIN SELECT * FROM t1 STRAIGHT_JOIN t2 ON (t1.a=t2.a);

If you use the join statement directly, the MySQL optimizer may select table t1 or t2 as the driving table, which will affect the execution process of our analysis of the SQL statement. Therefore, in order to facilitate the analysis of performance issues during execution, I instead straight_join asked MySQL to use a fixed connection method to execute the query, so that the optimizer would only join in the way we specified. In this statement, t1 is the driving table and t2 is the driven table.

It can be seen that in this statement, there is an index on field a of the driven table t2, and the join process uses this index, so the execution flow of this statement is as follows:

Read a row of data R from table t1;
From data row R, take out field a and search it in table t2;
Take out the rows that meet the conditions in table t2 and form a row with R as part of the result set;
Repeat steps 1 to 3 until the loop ends at the end of table t1.

This process is to first traverse table t1, and then go to table t2 to find records that meet the conditions based on the a value in each row of data taken from table t1. Formally, this process is similar to the nested query when we write a program, and can use the index of the driven table, so we call it "Index Nested-Loop Join", or NLJ for short.

Its corresponding flow chart is as follows:

Insert image description here

In this process:

A full table scan was performed on the driver table t1. This process required scanning 100 rows;
For each row of R, table t2 is searched based on the a field, using a tree search process. Since the data we construct has a one-to-one correspondence, only one row is scanned in each search process, and a total of 100 rows are scanned;
Therefore, the total number of scanned lines in the entire execution process is 200.

Two conclusions:

Using the join statement, the performance is better than forcibly splitting it into multiple single tables to execute SQL statements;

If you use the join statement, you need to make the small table the driving table.

Simple Nested-Loop Join
Block Nested-Loop Join

Insert image description here

The execution flow chart becomes like this:

Insert image description here

When deciding which table should be used as the driving table, the two tables should be filtered according to their respective conditions. After the filtering is completed, the total data volume of each field participating in the join is calculated. The table with the smaller data volume is the "small table". as a driver table.

Summarize

Ensure that the JOIN field of the driven table has an index created
For fields that require JOIN, the data types must be kept absolutely consistent.
When LEFT JOIN, select the small table as the driving table 大表作为被驱动表. Reduce the number of outer loops.
When INNER JOIN, MySQL will automatically 小结果集的表选为驱动表. Choose to trust MySQL optimization strategies.
If you can directly associate multiple tables, try to associate them directly without using subqueries. (Reduce the number of queries)
It is not recommended to use subqueries. It is recommended to split the subquery SQL and combine the program with multiple queries, or use JOIN instead of subqueries.
The derived table cannot create an index.

4. Subquery optimization

MySQL supports subqueries starting from version 4.1. You can use subqueries to perform nested queries of SELECT statements, that is, the results of one SELECT query serve as the conditions for another SELECT statement. Subqueries can complete many SQL operations that logically require multiple steps to complete in one go .

Subquery is an important function of MySQL, which can help us implement more complex queries through a SQL statement. However, the execution efficiency of subqueries is not high

① When executing a subquery, MySQL needs to create a temporary table for the query results of the inner query statement, and then the outer query statement queries records from the temporary table. After the query is completed, these temporary tables are revoked. This will consume too much CPU and IO resources and generate a large number of slow queries.

② The temporary table stored in the result set of the subquery will not have an index, whether it is a memory temporary table or a disk temporary table, so query performance will be affected to a certain extent.

③ For subqueries that return a larger result set, the impact on query performance will be greater. In MySQL, you can use join (JOIN) queries instead of subqueries. Join queries do not need to create temporary tables and are faster than subqueries. If indexes are used in the query, the performance will be better.

Conclusion: Try not to use NOT IN or NOT EXISTS, use LEFT JOIN xxx ON xx WHERE xx IS NULL instead