Mysql advanced - index creation and use

Index creation

1. Declaration and use of index

1.1 Classification of indexes

MySQL's indexes include ordinary indexes, unique indexes, full-text indexes, single-column indexes, multi-column indexes and spatial indexes, etc.

From a functional logic perspective, there are four main types of indexes, namely ordinary indexes, unique indexes, primary key indexes, and full-text indexes.

According to physical implementation, indexes can be divided into two types: clustered indexes and non-clustered indexes.

It is divided according to the number of active fields and divided into single column index and joint index.

1. Ordinary index
2. unique index
3. primary key index
4. Single column index
5. Multi-column (combination, union) index
6. Full text index
7. Supplement: spatial index

Summary: Different storage engines support different index types. InnoDB: supports B-tree, Full-text and other indexes, but does not support Hash indexes; MyISAM: supports
B-tree, Full-text and other indexes, but does not support Hash indexes; Memory : Supports B-tree, Hash and other
indexes, does not support Full-text index; NDB: supports Hash index, does not support B-tree, Full-text and other indexes; Archive: does not
support Hash, Full-text and other indexes ;

1.2 Create index

CREATE TABLE dept(
dept_id INT PRIMARY KEY AUTO_INCREMENT,
dept_name VARCHAR(20)
);
CREATE TABLE emp(
emp_id INT PRIMARY KEY AUTO_INCREMENT,
emp_name VARCHAR(20) UNIQUE,
dept_id INT,
CONSTRAINT emp_dept_id_fk FOREIGN KEY(dept_id) REFERENCES dept(dept_id)
);	

If you create an index when explicitly creating a table, the basic syntax format is as follows:

CREATE TABLE table_name [col_name data_type]
[UNIQUE | FULLTEXT | SPATIAL] [INDEX | KEY] [index_name] (col_name [length]) [ASC |
DESC]

• UNIQUE, FULLTEXT and SPATIAL are optional parameters, representing unique index, full-text index and spatial index respectively;
• INDEX and KEY are synonyms. They have the same function and are used to specify index creation;
• index_name specifies the name of the index, which is an optional parameter. If not specified, MySQL defaults to col_name as the index name;
• col_name is the field column that needs to be indexed, and this column must be selected from multiple columns defined in the data table;
• length is an optional parameter, indicating the length of the index. Only string type fields can specify the index length;
• ASC or DESC specifies index value storage in ascending or descending order.

Create a normal index

Create a normal index on the year_publication field in the book table. The SQL statement is as follows:

CREATE TABLE book(
    book_id INT ,
    book_name VARCHAR(100),
    authors VARCHAR(100),
    info VARCHAR(100) ,
    comment VARCHAR(100),
    year_publication YEAR,
    INDEX(year_publication)
);

Create unique index

CREATE TABLE test1(
    id INT NOT NULL,
    name varchar(30) NOT NULL,
    UNIQUE INDEX uk_idx_id(id)
);

primary key index

After setting the primary key, the database will automatically create an index. Innodb is a clustered index. Syntax:

• Create an index along with the table:

CREATE TABLE student (
    id INT(10) UNSIGNED AUTO_INCREMENT ,
    student_no VARCHAR(200),
    student_name VARCHAR(200),
    PRIMARY KEY(id)
);

• Delete primary key index:

ALTER TABLE student
drop PRIMARY KEY ;

• Modify the primary key index: you must first delete (drop) the original index, and then create (add) the index

Create a single column index

CREATE TABLE test2(
    id INT NOT NULL,
    name CHAR(50) NULL,
    INDEX single_idx_name(name(20))
);

Create a composite index

Example: Create table test3 and create a combined index on the id, name and age fields in the table. The SQL statement is as follows:

CREATE TABLE test3(
    id INT(11) NOT NULL,
    name CHAR(30) NOT NULL,
    age INT(11) NOT NULL,
    info VARCHAR(255),
    INDEX multi_idx(id,name,age)
);

Create a full-text index

Example 1: Create table test4 and create a full-text index on the info field in the table. The SQL statement is as follows:

CREATE TABLE test4(
    id INT NOT NULL,
    name CHAR(30) NOT NULL,
    age INT NOT NULL,
    info VARCHAR(255),
    FULLTEXT INDEX futxt_idx_info(info)
) ENGINE=MyISAM;

In MySQL 5.7 and later versions, you do not need to specify the last ENGINE because InnoDB supports full-text indexing in this version.

Created a table with full-text index added to title and body fields.

CREATE TABLE `papers` (
    `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
    `title` varchar(200) DEFAULT NULL,
    `content` text,
    PRIMARY KEY (`id`),
    FULLTEXT KEY `title` (`title`,`content`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;

Query different from like method:

SELECT * FROM papers WHERE content LIKE ‘%查询字符串%’;

Full-text index query using match+against method:

SELECT * FROM papers WHERE MATCH(title,content) AGAINST (‘查询字符串’);

important point

1. Before using full-text indexing, find out the version support status;
2. Full-text indexing is N times faster than like + %, but there may be accuracy issues;
3. If a large amount of data needs to be indexed in full text, it is recommended to add the data first and then create the index.

Create spatial index

When creating a spatial index, fields of spatial type must be non-null.

Example: Create table test5 and create a spatial index on the field with spatial type GEOMETRY. The SQL statement is as follows:

CREATE TABLE test5(
    geo GEOMETRY NOT NULL,
    SPATIAL INDEX spa_idx_geo(geo)
) ENGINE=MyISAM;

2. Create an index on an existing table

To create an index on an existing table, use the ALTER TABLE statement or the CREATE INDEX statement.

1. Create an index using the ALTER TABLE statement. The basic syntax of the ALTER TABLE statement to create an index is as follows:

ALTER TABLE table_name ADD [UNIQUE | FULLTEXT | SPATIAL] [INDEX | KEY]
[index_name] (col_name[length],...) [ASC | DESC]

2. Use CREATE INDEX to create an index. The CREATE INDEX statement can add an index to an existing table. In MySQL,
   CREATE INDEX is mapped to an ALTER TABLE statement. The basic syntax structure is:

CREATE [UNIQUE | FULLTEXT | SPATIAL] INDEX index_name
ON table_name (col_name[length],...) [ASC | DESC]

2.1 Delete index

1. Use ALTER TABLE to delete an index. The basic syntax format of ALTER TABLE to delete an index is as follows:

ALTER TABLE table_name DROP INDEX index_name;

2. Use the DROP INDEX statement to delete an index. The basic syntax format of DROP INDEX to delete an index is as follows:

DROP INDEX index_name ON table_name;

Tip: When you delete a column in a table, if the column you want to delete is part of an index, the column is also deleted from the index. If
all columns that make up the index are dropped, the entire index will be dropped.

3.New features of MySQL8.0 index

Support descending index

Create data table ts1 in MySQL version 5.7 and MySQL version 8.0 respectively. The results are as follows:

CREATE TABLE ts1(a int,b int,index idx_a_b(a,b desc));

Check the structure of data table ts1 in MySQL version 5.7. The results are as follows:

Check the structure of data table ts1 in MySQL version 8.0. The results are as follows:

As can be seen from the results, the index is already in descending order. Let's continue to test the performance of descending index in the execution plan.

Insert 800 pieces of random data into the data table ts1 of MySQL version 5.7 and MySQL version 8.0 respectively. The execution statement is as follows:

DELIMITER //
    CREATE PROCEDURE ts_insert()
    BEGIN
    DECLARE i INT DEFAULT 1;
    WHILE i < 800
    DO
    insert into ts1 select rand()*80000,rand()*80000;
    SET i = i + 1;
    END WHILE;
    commit;
    END //
    DELIMITER ;
#调用
CALL ts_insert();

Check the execution plan of data table ts1 in MySQL version 5.7. The results are as follows:

EXPLAIN SELECT * FROM ts1 ORDER BY a,b DESC LIMIT 5;

As can be seen from the results, the number of scans in the execution plan is 799, and Using filesort is used.

Tip: Using filesort is a relatively slow external sort in MySQL, and it is best to avoid it. In most cases, administrators
can optimize indexes to avoid Using filesort, thereby improving database execution speed.

View the execution plan of data table ts1 in MySQL version 8.0. As can be seen from the results, the number of scans in the execution plan is 5, and
Using filesort is not used.

Note that the descending index is only effective for the specific sort order in the query. If used improperly, the query efficiency will be lower. For example, if the above
query sorting condition is changed to order by a desc, b desc, the execution plan of MySQL 5.7 is significantly better than that of MySQL 8.0.

After changing the sorting conditions to order by a desc, b desc, let’s compare the effects of execution plans in different versions.
Check the execution plan of data table ts1 in MySQL version 5.7 . The results are as follows:

EXPLAIN SELECT * FROM ts1 ORDER BY a DESC,b DESC LIMIT 5;

View the execution plan of data table ts1 in MySQL version 8.0.
It can be seen from the results that the execution plan of MySQL 5.7 after modification is significantly better than that of MySQL 8.0.

Hidden index

In MySQL 5.7 and earlier, indexes can only be deleted explicitly. At this time, if an error occurs after deleting the index, the deleted index can only be created back by explicitly creating the index. If the amount of data in the data table is very large, or the data table itself is relatively large, this operation will consume too many system resources and the operation cost will be very high.

Hidden indexes (invisible indexes) are supported starting from MySQL 8. The index will not be used), confirm that the system will not respond after setting the index to hidden index, and you can delete the index completely. This method of first setting the index to a hidden index and then deleting the index is soft deletion.

Create a hidden index directly when creating a table. Creating a hidden index in MySQL is achieved through the SQL statement INVISIBLE. Its syntax is as follows:

CREATE TABLE tablename(
    propname1 type1[CONSTRAINT1],
    propname2 type2[CONSTRAINT2],
    ……
    propnamen typen,
    INDEX [indexname](propname1 [(length)]) INVISIBLE
);

The above statement has one more keyword INVISIBLE than the ordinary index, which is used to mark the index as an invisible index.

Create on existing table

Hidden indexes can be set for existing tables. The syntax is as follows:

CREATE INDEX indexname
ON tablename(propname[(length)]) INVISIBLE;

Created via ALTER TABLE statement

The syntax is as follows:

ALTER TABLE tablename
ADD INDEX indexname (propname [(length)]) INVISIBLE;

Switch the visible state of the index. An existing index can switch the visible state through the following statement:

ALTER TABLE tablename ALTER INDEX index_name INVISIBLE; #切换成隐藏索引
ALTER TABLE tablename ALTER INDEX index_name VISIBLE; #切换成非隐藏索引

If you switch the index_cname index to the visible state and view the execution plan through explain, you will find that the optimizer selected the index_cname index.

Note that when an index is hidden, its contents are still updated in real time like a normal index. If an index needs to be hidden for a long time
, it can be deleted because the existence of the index will affect the performance of inserts, updates, and deletes.

You can see how indexes can help with tuning by setting the visibility of hidden indexes.

In the MySQL 8.x version, a new way to test indexes is provided. You can turn on a certain setting through a switch of the query optimizer (use_invisible_indexes) to make hidden indexes visible to the query optimizer. If use_invisible_indexes is set to off (default), the optimizer ignores hidden indexes. If set to on, the optimizer still considers hidden indexes when generating execution plans, even if they are not visible.

(1) Execute the following command on the MySQL command line to view the switch settings of the query optimizer.

mysql> select @@optimizer_switch \G

Find the following attribute configuration in the output result information.

use_invisible_indexes=off

The configuration value of this attribute is off, which means that the hidden index is not visible to the query optimizer by default.

(2) To make the hidden index visible to the query optimizer, you need to execute the following command on the MySQL command line:

mysql> set session optimizer_switch="use_invisible_indexes=on";
Query OK, 0 rows affected (0.00 sec)

The SQL statement is executed successfully. Check the query optimizer switch settings again.

mysql> select @@optimizer_switch \G
*************************** 1. row ***************************
@@optimizer_switch:
index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_
intersection=on,engine_condition_pushdown=on,index_condition_pushdown=on,mrr=on,mrr_co
st_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on
,loosescan=on,firstmatch=on,duplicateweedout=on,subquery_materialization_cost_based=on
,use_index_extensions=on,condition_fanout_filter=on,derived_merge=on,use_invisible_ind
exes=on,skip_scan=on,hash_join=on
1 row in set (0.00 sec)

At this time, you can see the following attribute configuration in the output result.

use_invisible_indexes=on

The value of the use_invisible_indexes attribute is on, indicating that the hidden index is visible to the query optimizer at this time.

(3) Use EXPLAIN to view the index usage when the field invisible_column is used as the query condition.

explain select * from classes where cname = '高一2班';

The query optimizer uses hidden indexes to query data.

(4) If you need to make the hidden index invisible to the query optimizer, you only need to execute the following command.

mysql> set session optimizer_switch="use_invisible_indexes=off";
Query OK, 0 rows affected (0.00 sec)

Check the query optimizer switch settings again.

mysql> select @@optimizer_switch \G

At this time, the value of the use_invisible_indexes attribute has been set to "off".