After a long-awaited call, MySQL 8.0 indexed the function index of the Three Musketeers

MySQL 8.0 New Features Column Directory

The descending index and invisible index of the three Musketeers of the index have been long-
awaited, MySQL 8.0 index The Three Musketeers function index
dual password, MySQL 8.0 innovative feature
sql_mode compatibility, MySQL 8.0 upgrade has stepped on the pit
beware of parameter changes, MySQL 8.0 upgrade Avoid stepping on the pit again

---

---

foreword

Dugu's nine swords, heavy swords without sharp edges, skillful and inexperienced, well-versed in sword intent, and unable to do anything. The first of the three Musketeers, function index.
The concept of functional index is not new. Oracle has supported functional index as early as ten years ago in Oracle10g. Functional index is widely used and mature in Oracle database, but MySQL has not developed related index functions. But the good news is that MySQL finally introduced this feature in version 8.0. Really, it came out after a lot of calls, but it came anyway.

Ordinary index is to index column value or column prefix value, while MySQL 8.0.13 and later supports functional index. Functional index is to index the structure after the expression calculation is performed on the column in the table, not the column or column prefix value. . Use functional indexes to index data that is not directly stored in the table.

Functional indexes provide applications with great convenience and performance improvements.

---

1. Create and use functional indexes

1.1 Create a functional index

The syntax for creating a functional index is not much different from that of an ordinary index. The only thing to note is that the expression corresponding to the functional index needs to be enclosed in ( ).

First, functional indexes can index a single-column expression, as follows:

# 创建单列表达式索引
alter table t_wang add index idx_func(date(col1));

Second, functional indexes can also index expressions that combine multiple columns, as follows:

# 创建多列组合表达式索引
alter table t_wang add index idx_func((col1 + col2));

Then, functional indexes can also combine expressions and ordinary columns to form a composite index, as follows:

# 创建组合索引
alter table t_wang add index idx_func(col1, (date(col1)));

Finally, functional indexes can also be used with other options, such as unique and asc, desc sorting, as follows:

# 创建表达式排序索引
alter table t_wang add unique index idx_func(col1, (date(col1)) desc);

---

1.2 Using functional indexes to improve performance

Functional indexes can index field expressions, which can significantly improve query performance when expressions are included in SQL statements.

Test Case:

a) Create a test table t_wang, import some test data, and create a common index on the time column

# 查看表结构，测试表的时间列有个普通索引
MySQL [test]> show create table t_wang\G
*************************** 1. row ***************************
       Table: t_wang
 CREATE TABLE `t_wang` (
  `id` int NOT NULL,
  `name` char(30) NOT NULL,
  `fmodify_time` datetime DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `idx_normal_time` (`fmodify_time`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1

b) Query the names of people who have been modified in April; although there is a common index on the time column, check that the execution plan is a full table scan.

# 测试查询4月份有修改过的人名
MySQL [test]> explain select name from t_wang where month(fmodify_time) = 4;
+----+-------------+--------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table  | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+----+-------------+--------+------------+------+---------------+------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | t_wang | NULL       | ALL  | NULL          | NULL | NULL    | NULL |   10 |   100.00 | Using where |
+----+-------------+--------+------------+------+---------------+------+---------+------+------+----------+-------------+

c) Add a functional index to the test table with the expression month(fmodify_time)

# 添加函数索引
alter table t_wang add index `idx_func_time`((month(fmodify_time)));

MySQL [test]> show create table t_wang\G
*************************** 1. row ***************************
       Table: t_wang
Create Table: CREATE TABLE `t_wang` (
  `id` int NOT NULL,
  `name` char(30) NOT NULL,
  `fmodify_time` datetime DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `idx_normal_time` (`fmodify_time`),
  KEY `idx_func_time` ((month(`fmodify_time`)))
) ENGINE=InnoDB DEFAULT CHARSET=latin1

d) Query the names of people who have been modified in April again; the execution plan is index scan

MySQL [test]> explain select name from t_wang where month(fmodify_time) = 4;
+----+-------------+--------+------------+------+---------------+---------------+---------+-------+------+----------+-------+
| id | select_type | table  | partitions | type | possible_keys | key           | key_len | ref   | rows | filtered | Extra |
+----+-------------+--------+------------+------+---------------+---------------+---------+-------+------+----------+-------+
|  1 | SIMPLE      | t_wang | NULL       | ref  | idx_func_time | idx_func_time | 4       | const |    1 |   100.00 | NULL  |
+----+-------------+--------+------------+------+---------------+---------------+---------+-------+------+----------+-------+

---

2. Implementation of functional index

2.1 The specific implementation of function index

MySQL 8.0 functional indexes are actually based on virtual columns (virtual generated columns) introduced in version 5.7. Actually when creating a functional index in MySQL 8.0, MySQL automatically creates a 隐藏的virtual column on the table and then creates the index on the virtual column. Note here: virtual columns do not actually occupy space, but functional indexes need to actually occupy space.

We can view the hidden virtual columns in the following ways. The Extra column has an obvious VIRTUAL GENERATED logo.

# 查看隐藏虚拟列
MySQL [test]> show extended columns from t_wang; 
+----------------------------------+----------+------+-----+---------+-------------------+
| Field                            | Type     | Null | Key | Default | Extra             |
+----------------------------------+----------+------+-----+---------+-------------------+
| id                               | int      | NO   | PRI | NULL    |                   |
| fmodify_time                     | datetime | YES  | MUL | NULL    |                   |
| name                             | char(30) | YES  |     | wang    |                   |
| bd47a6cb20076f31a47803e887053c31 | date     | YES  | MUL | NULL    | VIRTUAL GENERATED |
| DB_TRX_ID                        |          | NO   |     | NULL    |                   |
| DB_ROLL_PTR                      |          | NO   |     | NULL    |                   |
+----------------------------------+----------+------+-----+---------+-------------------+

Since functional indexes are implemented based on virtual columns, some limitations of virtual columns also apply to functional indexes.

• Every time a functional index is added, a virtual column will be added at the same time and it will be included in the limit of the total number of table columns;
• Functional indexes cannot be used as primary key indexes;
• A functional index cannot be a spatial or full-text index;
• Functional indexes cannot be used as foreign key index constraints.

---

2.2 Comparison of function index and prefix index

Prefix index, that is, only the first few characters of the field are indexed, which reduces the index length while optimizing the field query efficiency.

# 创建前缀索引
alter table t_wang add index `idx_prefix` (name(4));

The functional index introduced by MySQL 8.0 can also achieve this ability. Use the SUBSTRING() function to take the first few characters of the field as an expression to build an index.

# 创建函数索引
alter table t_wang add index `idx_substring` ((substring(name, 1, 4));

Prefix index and function index can also index field prefixes, but in order to use function index to improve query performance, you need to use the same function as the function definition in the predicate part of the query statement . That is to say, if we want the execution plan to go to the functional index, the query statement needs to be written like this: select * from t_wang where substring(name, 1, 4) = 'wang' . The prefix index does not have this limitation.

However, with the help of the SUBSTRING() function, functional indexes can even implement indexes on any subset of fields, or even on the intersection and union of different fields. It is more flexible to use and can be applied to various scenarios.

---

3. Application of functional index in JSON data query

Functional indexes provide great convenience for applications. We can optimize query performance, reduce result sets, and reduce data transmission by adjusting query conditions. Functional indexes can also have a similar effect on JSON data access. We can create functional indexes on JSON columns to simplify key-value queries.

However, it should be noted here: the JSON operator ->> corresponds to the JSON_UNQUOTE() function, which returns a string with the collation utf8mb4_bin; and the CAST() function returns a string with the collation utf8mb4_0900_ai_ci (system default collation) string. Therefore, escaping is required when querying to take advantage of functional indexes.

For example, directly check the JSON key->value key-value pair:

# 方式1:
CREATE TABLE employees (
  data JSON,
  INDEX idx ((CAST(data->>"$.name" AS CHAR(30)) COLLATE utf8mb4_bin))
);

SELECT * FROM employees WHERE data->>'$.name' = 'James';

# 或者方式2：
CREATE TABLE employees (
  data JSON,
  INDEX idx ((CAST(data->>"$.name" AS CHAR(30))))
);

SELECT * FROM employees WHERE CAST(data->>'$.name' AS CHAR(30)) = 'James';

---

4. Thinking: Several ways for expressions to optimize queries

Let's assume the following scenario: In a system that allows name change, we want to query the person's name that has been changed in April every year. How many ways can this be achieved?

# 我们使用上面例子中的表结构
 CREATE TABLE `t_wang` (
  `id` int NOT NULL,
  `name` char(30) NOT NULL,
  `fmodify_time` datetime DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1

Method 1: Use a common index on the time column

We can add a common index to the time column, and then traverse the data in April every year since the system was launched.

# 在fmodify_time列加个普通索引
alter table t_wang add index idx_normal_time(fmodify_time);

# 对应查询语句
select name from t_wang where (fmodify_time between '2015-04-01 00:00:00' and '2015-05-01 00:00:00') or (fmodify_time between '2016-04-01 00:00:00' and '2016-05-01 00:00:00') or ... or (fmodify_time between '2022-04-01 00:00:00' and '2022-05-01 00:00:00');   

Method 2: Use virtual columns and add indexes on virtual columns

We can still use MySQL 5.7's virtual columns to optimize queries, add a virtual column on the table, and then add a normal index on the virtual column.

# 添加虚拟列和对应索引
alter table t_wang add column `ftime_generated` int GENERATED ALWAYS AS (month(fmodify_time));
alter table t_wang add index `idx_generated_time` (ftime_generated);

# 对应查看语句
select name from t_wang where ftime_generated = 4;

Method 3: Use function index

We can use the functional index introduced in MySQL 8.0 to add a functional index on the time column.

# 添加函数索引
alter table t_wang add index `idx_functional_time` ((month(fmodify_time)));

# 对应查询语句
select name from t_wang where month(fmodify_time) = 4;

Method 4: Use the default value of the expression

In fact, we can also have another way to optimize the query, also in MySQL 8.0 also introduced another feature, expression default value .

Starting with MySQL 8.0.13, the default value specified in the field's DEFAULT clause can be a constant or an expression. Using column-based expressions to evaluate values ​​as default values ​​enables virtual column-like capabilities.

# 向表中添加一列，将时间列的表达式作为该列的默认值；然后再在该列添加一个普通索引。
alter table t_wang add column `ftime_default` int NOT NULL DEFAULT (month(fmodify_time));
alter table t_wang add index `idx_default_time` (ftime_default);

# 对应查询语句
select name from t_wang where ftime_default = 4;

---

Summarize

MySQL 8.0 introduces a new feature of functional index, which improves the convenience and performance of business programs. With functional indexes, businesses can enjoy the performance improvement brought by virtual columns without manually adding virtual columns to the table. Compared with other similar implementations, such as virtual columns and expression defaults, functional indexes are more concise and easier to maintain. Functional indexes can also be used for querying JSON data. The new features introduced by MySQL 8.0 in terms of indexes are not only functional indexes, but also inverted indexes and invisible indexes. These features are also of great benefit to business queries.

喜欢的同学麻烦点个关注和点赞