Why exactly is the TRUNCATE statement slow?

Yuan Chuanghui, offline restart! July 1, 2023 Shenzhen Station - Basic software technology interview! Free tickets are on sale for a limited time only!

The author analyzes the reason and solution of the slow SQL formed by the truncate statement through the source code, and compares it with MySQL 5.7 on the relevant implementation logic.

Author: Li Xichao

A database engineer of Jiangsu Suning Bank who loves to laugh, mainly responsible for the daily operation and maintenance of the database, automation construction, and DMP platform operation and maintenance. Good at MySQL, Python, Oracle, like cycling and research technology.

Source of this article: original contribution

• Produced by the Aikesheng open source community, the original content is not allowed to be used without authorization, please contact the editor and indicate the source for reprinting.

problem phenomenon

Feedback was received that when a test environment executes batch operations, there are truncatestatements in the slow query log. I am worried that the database may be affected after going online, so I request the DBA to cooperate with the analysis.

key configuration

configuration item	illustrate
database version	MySQL 5.7
Parameter long_query_time<br/> Slow query threshold, in seconds	0.1 (100 milliseconds)
parameterinnodb_adaptive_hash_index	ON

Problem Analysis Summary

In summary, there are several main questions:

Q1: How is the TRUNCATE statement executed? The fd handle doesn't change? Why is the execution time long?

How is the TRUNCATE statement executed?

critical stack:

Key operation debug:

Why is the execution time long?

As can be seen from the above stack, the time-consuming process is mainly row_drop_table_for_mysql, os_file_delete_func.

Among them: row_drop_table_for_mysqlmainly call btr_drop_ahi_for_tableto delete the page of AHI. os_file_delete_funcMain call to unlinkperform file cleanup.

Why doesn't the handle change?

If the fd allocated to the truncaterequired table is 43, truncatethe table will be allocated first during the process rename. At this time, the fd will be closed, and 43 will be released. Then perform create tablethe action . Generally, this gap process is very short, so the newly created table can use the released 43, so you will see no change in fd.

renameIf create tablea new file is opened after and before the internal execution of , then fd 43 will be held by other opened files at this time, and truncatethen the fd of the table will also change.

Note: MySQL 8.0 is really implemented using rename++ , but MySQL 5.7 is implemented .createdroptruncatetruncate

Q2: How to analyze the problem of slow TRUNCATE?

Method 1: Slow log?

Only slow results can be seen, and the cause cannot be confirmed.

Method 2: Execute the plan?

truncatestatement is not supported .

Method 3: profile

Judging from profilethe results , for truncatethe statement, we can only see that the time-consuming process is System lockon , and we cannot see the reason for the further step.

Method 4: DEBUG

// 推荐设置
// 其中 T 其实是 MySQL 支持（在 trace 中打印时间）的，但官方文档中缺少了说明。已提交bug说明：Bug #111174

set global debug='d:t:T:i:n:N:o,/tmp/debug_3306.trace.f';
set global debug='';

• ① represents the thread ID show processlistof
• ② Execution time
• ③ Function call level
• ④ Function name

MySQL 8.0 switching comparison

// TRUNCATE
// 默认规范配置
// innodb_flush_method = on & innodb_flush_method = O_DIRECT
([email protected]) [eolbimsdb] 08:44:46 15> truncate table t5;
Query OK, 0 rows affected (0.98 sec)

// 设置 innodb_adaptive_hash_index = off
([email protected]) [eolbimsdb] 08:52:03 5> truncate table t5;
Query OK, 0 rows affected (0.03 sec)

// 设置 innodb_flush_method = fsync
([email protected]) [eolbimsdb] 09:03:34 28> truncate table t5;
Query OK, 0 rows affected (1.04 sec)

// 设置  innodb_adaptive_hash_index = off & innodb_flush_method = fsync
([email protected]) [eolbimsdb] 09:20:24 5> truncate table t5;
Query OK, 0 rows affected (0.22 sec)


// DROP
// 默认规范配置
// innodb_flush_method = on & innodb_flush_method = O_DIRECT
([email protected]) [eolbimsdb] 10:05:41 9> drop table t5;
Query OK, 0 rows affected (0.94 sec)


// 设置 innodb_adaptive_hash_index = off & innodb_flush_method = O_DIRECT
([email protected]) [eolbimsdb] 09:44:24 5> drop table t5;
Query OK, 0 rows affected (0.01 sec)

// 设置 innodb_flush_method = on & innodb_flush_method = fsync
([email protected]) [eolbimsdb] 09:32:15 13> drop table t5;
Query OK, 0 rows affected (1.13 sec)

// 设置  innodb_adaptive_hash_index = off & innodb_flush_method = fsync
([email protected]) [eolbimsdb] 09:25:10 14> drop table t5;
Query OK, 0 rows affected (0.19 sec)

Q3: Can it be optimized? Where is the slowness? How post_ddl is called?

It can be seen from the results of Q1 that the main execution time row_drop_table_for_mysqlis os_file_delete_func:

Optimization measures for MySQL 8.0

1. row_drop_table_for_mysqlThe problem of slowness can innodb_adaptive_hash_index = offbe ;
2. os_file_delete_funcSlow problem can be optimized by setting innodb_flush_method = O_DIRECTor configuring the HARD LINK of the table.

Optimizations in MySQL 5.7

Please refer to the following sections 3-Q1 and 3-Q4 for details.

post_ddlHow to call?

MySQL 8.0 introduces the scope guard function: when the scope guard is defined, a Scope_guard object will be created. Normally, the logic defined by the scope guard will be executed before returnthe operation . Unless committhe operation . The file deletion function is actually called in the cleanup_base phase of scope guard.

Q4: Are there hidden dangers in executing TRUNCATE in production?

From the perspective of the implementation mechanism, the main risks are as follows:

IO pressure

When truncatethe operation , the file needs to be processed by the database thread in a short time unlinkor truncate, if the processed file is large, the IO pressure of the server may affect the normal database request.

memory concurrency

truncateIn the process of executing drop, due to the need to clean up the data in the memory, especially scan the LRU and flush_LRU, and release the corresponding data blocks. This process needs to buffer pool instanceobtain . If it is during a peak business period, especially if it is buffer poollarge , it may affect normal business conditions.

At the same time, the X lock (RW_X_LATCH) is required to perform create drop tablethe operation dict_operation_lock, and some other background threads, such as the Main Thread, also need to dict_operation_lockacquire , so they are blocked. Then the user thread may be in a suspended state because it cannot acquire the lock, when the lock cannot be acquired immediately. More reference: "Drop Table Performance Impact Analysis on MySQL" .

Q5: Are there any differences in the implementation of TRUNCATE in different versions?

By comparing 2-Q1 with 3-Q4:

truncateThe implementation of MySQL 8.0 is basically dropthe same as the implementation of , including the main time-consuming positions (all in row_drop_table_for_mysql, os_file_delete_func) are the same.

The implementation of truncateand dropis quite different, and the entire implementation process is almost completely independent code. truncateUse row_truncate_table_for_mysql, dropuse row_drop_table_for_mysql; truncatethe main time-consuming operations are dict_drop_index_tree, os_file_truncate.

DROP TABLE optimization failure analysis

Let's take a look at a MySQL 5.7 test environment where the DROP TABLE optimization solution fails.

Q1: Why did the online launch fail? Why does HARD LINK not take effect? Why is the AHI not in effect?

• When MySQL 5.7 is started with the standard configuration, from the perspective of debug-tracethe process , there is no time-consuming during the execution of the function, so after implementing the optimization plan, it has no effect row_drop_single_table_tablespace;row_drop_table_from_cache
• time-consuming process in que_eval_sql: query: PROCEDURE DROP_TABLE_PROC ---> dict_drop_index_tree;
• row_drop_single_table_tablespaceinnodb_flush_method=O_DIRECTThe time consumption is optimized by MySQL 5.7 configuration .

Q2: Does this optimization apply to MySQL 8.0?

innodb_flush_method=O_DIRECTThe optimization operation set also applies to MySQL 8.0.

Q3: How does MySQL 8.0 solve the problem of DROP_TABLE_PROCslow ?

• WL#9536: InnoDB_New_DD: Support crash-safe DDL；
• Depends on NEW DD from Version 8.0.3;
• Whole dropslow que_eval_sql, DROP_TABLE_PROCcut off as a whole;
• dict_drop_index_treeThe entire function, including , has been cut;
• For the specific implementation mechanism, refer to analyzing the implementation method of NEW DD.

Q4: What is the difference between MySQL 5.7 DROP TABLE and TRUNCATE in terms of implementation mechanism and optimization measures?

• The time-consuming to execute truncatethe operation is still in dict_drop_index_treethese os_file_truncatetwo stages;
• os_file_truncateTime-consuming: can be optimized by setting innodb_flush_method=O_DIRECTthe time (not hard linkoptimized by );
• dict_drop_index_treeTime-consuming, there is no optimization idea for the time being. Learn more: Physical file structure of InnoDB file system .

Q5: Why is 5.7 slow query sometimes recording TRUNCATE execution slow, and sometimes not recording?

According to the source code, there are two main dimensions when judging whether MySQL records slow queries: one is the execution time (not included utime_alter_lock); the other is the number of rows scanned, and special statements (such as call) are ignored. For truncateoperations , the number of rows scanned is 0, regardless of the execution time. When configured with min_examined_row_limita value greater than 0, general truncateoperations will not be recorded in slow queries because they do not meet this condition.

But when truncatethe operation is in the stored procedure, truncatethere are other DML operations (such as insert selecct) before the operation. At this time, because it is under the same THD, the result thd->get_examined_row_count()returned is actually the previous DML statement (this should be a defect) . If the execution time of truncatethe operation long_query_time, the truncatestatement .

At the same time, in MySQL call8.0 , the statements for will not be recorded separately. Instead, it is recorded as a unified callstatement . If you need to check the execution status of the statements in the stored procedure, you can use show profilesView .

Slow Query Record Stack

test stored procedure

DROP PROCEDURE truncate_test;
DELIMITER //
CREATE PROCEDURE truncate_test()
BEGIN
  insert into t1 select * from t1_bak;
  truncate table t1;
END
//
DELIMITER ;

call truncate_test();

mysql> call truncate_test();
Query OK, 0 rows affected (1 min 59.58 sec)
# Time: 2023-06-08T00:28:30.969993+08:00
# User@Host: root[root] @ localhost []  Id:     2
# Schema: db2  Last_errno: 0  Killed: 0
# Query_time: 119.177518  Lock_time: 0.000233  Rows_sent: 0  Rows_examined: 131072  Rows_affected: 131072
# Bytes_sent: 0
# Stored_routine: db2.truncate_test
use db2;
SET timestamp=1686155310;
insert into t1 select * from t1_bak;
# Time: 2023-06-08T00:28:31.375873+08:00
# User@Host: root[root] @ localhost []  Id:     2
# Schema: db2  Last_errno: 0  Killed: 0
# Query_time: 0.405734  Lock_time: 0.003310  Rows_sent: 0  Rows_examined: 131072  Rows_affected: 0
# Bytes_sent: 0
# Stored_routine: db2.truncate_test
SET timestamp=1686155311;
truncate table t1;

Comparison with MySQL 8.0

mysql> call truncate_test();
Query OK, 0 rows affected (2 min 28.51 sec)

# Time: 2023-06-07T17:18:39.215632Z
# User@Host: root[root] @ localhost []  Id:     8
# Query_time: 148.516478  Lock_time: 0.000372 Rows_sent: 0  Rows_examined: 172032
use testdb;
SET timestamp=1686158318;
call truncate_test();

How does MySQL 8.0 track?

mysql> call truncate_test();
Query OK, 0 rows affected (2 min 24.84 sec)

mysql> 
mysql> 
mysql> show profiles;
+----------+--------------+-----------------------------------------+
| Query_ID | Duration     | Query                                   |
+----------+--------------+-----------------------------------------+
|        1 | 144.55113600 | insert into ltb2 select * from ltb3_bak |
|        2 |   0.29312375 | truncate table ltb2                     |
+----------+--------------+-----------------------------------------+
2 rows in set, 1 warning (0.00 sec)

The above includes the analysis of slow truncateexecution . If you have any questions or suggestions about the details, please leave a message for exchange.

About SQLE

The SQLE of the Akson open source community is a SQL auditing tool for database users and managers, which supports multi-scenario auditing, supports standardized online processes, natively supports MySQL auditing, and has scalable database types.

SQLE get

type	address
Repository	https://github.com/actiontech/sqle
document	https://actiontech.github.io/sqle-docs/
release news	https://github.com/actiontech/sqle/releases
Data audit plug-in development documentation	https://actiontech.github.io/sqle-docs-cn/3.modules/3.7_auditplugin/auditplugin_development.html

{{o.name}}

{{m.name}}