Processing strategy of different versions of Mysql online DDL

Regarding the ONLINE and OFFLINE of DDL, in layman's terms:

• ONLINE: Allows DML operations while executing DDL

• OFFLINE: DML operations are not allowed while DDL is being executed

In the production scenario, the business is running, and the DBA needs to add new fields to the table without affecting the write operation of the table. This is ONLINE DDL. In other words, the so-called ONLINE and OFFLINE are more accurate for the ALTER TABLE statement.

After MySQL 5.6, the ALGORITHM parameter was introduced to control the algorithm of the ALTER TABLE statement during concurrency. Officially, it is not recommended to specify the value of this parameter, and MySQL itself will choose the optimal algorithm. The syntax is as follows:

ALTER TABLE TABLE_NAME ADD COLUMN COLUMN_NAME COLUMN_TYPE ALGORITHM=[ COPY ｜ INPLACE ｜INSTANT ]；

MySQL DDL will go through three phases during operation: preparation phase, execution phase, and submission phase. Different algorithms do different processing in different phases.

The difference between these three algorithms is introduced below:

COPY algorithm:

It will lock at the SERVER layer, generate a temporary table, and copy the data of the original table to the temporary table row by row. At this time, the DML operation will be blocked, so the COPY algorithm generates OFFLINE DDL. The three phases of the COPY algorithm perform the following operations:

Preparation Phase:

1. Add a shared lock to the metadata of the table, and read the metadata of the table from the *.frm file. At this time, other concurrent DDL cannot be executed, but DML can be executed.

2. The shared lock is upgraded to an exclusive lock, and neither DDL nor DML can be executed at this time.

3. At the SERVER layer, the temporary table is created through the CREATE TABLE ... LIKE ... statement, and the corresponding *.ibd and *.frm files are also generated at the storage engine layer.

Execution phase, submission phase:

1. Modify temporary table metadata.

2. Copy the original table data to the temporary table.

3. Rename temporary tables and files.

4. Delete the original table and file.

5. Commit the transaction and release the lock.

INPLACE algorithm:

No temporary table will be created, and in some cases, the IN-PLACE method is required to rebild the table. Metadata locks will be added during the preparation phase and the submission phase, and the DML will not be blocked during the execution phase. The three stages of the INPLACE algorithm perform the following operations:

Preparation Phase:

1. Add a metadata lock to the table and upgrade from a shared lock to an exclusive lock. At this time, concurrent DML cannot operate.

2. Judge rebuild table or no-rebuild table according to different statements. If it is no-rebuild, temporary transit files of *.frm and *.ibd will be created under the original table path. no-rebuild Only create *.frm files in addition to creating secondary indexes; when creating secondary indexes, *.ibd files will not be generated, but will be modified in the original *.ibd files, which will be specified in the parameter tmpdir A temporary file is generated under the path to store the index sorting results, and then merged into the original *.ibd file.

3. If it is rebuild, it will apply for row log space to store the DML operations executed concurrently in the DDL execution phase. If it is no-rebuild, this operation will not be performed.

Execution phase:

1. The exclusive lock is released, and the metadata shared lock is reserved, and concurrent DML can be executed at this time.

2. Scan the primary key of the metatable and all data pages of the secondary index, generate B+TREE and store it in a temporary file.

3. If it is a rebuild, all DML operations on the original table will be recorded in the row log file.

Commit phase:

1. Upgrading the metadata lock is upgraded from a shared lock to an exclusive lock, and DML cannot be executed at this time.

2. If rebuild redo the content in the row log file, no-rebuild does not have this operation.

3. Rename the original table file, change the temporary table name to the original table name, and delete the original table file.

4. Commit the transaction to complete the change.

INSTANT algorithm:

This algorithm is a new feature of MySQL version 8.0.12. During operation, this algorithm only modifies the metadata in the data dictionary, does not copy data or rebuild tables, and does not add metadata exclusive locks. The original table data is not affected by impact, DML will not be blocked, and the whole process is completed instantaneously. The original table will only be operated when it is triggered later. The INSTANT algorithm operations supported by the current version of the InnoDB storage engine are:

1. ADD COLUMN

2. ALTER TABLE TABLE_NAME ADD/DROP INDEX INDEX_NAME（COLUMN）

3. ALTER TABLE TABLE_NAME RENAME NEW_TABLE_NAME

4. set/remove default

5. Add/remove virtual columns

Supplement: rebuild refers to the behavior of using CREATE TABLE LIKE to create tables at the SERVER layer, no-rebuild means not to create tables at the SERVER layer through CREATE TABLE LIKE, but it is possible to create dumped *.ibd files and *.frm at the storage engine layer document.

[MySQL] Detailed explanation of Online DDL