Article Directory
One, some small concepts
1.1 MySQL shutdown related parameters
What operations need to be performed when MySQL is normally shut down are generally controlled by the parameter innodb_fast_shutdown. If MySQL shuts down crash abnormally, it is equivalent to innodb_fast_shutdown=2. The following is a detailed introduction of this parameter:
-
innodb_fast_shutdown
0: The operation done when closing MySQL in this mode is the most and also the most time-consuming, but the startup of MySQL in this mode is the fastest, mainly to do three actions:
1) flush dirty pages (dirty pages fall Disk)
2) merge change buffer (mainly for non-unique secondary indexes)
3) full purge (delete useless undo)
When upgrading the innodb plugin, it will be set to 0, and then the database will be shut down normally.
1: The default value. In this mode, you only need to flush dirty pages when MySQL is closed.
2: This mode is equivalent to the abnormal shutdown of MySQL. When MySQL is closed, you only need to flush the logs in the log buffer to logfile. In this mode, MySQL is closed. It takes the shortest time, but MySQL takes the longest time to start, and it needs to perform crash recovery based on the log file.
1.2 MySQL startup related parameters
- innodb_force_recovery
What operations do MySQL do during recovery are mainly controlled by the parameter innodb_force_recovery. The smaller the parameter, the more detailed the inspection items, and the better the data security. The following is a detailed explanation of this parameter
0：默认值、所有都检查
1：忽略检查corrupt页
2：阻止master线程运行，如果master线程需要执行full purge操作会导致crash
3：不执行事务回滚操作
4：不执行change buffer的合并操作
5：不检查undo log，innodb引擎会将未提交事务视为已提交
6：不执行前滚操作
When the value of innodb_force_recovery is 1-3, only SELECT TABLE, DROP or CREATE tables are allowed; when the value of innodb_force_recovery is >=4, the version before 5.7.17 supports DROP TABLE, and the version after 5.7.18 does not.
1.3 Two-phase commit of innodb
[External link image transfer failed. The source site may have an anti-leech link mechanism. It is recommended to save the image and upload it directly (img-VrkDigYA-1593157663191)(http://note.youdao.com/yws/res/57689/D6BE9FF0C25E41F5B32F40B2659FBC7D)]
For an update operation, the main two-phase submission process is:
1) The executor first finds the engine and fetches the line ID=2. ID is the primary key, and the engine directly uses the tree search to find this row. If the data page where the row of ID=2 is originally in the memory, it is directly returned to the executor; otherwise, it needs to be read from the disk into the memory and then returned.
2) The executor gets the row data given by the engine and adds 1 to this value. For example, it was originally N, but now it is N+1 to get a new row of data, and then call the engine interface to write this new row of data.
3) The engine updates this new row of data into the memory, and at the same time records the update operation in the redo log, at this time the redo log is in the prepare state. Then inform the executor that the execution is complete and the transaction can be submitted at any time.
4) The executor generates the binlog of this operation and writes the binlog to disk. (Write once at the time of submission)
5) The executor calls the commit transaction interface of the engine, and the engine changes the redo log that has just been written to the commit state, and the update is completed.
Two, MySQL crash recovery
2.1 Roll forward
1. LSN point acquisition
When MySQL is shut down normally, the flush dirty pages operation will not only flush all dirty pages in the buffer pool, but also record the latest checkpoin lsn to the first page of the iddata file. If MySQL is normally shut down and started normally, it will read the lsn point in the first page of the ibdata file when it is started, and then start the subsequent redo recovery operation from the lsn point. If the data page is found to be damaged when obtaining the lsn point in the first data page of ibdata, MySQL will read the subsequent data pages in turn until there is a complete data page to record the lsn point, and then perform subsequent redo recovery based on the lsn point operating.
2. Redo log roll forward
MySQL scans the redo log based on the LSN points found above, looking for the location information of mlog_checkpoint. The information between checkpoint lsn and mlog_checkpoint indicates all data that needs to be recovered. The process is mainly divided into three scenarios:
1) If the mlog_checkpoint location information is not found in the redo log, it means that no data update has occurred after the checkpoint lsn, or that MySQL is normally closed
2) If multiple mlog_checkpoint location information is found in the redo log, it means that the redo log has been damaged, and InnoDB reports an error and exits
3) If a mlog_checkpoint location information is scanned in the redo log, it means that this part of the data is the changed operation data after the last checkpoint and has not been flushed, and this part of the log needs to be used for recovery.
2.2 Rollback
1. Undo recovery
The undo log of all database changes (except temporary tables) will also be recorded in the redo log, so after the redo log is applied, innodb will initialize the undo, according to whether the slot in each rollback segment page is used or not Restore the corresponding undo log. This step restores the database to the complete transaction state before the crash.
Start a background thread to do transaction rollback and cleanup operations. For the transaction in the active state, the transaction is directly rolled back (the transaction before the prepare state). For the transaction that is neither active nor prepare (the transaction has been committed), it is directly considered as committed and the transaction object is directly released. This step is mainly undo recovery. After undo recovery is completed, theoretically there are only transactions in the prepare state on the transaction list, and these transactions need to enter XA recovery through binlog.
2. Binlog recovery
After the engine is initialized, the Server layer will scan the last Binlog file, collect the xid recorded in it (MYSQL_BIN_LOG::recover), and compare it with the transaction xid of the InnoDB layer. If the xid already exists in the binlog, the corresponding transaction needs to be committed, otherwise the transaction needs to be rolled back (after preparing the node, determine whether to roll back according to whether the binlog of the transaction is complete).
Document reference:
https://www.cnblogs.com/xinysu/p/6586386.html#_lab2_1_0
http://mysql.taobao.org/monthly/2015/04/01/
http://www.sysdb.cn/index.php/2016/01/14/innodb-recovery/#more-17
https://www.cnblogs.com/coderyuhui/p/7191413.html