MySQL Basics (33) MySQL Transaction Log

Transactions have four characteristics: atomicity, consistency, isolation and durability. So on what mechanism are the four characteristics of transactions based on?

• Transaction isolation is 锁机制achieved by .
• The atomicity, consistency and durability of the transaction are guaranteed by the redo log and undo log of the transaction.
  • REDO LOG is called 重做日志, provides rewrite operations, restores the page operations modified by the submitted transaction, and is used to ensure the durability of the transaction.
  • UNDO LOG is called 回滚日志rolling back row records to a specific version to ensure the atomicity and consistency of transactions.

Some DBAs may think that UNDO is the reverse process of REDO, but this is not the case.

1. redo log

1.1 Why REDO logs are needed

On the one hand, the buffer pool can help us eliminate the gap between the CPU and the disk, and the checkpoint mechanism can ensure the final placement of the data. However, due to checkpoint, the master thread 并不是每次变更的时候就触发processes it at intervals. So the worst case scenario is that after the transaction is committed, the buffer pool has just been written and the database is down. Then this data is lost and cannot be recovered.

On the other hand, transactions include 持久性the characteristic that for a committed transaction, even if the system crashes after the transaction is submitted
, the changes made by this transaction in the database cannot be lost.

So how to ensure this durability? 一个简单的做法: Flush all pages modified by the transaction to disk before the transaction is committed, but this simple and crude approach has some problems

另一个解决的思路: We just want the modifications made by the submitted transactions to the data in the database to take effect permanently. Even if the system crashes later, this modification can be restored after restarting. So we don't actually need to refresh all the pages modified in the memory by the transaction to the disk every time a transaction is committed. We only need to change what is 修改modified 记录一下. For example, a transaction changes the value of the byte 第10号at offset in the system table space to . We only need to record this: update the value at offset 100 of page 10 in table space 0 to 2.10012

1.2 Benefits and features of REDO logs

1. Benefits

• The redo log reduces the frequency of disk flushing
• The redo log takes up very little space

2. Features

• Redo logs are written to disk sequentially
• During transaction execution, redo log continuously records

1.3 Composition of redo

Redo log can be simply divided into the following two parts:

• 重做日志的缓冲 (redo log buffer), stored in memory, is volatile.

Parameter settings: innodb_log_buffer_size:
redo log buffer size, default 16M, maximum value is 4096M, minimum value is 1M.

mysql> show variables like '%innodb_log_buffer_size%';
+------------------------+----------+
| Variable_name | Value |
+------------------------+----------+
| innodb_log_buffer_size | 16777216 |
+------------------------+----------+

• Redo log file (redo log file), saved on the hard disk, is persistent.

1.4 The overall process of redo

Taking an update transaction as an example, the redo log flow process is as shown in the following figure:

Step 1: First read the original data from the disk into the memory, and modify the memory copy of the data.
Step 2: Generate a redo log and write it into the redo log buffer, which records the modified value of the data.
Step 3 : When the transaction commits, refresh the contents in the redo log buffer to the redo logfile, and use append writing to the redo log file.
Step 4: Regularly refresh the modified data in the memory to the disk.

Experience: Write-Ahead Log (pre-log persistence): Before persisting a data page, first persist the corresponding log page in memory.

1.5 Redo log flushing strategy

The writing of redo log is not directly written to the disk. The InnoDB engine will first write the redo log buffer when writing the redo log, and then flush it 一定的频率into the real redo log file. How do you think about the certain frequency here? This is the brush strategy we are talking about.

Note that the process of flushing the redo log buffer to the redo log file is not really flushing to the disk, but just flushing it to the 文件系统缓存(page cache) (this is an optimization made by modern operating systems to improve file writing efficiency) , the actual writing will be left to the system itself (for example, the page cache is large enough). Then there is a problem for InnoDB. If it is left to the system for synchronization, if the system goes down, the data will also be lost (although the probability of the entire system going down is still relatively small).

In response to this situation, InnoDB provides innodb_flush_log_at_trx_commitparameters, which control how to flush the logs in the redo log buffer to the redo log file when commit commits the transaction. It supports three strategies:

• 设置为0: Indicates that no flush operation will be performed every time a transaction is submitted. (The system defaults to master thread synchronizing redo logs every 1 second)
• 设置为1: Indicates that synchronization and disk flushing operations will be performed every time a transaction is submitted ( 默认值)
• 设置为2: Indicates that only the contents of the redo log buffer are written to the page cache each time a transaction is submitted, without synchronization. It is up to the OS to decide when to synchronize to the disk file.

1.6 Demonstration of different brushing strategies

1. Flowchart

1.7 redo log file

1. Related parameter settings

• innodb_log_group_home_dir: Specify the path where the redo log file group is located. The default value is ./, which means it is in the data directory of the database. var/lib/mysqlThere are two files named ib_logfile0and by default in MySQL's default data directory ( ) ib_logfile1. The logs in the log buffer are flushed to these two disk files by default. The location of this redo log file can also be modified.

• innodb_log_files_in_group: Specify the number of redo log files, and the naming method is: ib_logfile0, iblogfile1...iblogfilen. Default is 2, maximum is 100.

  mysql> show variables like 'innodb_log_files_in_group';
  +---------------------------+-------+
  | Variable_name | Value |
  +---------------------------+-------+
  | innodb_log_files_in_group | 2 |
  +---------------------------+-------+
  #ib_logfile0
  #ib_logfile1
  

• innodb_flush_log_at_trx_commit: Control the strategy of redo log flushing to disk, the default is 1.

• innodb_log_file_size: Set the size of a single redo log file, the default value is 48M. The maximum value is 512G. Note that the maximum value refers to the sum of the entire redo log series files, that is, (innodb_log_files_in_group * innodb_log_file_size) cannot be greater than the maximum value of 512G.

  mysql> show variables like 'innodb_log_file_size';
  +----------------------+----------+
  | Variable_name | Value |
  +----------------------+----------+
  | innodb_log_file_size | 50331648 |
  +----------------------+----------+
  

Modify its size based on the business to accommodate larger transactions. Edit the my.cnf file and restart the database to take effect, as shown below

[root@localhost ~]# vim /etc/my.cnf
innodb_log_file_size=200M

2. Log file group

The total redo log file size is actually: innodb_log_file_size × innodb_log_files_in_group.

If you write data to the redo log file group in a circular manner, will the redo log written later overwrite the redo log written earlier? certainly! So the designers of InnoDB proposed the concept of checkpoint.

3. If the checkpoint

write pos catches up with the checkpoint, it means 日志文件组it is full. At this time, no new redo log records can be written. MySQL has to stop, clear some records, and advance the checkpoint.

2. Undo log

Redo log is a guarantee of transaction durability, and undo log is a guarantee of transaction atomicity. The pre-operation to update data in a transaction is actually to
write an undo log first.

2.1 How to understand Undo logs

Transactions need to be atomic, that is, all operations in the transaction must be completed or nothing must be done. But sometimes some situations may occur in the middle of transaction execution, such as:

• Situation 1: Various errors may be encountered during transaction execution, such as 服务器本身的错误, 操作系统错误or even sudden 断电errors.
• Scenario 2: Programmers can manually enter statements during transaction execution ROLLBACKto end the execution of the current transaction.

When the above situation occurs, we need to change the data back to its original appearance. This process is called "transaction" 回滚. This can create an illusion: this transaction seems to have done nothing, so it meets 原子性the requirements.

2.2 The role of Undo log

• Function 1: Roll back data
• Function 2: MVCC

2.3 Storage structure of undo

1. Rollback segment and undo page

InnoDB uses segments to manage undo logs, that is 回滚段（rollback segment）. 1024Each rollback segment records undo log segment, and applications are made in each undo log segment undo页.

• In InnoDB1.1版本之前(excluding version 1.1), there is only one rollback segment, so the number of simultaneous online transactions supported is limited to 1024. Although it is sufficient for most applications.

• Starting from version 1.1, InnoDB supports the largest number 128个rollback segment, so the limit of simultaneous online transactions it supports has been increased 128*1024.

  mysql> show variables like 'innodb_undo_logs';
  +------------------+-------+
  | Variable_name | Value |
  +------------------+-------+
  | innodb_undo_logs | 128 |
  +------------------+-------+
  

2. Rollback segments and transactions

1. Each transaction will only use one rollback segment, and one rollback segment may serve multiple transactions at the same time.
2. When a transaction starts, a rollback segment will be created. During the transaction, when the data is modified, the original data will be copied to the rollback segment.
3. In a rollback segment, transactions continue to fill extents until the transaction ends or all space is used up. If the current extent is not enough, the transaction will request the expansion of the next extent in the segment. If all allocated extents are used, the transaction will overwrite the original extent or expand the new extent if the rollback segment allows. panels to use.
4. Rollback segments exist in undo table spaces. Multiple undo table spaces can exist in the database, but only one undo table space can be used at the same time.
5. When a transaction commits, the InnoDB storage engine does the following two things:
   • Put the undo log into the list for subsequent purge operations
   • Determine whether the page where the undo log is located can be reused, and if it can be allocated to the next transaction,

3. Data classification in rollback segment

1. 未提交的回滚数据(uncommitted undo information)
2. 已经提交但未过期的回滚数据(committed undo information)
3. 事务已经提交并过期的数据(expired undo information)

2.4 Types of undo

In the InnoDB storage engine, undo log is divided into:

• insert undo log
• update undo log

2.5 Life cycle of undo log

1. Brief generation process

Only Buffer Pool process:

after having Redo Log and Undo Log:

2. Detailed generation process

When we perform an INSERT:

begin;
INSERT INTO user (name) VALUES ("tom");

When we execute UPDATE:

UPDATE user SET id=2 WHERE id=1;

3. How to roll back undo log

Taking the above example, assuming rollback is executed, the corresponding process should be as follows:

1. Delete the data with id=2 through undo no=3 log
2. Restore the deletemark of the data with id=1 to 0 through the log of undo no=2
3. Restore the name of the data with id=1 to Tom through the log of undo no=1
4. Delete the data with id=1 through undo no=0 log

4. Deletion of undo log

• For insert undo log

  Because the record of the insert operation is only visible to the transaction itself and not to other transactions. Therefore, the undo log can be deleted directly after the transaction is submitted, and no purge operation is required.

• For the update undo log,
  the undo log may need to provide an MVCC mechanism, so it cannot be deleted when the transaction is committed. When submitting, put it into the undo log list and wait for the purge thread to perform the final deletion.

2.6 Summary

Undo log is a logical log. When rolling back a transaction, it only logically restores the database to its original state.
Redo log is a physical log, which records the physical changes of data pages. Undo log is not the reverse process of redo log.