This paper explains the structure and implementation of the basic knowledge of Mysql InnoDB transaction engine, as well as redo logs, the log characteristics . See text at the end of this reference
Implementing transactions
Transaction isolation is achieved by a database locks atomicity, isolation, durability of the redo log and undo log is achieved (wherein redo logs to achieve durability of transactions), redo log and undo log are a kind of recovery operation
redo recovery commit the transaction to modify the operation of the page, and undo the rollback rows to a specific version.
usually physical redo log record page is physical modification operations, is a logical Use the undo log record according to each line.
redo log
Two parts of the log
This log has two parts, one part is the in-memory buffer redo log (redo log Buffer) , which is volatile , the second part of the redo log files (redo log file) that is long-lasting .
Force Log at Commit (log written to the log file)
InnoDB transactional storage engine is the engine, which is by Force Log at Commit mechanism to achieve durability of transactions - when the transaction is committed, the transaction must be written to the log of all redo log file for persistence, pending transactions COMMIT is not complete until the operation is complete .
Characteristics of two log (no reading operation, random access)
Among them, redo log includes two parts (redo and undo two parts), redo log to ensure the durability of transactions, undo log to help roll back the transaction and MVCC features.
- redo is written in order, you do not need to run the database when the redo log file read operations
- And the undo log is a need for a random access (perhaps because rollback)
Ensure that the log written to the log file
When O_DIRECT option is not used, the database will be following diagram approach ( O_DIRECT option is the option in the Linux system, using this option, the document IO operations directly, without going through the file system cache, write directly to disk )
Improve database performance
We know fsync operation is written to disk is very time-consuming, so in order to improve performance, we can choose when the transaction commits, the log is not written to the redo log files, but to wait for a time during this operation fsync after cycle, not forced once fsync operations until the transaction commits.
Cons: When the database is down, as some logs are not flushed to disk will be lost transactions last a period of time.
But keep in mind that this method loses the ACID properties of a transaction , when inserting large quantities of data, it should be done after the insertion, once a commit operation, rather than a commit operation after inserting one, to do so can roll back the transaction method at the time of the transaction is rolled back to the beginning of the state.
Redo log storage structure
Redo log is established, for InnoDB engine, the redo log 512 bytes are stored in the engine. This means that the redo log buffer, redo log files are to "block" the way to preserve, called "redo log block" (the disk sector size is 512 bytes).
If the number of pages in a redo log block generated larger than 512 bytes, then the need to split the plurality of memory blocks redo logs. As are 512 bytes, writes the redo log block can guarantee atomicity, no doublewrite technology.
The following look: redo log buffer structure!
接着重点讲一下，FIRST_REC_GROUP 的意思
重做日志组（重做日志文档的组合）
log group 为重做日志组，其中与许多重做日志文档，InnoDB 实际只有一个log group（源码支持多个，但是在配置文档中禁止了）。
它是一个逻辑上的概念，实际上没有一个存储的物理文档来表示log group信息。
它由许多个重做日志文档组成，每个日志文档的大小都是相同的。在 InnoDB 的 1.2版本之前，重做日志文档的总大小小于 4GB，之后提高为512 GB .
重做日志文档存储的就是log buffer中保存的 log block，也是按照块的方式去存储。
log buffer 的提交
-
当事务提交的时候
-
当log buffer 中有一般的内存空间已经被使用时
-
log checkpoint时
存在以上三种情况的时候，内存中的log block会刷新到磁盘上
LSN (Log Sequence Number/日志序列号)
在InnoDB引擎中，LSN占用8字节，并且单调递增。LSN表示的含义有：
- 重做日志的写入的总量
- checkPoint的位置
- 页的版本
如果当前重做日志的LSN为1000，一个事务写入了100字节的重做日志，那么LSN变为1100，又有事务T2写入了200字节的重做日志，那么lsn就变成了1300
LSN不仅记录在重做日志中，还存在每个页中，在每个页的头部，有一个值为FIL_PAGE_LSN，记录了该页的LSN.
在该页中，LSN表示该页最后刷新时 LSN 的大小，因为重做日志记录的是每个页的日志，因此页中的LSN用来判断页是否需要进行恢复操作。
如：
页P1的LSN为10000，那么数据库启动的时候,InnoDB检测到写入重做日志的LSN为13000，且事务已经提交。
Then the database needs to be a recovery operation, the redo logs to the P1 page.
If redo log LSN LSN less than P1 page, you do not need to redo, because P1 page LSN represents the page has been refreshed
Command SHOW ENGINE INNODB STATUS to see the situation the LSN, log part of the following parameters will appear
- Log sequence number represents the current LSN (log cache them)
- Log flushed up to represent flushed to the redo log file of LSN
- Last checkpoint at representation of LSN flushed to disk
restore
InnoDB storage engine at startup, regardless of whether the database was last shut down properly, will try to recover, checkPoint represent LSN flushed to disk pages, during the recovery process, only part of the recovery log checkPoint began.
References:
https://juejin.im/post/5c3c5c0451882525487c498d#heading-17
"Mysql storage engine technology insider --InnoDB"
Original: Large column Mysql transaction log log --redo