double write

1.1 What is double write

  The default data page of MySQL is 16K, and the data page of the file system is 4K, and IO operations are read and written in page units. It is possible that after the database modifies a 16K data page, the operating system starts to write to the disk, but during this process the database is down and the 16K data page is not completely written to the disk. After the database is restarted, check the data page and find that there is an incomplete data page, so it can't get up (redo is a recovery based on the complete data page).
  To solve this problem, MySQL introduced the double write feature. Double write is for dirty data, improving the reliability of InnoDB, and is used to solve partial page write failure. For data persistence, dirty data needs to be flushed to the disk, and double write occurs in the process of flushing dirty data. Flushing is a copy of dirty data written to the shared tablespace ibdata, and a copy written to the real data file for permanent storage. Dirty data written twice is called double wriete.

1.2 double write原理

  In simple terms, double write is to first put the modified dirty pages in the double write buffer area. This area occupies 2M memory space. The buffer space is full or triggered by other conditions, so that the dirty pages stored in the double write buffer are written to the shared table space first. (Before MySQL 8.0.20, the doublewrite buffer storage area is located in the InnoDBsystem table space. Starting from MySQL 8.0.20, the doublewrite buffer storage area is located in the doublewrite file), and then write the data file. At this time, if an incomplete page is written, it can be overwritten with a complete page in the shared table space. When the data page is complete, the database can be pulled up. All subsequent recovery depends on the redo log. Redo log records the log in the form of data blocks, and it records the modification according to the offset. Although the data is written twice, the double write buffer does not require twice the I/O overhead or twice the I/O operations. You only need to fsync()call the operating system once . Data can be written to the buffer doublewrite (except as sequential blocks larger innodb_flush_methodset O_DIRECT_NO_FSYNC).
  After 8.0.20, by default, two double write files are created for each buffer pool instance: refresh list double write files and LRU list double write files.

• The refresh list double-write file is used to refresh the pages in the list from the buffer pool. The default size of the doublewrite file to refresh the list is InnoDB page size * doublewrite page bytes.

• The LRU list double-write file is used to refresh pages from the buffer pool LRU list. It also contains slots for single page refresh. The default size of the doublewrite file of the LRU list is InnoDB page size * (doublewrite pages + (512 / the number of buffer pool instances)), where 512 is the total number of slots reserved for single page refresh.

  There are at least two double-write files. The maximum number of double-write files is twice the number of buffer pool instances.
The unit written by redolog is 512 bytes, which is the smallest unit of disk IO, so there is no data corruption.

1.3 Double write recovery process

There are three scenarios for data recovery

Dirty data is successfully written to disk

  If the flashing is successful, you can find the checkpoint, roll forward and roll back.

Shared table space ibdata write failure

  If it fails to write to the shared tablespace, the data will not be written to the data file, and the database will think that this flash disk has never occurred. MySQL will load the original data from the disk at this time, and then find the checkpoint, redo log Just roll forward and roll back.

Failed to flush data file with dirty data

  Writing to the shared table space is successful, but writing to the data file fails. When recovering, MySQL directly compares the checksum of the page. If it is not correct, it directly finds a recent copy of the page from the double write in the shared table space and copies it to Table space file, and then apply redo log, the recovery process is completed. Because there is a copy, I don't worry about whether the data pages in the table space are damaged.

1.4 Doublewrite load

  Double write is a buffer, but in fact it is a buffer opened on a physical file, which is actually a file, so it will cause more fsync operations in the system, and the fsync performance of the hard disk is very slow, so it will decrease The overall performance of mysql.

2. Monitor double write workload

mysql> show global status like '%dblwr%';
+----------------------------+-------+
| Variable_name              | Value |
+----------------------------+-------+
| Innodb_dblwr_pages_written | 44    |
| Innodb_dblwr_writes        | 8     |
+----------------------------+-------+
2 rows in set (0.00 sec)

Points of concern: Innodb_dblwr_pages_written / Innodb_dblwr_writes
  After doublewrite is enabled, doublewrite must be written every time a dirty page is refreshed, and doublewrite exists on the disk as two consecutive areas, each area is composed of consecutive pages. Generally, one area has the most There are 64 pages, so one IO write should be able to write up to 64 pages.

Suitable for closing double write

• Mass DML

• Not afraid of data corruption and loss

• System write load becomes the main load

1.5 Related parameters

mysql> show variables like '%double%';
+-------------------------------+-------+
| Variable_name                 | Value |
+-------------------------------+-------+
| innodb_doublewrite            | ON    |
| innodb_doublewrite_batch_size | 0     |
| innodb_doublewrite_dir        |       |
| innodb_doublewrite_files      | 2     |
| innodb_doublewrite_pages      | 8     |
+-------------------------------+-------+
5 rows in set (0.00 sec)

#innodb_doublewrite：是否开启doublewrite
#innodb_doublewrite_batch_size：定义要批量写入的双写页面数
#innodb_doublewrite_dir：定义双写缓冲区文件目录
#innodb_doublewrite_files：定义双写文件的数量
#innodb_doublewrite_pages：定义批量写入时每个线程的最大双写页数