logical storage structure
The logical storage structure of InnoDB is shown in the figure below:
1). Table space is the highest level of the logical structure of the InnoDB storage engine. If the user enables the parameter innodb_file_per_table (default in version 8.0 If enabled), each table will have a table space (xxx.ibd). One mysql instance can correspond to multiple table spaces,used to store records, indexes and other data< a i=4>.
2). segment, divided into data segment (Leaf node segment), index segment (Non-leaf node segment), and rollback Segment (Rollback segment), InnoDB is an index-organized table, the data segment is the leaf node of the B+ tree, and the index segment is the non-leaf node of the B+ tree. Section is used to manage multiple Extents (areas).
3). area, the unit structure of the table space, the size of each area is 1M. By default, the InnoDB storage engine page size is 16K, that is, there are a total of 64 consecutive pages in one area.
4). Page is the smallest unit of disk management of the InnoDB storage engine. The default size of each page is 16KB. In order to ensure page continuity, the InnoDB storage engine allocates 4-5 areas from the disk each time.
5). rows, InnoDB storage engine data is stored in rows.
In a row, there are two hidden fields by default:
Trx_id: Every time a record is modified, will assign the corresponding transaction id to the trx_id hidden column. Roll_pointer: Every time a certain index record is changed, the old version will be written to the undo log, and then this hidden column is equivalent to a pointer, which can be used to find the roll_pointer. Record the information before modification.
Architecture Overview
Starting from MySQL version 5.5, the InnoDB storage engine is used by default. It is good at transaction processing and has crash recovery features. It is widely used in daily development. The following is the InnoDB architecture diagram, with the memory structure on the left and the disk structure on the right.
memory structure
The memory structure on the left is mainly divided into four major blocks: Buffer Pool, Change Buffer, Adaptive Hash Index, and Log Buffer. Next, we introduce these four parts.
Buffer Pool buffer pool
The InnoDB storage engine is based on disk file storage. There is a big speed difference between accessing the physical hard disk and accessing it in memory. In order to make up for the difference in I/O efficiency between the two as much as possible, it is necessary to frequently The used data is loaded into the buffer pool to avoid disk I/O for each access. InnoDB's buffer pool not only caches index pages and data pages, but also includes undo pages and insertion caches. , adaptive hash index and InnoDB lock information and so on.
The buffer pool is based on Page pages, and the underlying layer uses a linked list data structure to manage Pages. According to the status, Page is divided into three types:
- free page: free page, not used.
- Clean page: The page is used and the data has not been modified.
- Dirty page: Dirty page is a used page, the data has been modified, and the data in the page is inconsistent with the data on the disk.
On dedicated servers, it is common to allocate up to 80% of physical memory to the buffer pool, see show variables like 'innodb_buffer_pool_size';
Change Buffer change buffer
Change Buffer, change the buffer ( for non-unique secondary index pages), when executing the DML statement, if these data If the Page is not in the Buffer Pool, it will not directly operate the disk. Instead, the data changes will be stored in the Change Buffer. When the data is read in the future, the data will be merged and restored to the Buffer Pool, and then the merged data will be Flush to disk. That is, the data does not exist in BufferPoo. It does not perform 10 operations specifically for modification, but waits for the next time the data is queried to initiate an IO operation to modify the data and present brand-new data to the user. , used to merge IO requests and reduce disk IO
Adaptive Hash Index Adaptive Hash Index
Adaptive hash index does not require manual intervention. It is automatically completed by the system according to the situation., similar to the index that cannot be used when querying. , all completed automatically by the system
Adaptive hash index, used to optimize queries for Buffer Pool data. Although MySQL's innoDB engine does not directly support hash indexes, it provides us with a function called adaptive hash indexes. As we mentioned before, the performance of hash index is generally higher than that of B+ tree when performing equivalent matching, because hash index generally only requires one IO, while B+ tree may require several matches, so hash index The efficiency is high, but the hash index is not suitable for range queries, fuzzy matching, etc. The InnoDB storage engine will monitor the query for each index page on the table. If it is observed that the hash index can improve the speed under specific conditions, a hash index will be established, which is called an adaptive hash index < /span>.
Log Buffer Log Buffer
Log Buffer: The log buffer is used to save the log data (redo log, undo log) to be written to the disk. The default size is 16MB. The logs in the log buffer will be refreshed to the disk regularly. If a transaction requires updating, inserting, or deleting many rows, increasing the log buffer size can save disk I/O.
parameter:
innodb_log_buffer_size: buffer size
innodb_flush_log_at_trx_commit: Timing of flushing logs to disk. The values mainly include the following three:
1: The log is written and flushed to disk every time a transaction commits, default value.
0: Write and flush the log to disk once per second.
2: The log is written after each transaction commits and flushed to disk once per second.
Disk structure
System Tablespace System tablespace
The system tablespace is the storage area for change buffers. It may also contain table and index data if the table was created in a system tablespace rather than a per-table file or a general tablespace. (MySQL5.x version also includes InnoDB data dictionary, undolog, etc.)
View the parameters of the system table space: innodb_data_file_path, system table space, the default file name is ibdata1.
File-Per-Table Tablespaces file tablespace
If the innodb_file_per_table switch is turned on, the file table space of each table contains the data and indexes of a single InnoDB table and is stored in a single data file on the file system.
Switch parameter: innodb_file_per_table, this parameter is enabled by default.
That is to say, every time we create a table, a table space file will be generated, as shown in the figure:
General Tablespaces General tablespaces
General table space needs to be created through CREATE TABLESPACE syntax. This table space can be specified when creating a table.
Create tablespace
CREATE TABLESPACE ts_name ADD DATAFILE 'file_name' ENGINE = engine_name;
Specify the table space when creating the table
CREATE TABLE xxx ... TABLESPACE ts_name;
Undo Tablespaces Undo tablespaces
Undo table space. When the MySQL instance is initialized, it will automatically create two default undo table spaces (initial size 16M) for storing undo logs.
Temporary Tablespaces Temporary tablespace
InnoDB uses session temporary tablespaces and global temporary tablespaces. Store data such as temporary tables created by users.
Doublewrite Buffer Files Doublewrite Buffer
Double-write buffer, before the innoDB engine refreshes the data page from the Buffer Pool to the disk, it first writes the data page into the double-write buffer file to facilitate data recovery when the system is abnormal.
Redo Log redo log
Redo logs are used to achieve transaction durability. The log file consists of two parts: the redo log buffer (redo log buffer) and the redo log file (redo log). The former is in memory and the latter is on disk. After the transaction is committed, all modification information will be stored in the log, which can be used for data recovery when an error occurs when flushing dirty pages to disk.
Write redo log files in a circular manner, involving two files:
background thread
The data we update in the memory is refreshed to the disk by the background thread at the appropriate time. In the background thread of InnoDB, it is divided into 4 categories, namely: Master Thread, IO Thread, Purge Thread, and Page Cleaner Thread.
Master Thread
The core background thread is responsible for scheduling other threads, and is also responsible for asynchronously refreshing the data in the buffer pool to the disk to maintain data consistency, including refreshing dirty pages, merging and inserting cache, and recycling undo pages.
IO Thread
AIO (Asynchronous non-blocking IO, high efficiency) is used extensively in the InnoDB storage engine to handle IO requests, which can greatly Improve the performance of the database, and IO Thread is mainly responsible for the callback of these IO requests.
Purge Thread
It is mainly used to recycle the undo log that has been submitted by the transaction. After the transaction is committed, the undo log may not be used, so use it to recycle it.
Page Cleaner Thread
A thread that assists the Master Thread in flushing dirty pages to disk. It can reduce the work pressure of the Master Thread and reduce blocking.