In this article, we try to study the database file storage form under the Innodb engine and MyISAM engine of MySQL in versions 5.7 and 8.0.
Storage engines such as InnoDB and MyISAM store tables on disks, and the structure used by the operating system to manage disks is called a file system. In other words, storage engines like InnoDB and MyISAM store tables on the file system. When we want to read data, these storage engines will read the data from the file system and return it to us. When we want to write data, these storage engines will write the data back to the file system.
【1】MySQL5.7
① InnoDB engine
As shown in the figure below, we randomly select a database to view, and we can see that there are three file formats: opt, frm, and ibd.
opt: Contains various attributes of the database, such as the character set and comparison rules of the database.
frm: stores the structure of the table and is a binary format file.
The table structure is the name of the table, how many columns there are in the table, the data type, constraints and indexes of each column, the character set used and comparison rules and other information, all of which are reflected in our table creation statement .
ibd: Independent table space, which stores the data and indexes of the table.
MyISAM
As shown below, there are four file formats: opt, frm, MYD, MYI. It can be seen that the difference from InnoDB is that there is no ibd file here, but it is split into MYD and MYI.
MYD : table data file.
MYI : table index file.
It can also be seen from here that MYISAM stores index and data files separately, while InnoDB stores index files as data files.
【2】MySQL8
① InnoDB
As shown below, there are only ibd files and nothing else. That is, database information, table structure, table data, and table indexes are all stored in the ibd file.
MYISAM
We create a MYIASM table, insert data, and view the data storage file as follows:
You can see that it has three files: sdi, MYD, and MYI. sdi is equivalent to the frm file we mentioned earlier, and it also stores table structure information.
③ about sdi
In MySQL8 we can see that there is no .frmfile, so where did it go? Just put it in the sdi file. Oracle official (MySQL8 belongs to Oracle) moves the frm file information and more information to the serialized dictionary information (Serialized Dictionary Information, SDI). SDI is written inside the ibd file under InnoDB.
In order to extract SDI information from ibd files, oracle provides an application ibd2sdi. This tool does not need to be downloaded, MySQL8 comes with it, as long as the environment variables are configured, it can be used.
We can go to the directory where the ibd file is stored and execute the following command:
ibd2sdi --dump-file=tb_sys_admin.txt tb_sys_admin.ibd
[3] Table space
InnoDB actually uses pages as the basic unit to manage storage space, and the default page size is 16KB. For the InnoDB storage engine, each index corresponds to a B+ tree, and each node of the B+ tree is a data page, and the data pages do not need to be physically continuous, because there is a doubly linked list between the data pages To maintain the order of these pages.
The leaf nodes of InnoDB's clustered index (that is, the primary key index file) store complete user records.
In order to better manage these pages, InnoDB proposes a 表空间concept 文件空间of (table space or file space). This table space is an abstract concept, which can correspond to one or more real files on the file system (the number of files corresponding to different table spaces may be different). Each table space can be divided into many 页, and our table data is stored in 表空间some under one 页.
① System table space
By default, InnoDB will create a file named ibdata1 with a size of 12M in the data directory. This file is the representation of the corresponding system table space on the file system. It is a self-expanding file, and it will increase its size when it is not enough.
We can configure the corresponding file paths and their sizes when MySQL starts. For example, my.cnf is configured as follows:
[server]
innodb_data_file_path=data1:512M;data2:512M:autoextend
In this way, these two 512M files will be created as the system tablespace after MySQL starts, and autoextend indicates that if these two files are not enough, the size of the data2 file will be automatically extended.
One thing to note is that in a MySQL server, there is only one copy of the system tablespace. In each version from MySQL5.5.7 to MySQL5.6.6, the data in our table will be stored in this system tablespace by default.
② independent table space (file-per-table-tablespace)
In MySQL5.6.6 and later versions, InnoDB does not store the data of each table in the system table space by default, but creates an independent table space for each table, that is to say, how many tables we have created, There are as many independent table spaces as there are.
.ibdIf you use an independent table space to store table data, a file representing the independent table space will be created in the subdirectory corresponding to the database to which the table belongs. The file name is the same as the table name, but an extension of is added.
③ Setting of system table space and independent table space
We can specify whether to use the system tablespace or an independent tablespace to store data. This function is innodb_file_per_tablecontrolled by startup parameters. For example, if we want to deliberately store all table data in the system tablespace, we can configure it like this when starting the MySQL server:
[server]
innodb_file_per_table=0 # 0 : 使用系统表空间；1 ： 使用独立表空间
The default value is 1, you can use the following command to view:
show variables like '%innodb_file_per_table%'
The modification of this parameter takes effect on newly created tables, but not on tables that have been allocated table spaces. If we want to transfer a table that already exists in the system tablespace to an independent tablespace, we can use the following syntax:
alter table 表名 tablespace [=] innodb_file_per_table;
Or transfer a table that already exists in an independent tablespace to the system tablespace, you can use the following syntax:
alter table 表名 tablespace [=] innodb_system;
④ Other types of table spaces
With the development of MySQL, in addition to the above two old table spaces, some different types of table spaces are now proposed, such as general table space (general tablespace), temporary tablespace (temporary tablespace), etc.