MySQL-4-storage engine (combined with case study)

MySQL architecture

1. Connection layer
   The top layer is some client and connection services
   , including local sock communication and most communication similar to TCP/IP based on client/server tools . It mainly completes some similar connection processing, authorization authentication, and related security solutions. On this layer, the concept of thread pool is introduced to provide threads for clients that pass authentication and secure access. SSL-based secure links can also be implemented on this layer. The server also verifies the operating authority it has for each client that accesses securely.
2. Service layer
   The second-tier architecture mainly completes most of the core service functions , such as SQL interface, and completes cached queries, SQL analysis and optimization, and
   execution of some built-in functions. All cross-storage engine functions are also implemented at this layer, procedures, functions, etc. At this layer, the server will parse
   the query and create a corresponding internal parsing tree, and optimize it accordingly, such as determining the order of table queries, whether to use indexes, etc., and finally
   generate corresponding execution operations. If it is a select statement, the server will also query the internal cache. If the cache space is large enough,
   this can greatly improve the system performance in an environment that solves a large number of read operations.
3. Engine layer
   Storage engine layer, the storage engine is really responsible for the storage and retrieval of data in MySQL, and the server communicates with the storage engine through API
   . Different storage engines have different functions, so we can choose the appropriate storage engine according to our needs. Indexes
   in databases are implemented at the storage engine layer.
4. Storage layer
   The data storage layer mainly stores data (such as: redolog, undolog, data, index, binary log, error log, query log, slow query log,
   etc.) on the file system and completes the interaction with the storage engine.
   Compared with other databases, MySQL is a bit different in that its architecture can be applied and function well in many different scenarios. It is mainly
   reflected in the storage engine. The plug-in storage engine architecture separates query processing from other system tasks and data storage and extraction.
   This architecture can select an appropriate storage engine according to business needs and actual needs.

storage engine

Storage Engine Introduction

The storage engine is the core of the mysql database, and it is necessary to select a suitable storage engine in a suitable scenario.
The storage engine is the implementation of technologies such as storing data, building indexes, and updating/querying data.
The storage engine is based on tables, not libraries , so storage engines can also be called table types.
We can specify the selected storage engine when creating a table. If not specified, the default storage engine will be automatically selected.

Through the table creation statement, we can see the following information:

ENGINE is the engine, here is the default InnoDB (AUTO_INCREMENT=6 means that when the next data is inserted, the id with the automatic growth attribute will be 6, CHARSET is the character encoding of the character set, and COLLATE is the sorting method).

grammar

Display the storage engines supported by the current database

show engines;

Among them, MySQL's early default storage engine was MyISAM, and the current default storage engine is InnoDB.

Specify the storage engine when creating the table

statement

CREATE TABLE 表名( 
字段1 字段1类型 [ COMMENT 字段1注释 ] , 
...... 
字段n 字段n类型 [COMMENT 字段n注释 ] 
) ENGINE = INNODB [ COMMENT 表注释 ] ;

• Create the table my_myisam and specify the MyISAM storage engine

create table my_myisam(
    id int,
    name varchar(10)
)engine = MyISAM ;

• Create table my_memory, specify Memory storage engine

create table my_memory(
    id int,
    name varchar(10)
)engine = MEMORY ;

Storage Engine Features

Mainly talk about the characteristics of the three storage engines InnoDB, MyISAM, and Memory:

InnoDB

1 Introduction

InnoDB is a general-purpose storage engine that takes into account high reliability and high performance. After MySQL 5.5, InnoDB is the default
MySQL storage engine.

2. Features

• DML operations follow the ACID model and support transactions ;
• Row-level locks to improve concurrent access performance;
• Support foreign key FOREIGN KEY constraints to ensure data integrity and correctness;

3. Documentation

xxx.ibd: xxx represents the table name. Each table in the innoDB engine will correspond to such a table space file, which stores the table structure
(frm-early, sdi-new version), data and indexes of the table .
Parameter: innodb_file_per_table (now enabled by default)
Query: show variables like 'innodb_file_per_table';

If this parameter is enabled, it means that for the tables of the InnoDB engine, each table corresponds to an ibd file .
We directly open the MySQL data storage directory (the unmodified default location is C:\ProgramData\MySQL\MySQL Server 8.0\Data), there are many folders in this directory, and different folders represent different databases. We directly open the study folder.
You can see that there are many ibd files in it, each ibd file corresponds to a table, such as the table account, there is such an account.ibd file, and in this ibd file not only the table structure and data are stored, but also Index information corresponding to this table.
The file is based on binary storage and cannot be opened directly based on Notepad. You can use the command ibd2sdi provided by mysql to extract sdi information from the ibd file, and the sdi data dictionary information contains the table structure of the table. .
To view the table structure of ibd, you can use the following methods:

1. Select the path box above, and enter cmd at the beginning of the path, which means to open the command line window under this path
   
2. Then enter ibd2sdi account.ibd, you can view the basic information of this table
   

4. Logical storage structure

• Tablespace: The highest level of the logical structure of the InnoDB storage engine. The ibd file is actually a tablespace file, which can contain multiple segments.
• Segment: A tablespace is composed of various segments. Common segments include data segment, index segment, and rollback segment. The management of segments in InnoDB is done by the engine itself, without manual control. A segment contains multiple areas.
• Area: Area is the unit structure of table space, and the size of each area is 1M. By default, the InnoDB storage engine page size is 16K, that is, there are 64 consecutive pages in an area.
• Page: A page is the smallest unit of an area, and a page is also the smallest unit of InnoDB storage engine disk management. The default size of each page is 16KB. In order to ensure the continuity of pages, the InnoDB storage engine applies for 4-5 areas from the disk each time.
• Row: The InnoDB storage engine is row-oriented, that is to say, data is stored by row. In addition to the fields specified when defining the table, each row contains two hidden fields (described in detail later).

MyISAM

1 Introduction

MyISAM was the default storage engine in the early days of MySQL.

2. Features

• Does not support transactions, does not support foreign keys
• Table locks are supported, but row locks are not supported
• fast access

3. Documentation

• xxx.sdi: store table structure information
• xxx.MYD: storage data
• xxx.MYI: storage index

  Here, we can directly view the sdi file, because the sdi file stores a dictionary in json format, so we can open the sdi file with Notepad

  and copy it to jscn.cn to convert it into a json file to view the content

Memory

1 Introduction

The table data of the Memory engine is stored in memory . Due to hardware problems or power failure problems, these tables can only be used as temporary tables or caches .

2. Features

• memory storage
• hash index (default)

3. Documentation

Only xxx.sdi: used to store table structure information, because its data is stored in memory, and will not be saved directly in the form of files.

Storage Engine Features

The focus is on the three major differences between InnoDB and MyISAM (transactions, row-level locks, foreign keys).

features	InnoDB	MyISAM	Memory
storage limit	64TB	have	have
transaction security	support	-	-
lock mechanism	row lock	table lock	table lock
B+tree index	support	support	support
Hash index	-	-	support
full text index	Supported (after version 5.6)	support	-
space use	high	Low	N/A
memory usage	high	Low	medium
Bulk Insertion Speed	Low	high	high
Support for foreign keys	support	-	-

Interview question:
What is the difference between the InnoDB engine and the MyISAM engine?
①. The InnoDB engine supports transactions, but MyISAM does not.
②. InnoDB engine supports row locks and table locks, while MyISAM only supports table locks and does not support row locks.
③. The InnoDB engine supports foreign keys, but MyISAM does not.
The main difference is the above three points. Of course, you can also answer more in-depth from the aspects of index structure and storage limit. For details, refer to
the following official documents:
innodb
myisam

Choice of storage engine

When selecting a storage engine, an appropriate storage engine should be selected according to the characteristics of the application system. For complex application systems, multiple storage engines can also be selected for combination according to the actual situation .

• InnoDB: If the application has relatively high requirements on the integrity of things, and requires data consistency under concurrent conditions . In addition to inserting and querying, data operations also include many update and delete operations, then InnoDB is a more suitable choice , because only InnoDB supports transactions .
• MyISAM: If the application is mainly read and insert operations , with only a few update and delete operations, and does not require high transaction integrity and concurrency, then this storage engine is very suitable.
• Memory: Store all data in memory, with fast access speed , usually used for temporary tables and caches. The disadvantage of Memory is that there is a limit to the size of the table. Too large a table cannot be cached in memory (other storage engines directly store data in the disk, and the size of the disk can be stored), and the security of the data cannot be guaranteed.

Footprints and comments in e-commerce are suitable for using the MyISAM engine, and cache is suitable for using the Memory engine.

Summarize