[Mysql] MySQL architecture, InnoDB, MyISAM storage engine, index structure, classification, syntax, performance analysis

1. MySQL architecture

1. connection layer

   It mainly completes some connection processing, authorization authentication, and related security solutions. The concept of thread pool is introduced on this layer to provide threads for clients that securely access through authentication. SSL-based secure links can also be implemented on this layer. The server also verifies the operating permissions it has for each client that securely accesses it.

2. service layer

   The second layer architecture mainly completes most of the core service functions, such as SQL interface, and completes cached queries, SQL analysis and optimization, and the execution of some built-in functions. All cross-storage engine functions are also implemented in this layer, such as procedures, functions, etc. At this layer, the server will parse the query and create the corresponding internal parse tree, and complete the corresponding optimization such as determining the order of table queries, whether to use indexes, etc., and finally generate the corresponding execution operations. If it is a select statement, the server will also query the internal cache. If the cache space is large enough, this can greatly improve the performance of the system in an environment that solves a large number of read operations.

3. Engine layer [storage engine]

   Storage engine layer, the storage engine is really responsible for the storage and retrieval of data in MySQL, and the server communicates with the storage engine through API. Different storage engines have different functions, so we can choose the appropriate storage engine according to our needs. After MySQL5.5, the default storage engine of MySQL is InnoDB. The default index structure used by InnoDB is the B+ tree. The above service layer interacts with the storage engine layer through the API interface.

4. storage layer

   The data storage layer mainly stores data on the file system and completes the interaction with the storage engine.

   Compared with other databases, MySQL is a little different. Its architecture can be applied and work well in many different scenarios. Mainly reflected in the storage engine, the plug-in storage engine architecture separates query processing from other system tasks and data storage and extraction. This architecture allows the selection of appropriate storage engines based on business needs and actual needs.

2. Storage engine

Unlike most databases, MySQL has the concept of a storage engine, and the optimal storage engine can be selected for different storage requirements .

The storage engine is the implementation of technologies such as storing data, creating indexes, updating query data, etc.The storage engine is based on tables, rather than library-based.

Databases such as Oracle and SqlServer have only one storage engine. MySQL provides a plug-in storage engine architecture. Therefore, MySQL has multiple storage engines, and you can use the corresponding engine according to your needs.

You can query the storage engines supported by the current database by specifying show engines:

show engines;

If you do not specify a storage engine when creating a new table, the system will use the default storage engine. The default storage engine before MySQL 5.5 was MyISAM, and after 5.5 it was changed to InnoDB.

View the default storage engine of Mysql database, command:

show variables like '%storage_engine%';

2.1. InnoDB

InnoDB storage engine is the default storage engine of Mysql. The InnoDB storage engine provides transaction security with commit, rollback, and crash recovery capabilities. However, compared with MyISAM's storage engine, InnoDB's write processing efficiency is less efficient, and it takes up more disk space to retain data and indexes.

The characteristics of InnoDB storage engine are different from other storage engines: transaction control , foreign key constraints , storage method

Foreign key constraints: MySQL storage engine that supports foreign keysOnly InnoDB, when creating a foreign key, the parent table must have a corresponding index, and the child table will automatically create a corresponding index when creating a foreign key.

Storage method
InnoDB stores tables and indexes in the following two ways:

①. Use shared table space storage. The table structure of the table created in this way is saved in the .frm file. The data and indexes are saved in the table space defined by innodb_data_home_dir and innodb_data_file_path, which can be multiple files.

②. Use multi-table space storage. The table structure of the table created in this way still exists in the .frm file, but the data and indexes of each table are saved separately in .ibd.
/var/lib/mysql/demo01

2.2. MyISAM

MyISAM does not support transactions or foreign keys. Its advantage is that it has fast access speed and does not require the integrity of transactions. Basically, applications based on SELECT and INSERT can use this engine to create tables. It has the following two important features: it does not support transactions and it does not support foreign keys.

File storage method

Each MyISAM is stored as 3 files on the disk. The file names are the same as the table names, but the extensions are:

.frm (storage table definition);

.MYD(MYData, storage data);

.MYI(MYIndex, storage index);

2.3. Storage engine selection

When selecting a storage alarm, you should select an appropriate storage alarm based on the characteristics of the application system. For complex application systems, multiple storage alarms can be selected and combined according to the actual situation.

lnnoDB : is the default storage engine of Mysql, supporting transactions and foreign keys. If the application has relatively high requirements for transaction integrity and requires data consistency under concurrent conditions,In addition to insertion and query, data operations also include many update and delete operations.Then the InnoDB storage engine is a more suitable choice.

MyISAM :If the application is mainly based on read operations and insert operations, with only a few update and delete operations, and the requirements for transaction integrity and concurrency are not very high,Then choosing this storage engine is very suitable.

3. Index

3.1 Index overview

MySQL’s official definition of index is: Index is a data structure (ordered) that helps MySQL obtain data efficiently. In addition to data, the database system also maintains data structures that satisfy specific search algorithms. These data structures reference (point to) the data in some way, so that advanced search algorithms can be implemented on these data structures. This data structure is an index . . As shown in the diagram below:

On the left is a data table with a total of two columns and seven records. The leftmost one is the physical address of the data record (note that logically adjacent records are not necessarily physically adjacent on the disk). In order to speed up the search of Col2, you can maintain a binary search tree as shown on the right. Each node contains the index key value and a pointer to the physical address of the corresponding data record. In this way, you can use binary search to quickly obtain the corresponding data.

Generally speaking, the index itself is also very large and cannot be stored entirely in memory, so the index is often stored on disk in the form of an index file. Indexes are the most common tool used in databases to improve performance .

3.2. Index advantages and disadvantages

Advantage:

1. Improve the efficiency of data retrieval and reduce the IO cost of the database.
2. Sort data through index columns to reduce the cost of data sorting and reduce CPU consumption.

Disadvantages:

1. Index columns also take up space.
2. Indexes greatly improve query efficiency, but at the same time, they also reduce the speed of updating tables. For example, when performing INSERT, UPDATE, and DELETE on the table, the efficiency decreases.

3.3. Index structure

Indexes are implemented in MySQL's storage engine layer, not in the service layer. Therefore, the indexes of each storage engine are not necessarily exactly the same, and not all storage engines support all index types. MySQL currently provides the following 4 index types:

• B+TREE index: The most common index type, most indexes support B-tree indexes.
• HASH index: only supported by the Memory engine, the usage scenario is simple.
• R-tree index (spatial index): Spatial index is a special index type of the MyISAM engine. It is mainly used for geospatial data types. It is usually less used and will not be specially introduced.
• Full-text (full-text index inverted index – ES): Full-text index is also a special index type of MyISAM, mainly used for full-text index. InnoDB supports full-text index starting from Mysql5.6 version.

MyISAM, InnoDB, and Memory storage engines support various index types

index	InnoDB engine	MyISAM engine	Memory engine
B+TREE index	support	support	support
HASH index	not support	not support	support
R-tree index	not support	support	not support
Full-text	Supported after version 5.6	support	not support

The indexes we usually refer to, unless otherwise specified, refer to indexes organized in a B+ tree (multi-way search tree, not necessarily binary) structure. Among them, clustered index, compound index, prefix index and unique index all use B+tree index by default, which are collectively called indexes.

3.3.0. Binary tree

3.3.1. B-TREE structure

Dynamic demonstration: https://www.cs.usfca.edu/~galles/visualization/Algorithms.html

Comic understanding B-Tree: https://zhuanlan.zhihu.com/p/54084335

3.3.2. B+Tree structure

Comic understanding B+Tree: https://zhuanlan.zhihu.com/p/54102723

3.3.3. B+Tree in database

Why does the InnoDB storage engine choose to use the B+tree index structure?

1. Compared with binary trees, there are fewer levels and high search efficiency.
2. For B-tree, data will be saved regardless of whether it is a leaf node or a non-leaf node. This will reduce the key values ​​stored in a page, and the pointers will also decrease. To save a large amount of data, you can only increase the height of the tree, resulting in reduced performance:
3. Compared with Hash index, B+tree supports range matching and sorting operations.

3.4. Index classification

Index classification

Classification	meaning	Features	Keywords
primary key index	Index created on the primary key in the table	Automatically created by default, there can only be one	PRIMARY
unique index	Avoid duplication of values ​​in a data column in the same table	There can be multiple	UNIQUE
regular index	Quickly locate specific data	There can be multiple
Full text index	Full-text indexing searches for keywords in the text rather than comparing values ​​in the index	There can be multiple	FULL TEXT

In InnoDB storage alerts, according to the storage form of the index, it can be divided into the following two types:

Classification	meaning	Features
Clustered Index	Putting data storage and index together, the leaf nodes of the index structure store row data	There must be, and only one
Secondary Index	Store data and indexes separately. The leaf nodes of the index structure are associated with the corresponding primary keys.	There can be multiple

3.5. Index syntax

1. Create index

CREATE [UNIOUE|FULLTEXT] INDEX index_name ON table_name ( index_col_name,... );

2. View index

SHOW INDEX FROM table_name;

3. Delete index

DROP INDEX index_name ON table_name;

The following is the creation statement of the student table. Please create relevant indexes according to your needs.

create table student(
id int primary key,
	 name varchar(20),
	 phone varchar(11),
	 email varchar(50),
	 profession varchar(30),
	 age int,
	 gender TINYINT,
	 status TINYINT,
	 createtime datetime
);

1. The name field is the name field. The value of this field may be repeated. Create an index for this field.
2. The value of the phone number field is non-empty and unique. Create a unique index for this field.
3. Create joint indexes for profession, age, and status.
4. Establish appropriate indexes for emails to improve query efficiency.

---

1. Create an index for the name field:

CREATE INDEX idx_name ON student(name);

2. Create a unique index for the phone field:

CREATE UNIQUE INDEX idx_phone ON student(phone);

3. Create a joint index for profession, age and status:

CREATE INDEX idx_profession_age_status ON student(profession, age, status);

4. Create an appropriate index for the email field:

CREATE INDEX idx_email ON student(email);

3.6. Performance analysis

3.6.1. SQL execution frequency

After the MVSOL client is successfully connected, server status information can be provided through the show sessionloloball status command. Through the following commands, you can check the access frequency of INSERT, UPDATE, DELETE, and SELECT of the current database:

SHOW GLOBAL STATUS LIKE 'Com___';

3.6.2. Slow query log

The slow query log records the logs of all SQL statements whose execution time exceeds the specified parameter (long_query_time, unit: seconds, default 10 seconds). MySQL's slow query log is not enabled by default and needs to be configured in the MySQL configuration file (/etc/my.cnf ) configure the following information:

# 开启MySQL慢日志查询开关
slow_query_log=1
# 设置慢日志的时间为2秒，SOL语句执行时间超过2秒，就会视为慢查询，记录慢查询日志
long_query_time=2

3.6.3.profile details

show profiles can help us understand where time is spent when doing SL optimization. Through the have profiling parameter, you can see whether the current MySQL supports profile operations:

SELECT @@have_profiling ;

Profiling is turned off by default. Profiling can be turned on at the session/global level through the set statement:

SET profiling = 1;

Execute a series of business SQL operations, and then check the execution time of the instructions through the following instructions:

#View the basic time-consuming situation of each SOL

show profiles;

#View the time-consuming status of each stage of the SQL statement with the specified query id

show profile for query query_id;

#Check the CPU usage of the SQL statement with the specified query id

show profile cpu for query query_id;

3.6.4. The explain execution plan
EXPLAIN or DESC command obtains information about how MySQL executes the SELECT statement, including how tables are connected and the order in which they are connected during the execution of the SELECT statement. grammar:

# 直接在select语句之前加上关键字 explain / desc
EXPLAIN SELECT 字段列表 FROM 表名 WHERE 条件

• The sequence number of the id
  select query indicates the order in which the select clause or table operation is executed in the query (the same id, the execution order is from top to bottom; the id is different, the larger the value, the earlier it is executed).
• select_type
  represents the type of SELECT. Common values ​​include SIMPLE (simple table, that is, no table connection or subquery is used), PRIMARY (main query, that is, the outer query), UNION (the second or subsequent query in UNION) statement), SUBQUERY (subquery included after SELECT/WHERE), etc.
• type
  represents the connection type. The connection types with good to poor performance are NULL, System, const, eq_ref, ref, range, index, and all.
• possible_keys
  displays one or more indexes that may be applied to this table.
  Generally speaking, we need to ensure that the query reaches at least range level, preferably ref.
• The index actually used by key
  . If it is NULL, no index is used.
• key_len
  represents the number of bytes used in the index. This value is the maximum possible length of the index field, not the actual length used. Without losing accuracy, the shorter the length, the better.
• rows
  The number of rows that MySQL considers necessary to execute the query. In the table where innodb raises the alarm, it is an estimate and may not always be accurate.
• filtered
  indicates the number of rows returned as a percentage of the number of rows to be read. The larger the value of filtered, the better.