write in front
Many small partners have been working for a long time, but their mastery of MySQL is only on the surface of CRUD, and they have little understanding of the deep principles and technical knowledge of MySQL. With the continuous increase of working years, the competitiveness of the workplace is not is continuously decreasing. Many times, when you go out for an interview, the phenomenon of being hanged and beaten by the interviewer has become a common occurrence.
Not just an interview, if you want to rise from a low-level programmer to a senior engineer, architect, etc., the underlying principles and technologies of MySQL are what you must master.
Let's learn the architecture of MySQL together today.
MySQL Architecture
Let's first take a look at the architecture diagram of MySQL, as shown below.
Note: The picture comes from the Internet
From the architecture diagram of MySQL, we can see that the architecture of MySQL can be roughly divided into four parts: network connection layer, database service layer, storage engine layer and system file layer from top to bottom. Next, let's briefly talk about the composition information of each part.
network connection layer
The network connection layer is located at the top layer of the entire MySQL architecture and mainly acts as a client connector. Provides the ability to establish a connection with the MySQL server, and supports almost all mainstream server-side languages, such as Java, C, C++, Python, etc. Each language establishes a connection with MySQL through its own API interface.
database service layer
The database service layer is the core of the entire database server, mainly including system management and control tools, connection pool, SQL interface, parser, query optimizer and cache.
connection pool
It is mainly responsible for storing and managing the connection information between the client and the database. A thread in the connection pool is responsible for managing the connection information from a client to the database.
System management and control tools
Provides database system management and control functions, such as backing up and restoring data in the database, ensuring the security of the entire database, providing security management, and coordinating and managing the entire database cluster.
SQL interface
It is mainly responsible for receiving various SQL commands sent by the client, sending the SQL commands to other parts, receiving the result data returned by other parts, and returning the result data to the client.
parse tree
It is mainly responsible for parsing the requested SQL into a "parse tree", and then performs further syntax verification on the "parse tree" according to some rules in MySQL to confirm whether it is legal.
query optimizer
In MySQL, if the "parse tree" passes the parser's grammar check, the optimizer will convert it into an execution plan, and then interact with the storage engine and the underlying data files through the storage engine.
cache
MySQL's cache is composed of a series of small caches. For example: MySQL's table cache, record cache, privilege cache in MySQL, engine cache, etc. The cache in MySQL can improve the query performance of data. If the result of the query can hit the cache, MySQL will directly return the result information in the cache.
storage engine layer
The storage engine layer in MySQL is mainly responsible for writing and reading data and interacting with the underlying files. It is worth mentioning that the storage engine in MySQL is plug-in, and the query execution engine in the server communicates with the storage engine through the relevant interface. At the same time, the interface shields the differences between different storage engines. In MySQL, the most commonly used storage engines are InnoDB and MyISAM.
InnoDB and MyISAM storage engines need to be mastered by small partners. The high-frequency interview test site is also a must-know content for architects.
system file layer
The system file layer mainly includes the underlying files that store data in MySQL, interacts with the upper-layer storage engine, and is the physical storage layer of files. The files it stores mainly include: log files, data files, configuration files, MySQL pid files and socket files, etc.
log file
The logs in MySQL mainly include: error log, general query log, binary log, slow query log, etc.
-
error log
The main storage is the error information generated during the operation of MySQL. The error log in MySQL can be viewed using the following SQL statement.
show variables like '%log_error%';
-
General query log
It mainly records general query information during the running process of MySQL. You can use the following SQL statement to view the general query log file in MySQL.
show variables like '%general%';
-
binary log
It mainly records the insertion, modification, and deletion operations performed on the MySQL database, and also records the execution time and execution time of the SQL statement, but the binary log does not record the SQL that does not modify the database, such as select and show. It is mainly used to restore database data and implement MySQL master-slave replication.
Check whether binary logging is enabled.
show variables like '%log_bin%';
View the parameters of the binary log.
show variables like '%binlog%'
View log files.
show binary logs;
-
slow query log
The slow query mainly records SQL statements whose execution time exceeds the specified time, which is 10 seconds by default.
Check whether the slow query log is enabled.
show variables like '%slow_query%';
Check the duration of the slow query setting.
show variables like '%long_query_time%'
data files
The data files mainly include: db.opt file, frm file, MYD file, MYI file, ibd file, ibdata file, ibdata1 file, ib_logfile0 and ib_logfile1 files, etc.
-
db.opt file
It mainly records information such as character set and inspection rules used by the current database.
-
frm file
The structure information of the data table is stored, mainly the metadata information related to the data table, including the table structure definition information of the data table. Each table will have a frm file.
It is worth noting that there is no frm file for the tables of the innodb storage engine in the MySQL8 version. (I will write some articles on the new features of MySQL 8 later, from the usage to the underlying principles and how they are different from MySQL 5).
-
MYD file
The file format dedicated to the MyISAM storage engine mainly stores the data in the MyISAM storage engine data table. Each MyISAM storage engine table corresponds to a .MYD file.
-
MYI file
The file format dedicated to the MyISAM storage engine mainly stores the index information related to the MyISAM storage engine data table. Each MyISAM storage engine table corresponds to a .MYI file.
-
ibd file
Store the data files and index files of the Innodb storage engine, mainly store the data and indexes of the exclusive table space, and each table corresponds to an .ibd file.
-
ibdata file
The data files and index files of the Innodb storage engine are stored, mainly storing the data and indexes of the shared tablespace. All tables share one (or more) .ibdata files. The number of shared .ibdata files can be specified according to the configuration.
-
ibdata1 file
The MySQL system tablespace data file mainly stores MySQL data table metadata, Undo logs and other information.
-
ib_logfile0 and ib_logfile1 files
The Redo log file in the MySQL database is mainly used for MySQL to achieve transaction durability. If MySQL fails at a certain point in time, and if there are dirty pages that are not written to the ibd file of the database, when MySQL is restarted, MySQL will redo the information according to the Redo Log, which will be written to the Redo Log and has not been written yet. The data written to the data table is persisted.
configuration file
It is used to store all the configuration information of MySQL. It is the my.cnf file in the Unix/Linux environment and the my.ini file in the Windows environment.
pid file
The pid file is a file that stores the process ID of the MySQL process when it is running. It mainly exists in the Unix/Linux environment. The specific storage directory can be configured in the my.cnf or my.ini file.
socket file
The socket file, like the pid file, is a file only available when MySQL runs in a Unix/Linux environment. In a Unix/Linux environment, clients can connect to MySQL directly through sockets.
Reprinted: Understand MySQL architecture in one article!