To understand the operating mechanism of mysql, you must first have a certain understanding of the mysql architecture.
Recently, due to some things, I have been hit hard. I feel that no matter how hard I try, my career will be like this. So I suddenly lost my original passion for researching technology and blogging.
But anyway, the road is chosen by myself, and I have to finish it when I cry. Besides, there seems to be no other way.
mysql architecture
From the figure, it can be seen that MySQL consists of the following parts:
- Connection pool component
- Management services and tool components
- SQL interface components
- Query Analyzer component
- Optimizer component
- Cache component
- Plug-in storage engine
- Physical file
Note: One
of the most important features that distinguishes MySql database from other databases is its plug-in table storage engine .
Note: The
storage engine is table-based, not database-based.
Based on this feature, most of the business databases in the project need to support transaction attributes, so the InnoDB storage engine is used. But for those tables with read-only characteristics, we can consider using the MyISAM storage engine, so that it will not destroy the transaction characteristics of the database.
Detailed description
The uppermost layer is the connection component. The following server is composed of connection pool, management tools and services, SQL interface, parser, optimizer, cache, storage engine, and file system.
Connection pool : Since it takes a lot of time to establish each time, the role of the connection pool is to cache these connections, and you can directly use the established connections next time to improve server performance.
Management tools and services : system management and control tools, such as backup and recovery, Mysql replication, clusters, etc.
SQL interface : accepts user's SQL commands and returns the results that users need to query. For example, select from is to call the SQL Interface
query parser : when the SQL command is passed to the parser, it will be verified and parsed by the parser (permissions, grammatical structure). The
main functions:
a. Decompose the SQL statement into a data structure and pass this structure In the subsequent steps, the subsequent transmission and processing of SQL statements is based on this structure.
b. If errors are encountered in the decomposition composition, then it means that the SQL statement is an unreasonable
query optimizer : query optimizer, SQL statement before query Will use the query optimizer to optimize the query. He uses the **"select-projection-join"** strategy to query.
It can be understood with an example: select uid,name from user where gender = 1;
This select query first selects based on the where statement, instead of first querying all the tables and then performing gender filtering.
This select query is based on uid and name. Attribute projection, instead of filtering all the attributes later,
join these two query conditions to generate the final query result.
In addition, the query optimizer can also control which index a SQL statement queries to, and automatically adjust the order of where conditions in the SQL statement to meet the rules of the leftmost matching principle of the composite index.
Cache : Query cache ( removed after MySQL 8.0 version ). If the query cache has a matching query result, the query statement can go directly to the query cache to fetch data.
The cold end of the data overflows through the LRU algorithm, and the data pages that must be refreshed to the disk in time in the future are called dirty pages.
This caching mechanism is composed of a series of small caches. Such as table cache, record cache, key cache, permission cache, etc.
It is not recommended to use cache for MySQL queries, because query cache invalidation may be very frequent in actual business scenarios. If you update a table, all query caches on this table will be cleared.
Therefore, we generally do not recommend using query caching in most cases.
After MySQL 8.0 version, the cache function was deleted. The official believes that this function is less in actual application scenarios, so it is simply deleted.
Plug-in storage engine : The storage engine is a way to manage operating data (store data, how to update, query data, etc.). Because the storage of data in a relational database is stored in the form of a table, the storage engine can also be called a table type (that is, the type of storage and operation of this table)
Physical files :
physical files include: log files, data files, configuration files, pid files, socket files, etc.
The execution flow of a query statement
Having said all the above, how is an sql statement executed? In fact, our sql can be divided into two types, one is query and the other is update (add, update, delete). We first analyze the query statement, the statement is as follows:
select * from tb_student A where A.age='18' and A.name=' 张三 ';
Combined with the above description, we analyze the execution flow of this statement:
1. Establish a connection
2. Call the sql interface
3. Authentication: first check whether the statement has permission, if there is no permission, return an error message directly
4. Cache judgment: if If you have permission, before MySQL8.0, the cache will be queried first. Use this SQL statement as the key to query whether there is a result in the memory. If there is a direct cache, if not, go to the next step.
5. Parse the sql statement:
perform lexical analysis through the analyzer to extract the key elements of the sql statement. For example, the above statement is to query select, and the table to be queried is named tb_student, and all columns need to be queried. The query conditions are for this table. id='1'. Then judge whether there are grammatical errors in the sql statement, such as whether the keywords are correct, etc., if the check is no problem, proceed to the next step.
6. Query optimization Perform sql on the sql statement.
Next is the optimizer to determine the execution plan. The above sql statement can have two execution plans:
a. First query the student named "Zhang San" in the student table, and then determine whether The age is 18.
b. First find out the 18-year-old student among the students, and then search for the student named "Zhang San".
Then the optimizer selects the most efficient solution based on its own optimization algorithm (the optimizer believes that sometimes it may not be the best). After confirming the execution plan, it is ready to start execution.
7. Execute the query and return the result
Physical file description
1. Log file
error log Error log troubleshooting /var/log/mysqld.log [enabled by default]
bin log Binary log backup incremental backup DDL DML DCL
Relay log Relay log replication reception replication master
slow log slow query log tuning query Time exceeds the specified value
The default location of the log file:
Error Log
log-error=/var/log/mysqld.log
Binary Log
log-bin=/var/log/mysql-bin/bin.log
server-id=2
Slow Query after version 5.7.x Log
slow_query_log=1|0
slow_query_log_file=/var/log/mysql-slow/slow.log
long_query_time=3
2. If the configuration file
/etc/my.cnf—>/etc/mysql/my.cnf—>/usr/etc/my.cnf—> ~/.my.cnf
conflicts, the last one read shall prevail
[ The mysqld] group includes the initialization parameters when the mysqld service is started. The
[client] group contains the parameters that can be read by the client tool program, in addition to other specific parameter groups for each client software, etc.
3. Data file
1.
No matter what storage engine the .frm file is, each table will have a .frm file named after the table name, and the metadata (meta) information related to the table is stored in this file, including the table structure Definition information, etc.
2. The .MYD file is
dedicated to the myisam storage engine and stores the data of the myisam table. Each myisam table will have a .MYD file corresponding to it, which is also stored in the directory of its own database.
3. The .MYI file
is also dedicated to the myisam storage engine and stores the index related information of the myisam table. For the myisam storage engine, the content that can be cached is mainly derived from the .MYI file.
Each myisam table corresponds to a .MYI file, and its storage location is the same as .frm and .MYD4
. .ibd files and ibdata files
store innoDB data files (including indexes). The innoDB storage engine has two table space methods: exclusive table space and shared table space.
Exclusive table space: Use .ibd files to store data, and each table has an .ibd file, which is stored in the same location as myisam data.
Shared table space: Use .ibdata files, and all tables share one (or more, self-configured) ibdata files.
ibdata1: system tablespace (data file) undo segment
ib_logfile0 redlog file group
ib_logfile1
db.opt file This file will be present in every self-built library, and it records the default character set and verification rules of this library.
4. The pid file The
pid file is a process file of the mysqld application in the Unix/Linux environment. Like many other Unix/Linux server programs, it stores its own process id.
5. Socket file
Socket file is only available in Unix/Linux environment. In Unix/Linux environment, users can directly use Unix Socket to connect to mysql instead of TCP/IP network.
Distinguish database, database instance, database server
Database:
A collection of data files on the operating system or storage. In the mysql database, the database files can be files ending with *.frm, .MYD, .MYI, *.ibd, etc. The file types are different for different storage engines.
More biased towards file storage.
The database instance (instance)
consists of background processes or threads and a shared memory area. Shared memory can be shared by running background threads.
Need to pay attention: the database instance is the real operation database.
More biased towards application operations.
The relationship between the database and the database instance:
Generally, there is a one-to-one correspondence between a database instance and a database, that is, one database instance corresponds to one database;
however, there are multiple database instances in a cluster environment that share a database.
For example, a table in a database can be distributed in multiple database instances after a table splitting strategy.
The database instance refers to the database application, and the database refers to the database created on the database instance.
For example, if a mysql application of port 3306 is deployed on the server, then it is a database instance;
if you continue to deploy a mysql application of 3309, then two instances of msql are deployed on one service.
On the mysql instance on port 3306, creating a database named order means creating a database.
The database server (database server)
deploys the server where the database instance is installed.
to sum up
1. The mysql architecture mainly includes connection pool components, management services and tool components
SQL interface components, query analyzer components, optimizer components, cache components, plug-in storage engines, and
physical files.
2. Analyze the execution process of a query statement: establish a connection—"authority verification—"query cache—"analyzer—"optimizer—"permission verification—"executor—"engine
3. Distinguish between databases, database instances, and databases server
[Happy moment]
More exciting, follow me.