Relational database management system query processing is generally divided into four phases:
See below
How to verify the correspondence between these stages in MySQL it?
Here the experimental version of the database: 5.6.16-64.2-56
OS: CentOS release 6.5
Kernel：2.6.32-431.el6.x86_64
Create a test database and table data:
root@localhost[(none)]:14: >CREATE DATABASE querydb /!40100 DEFAULT CHARACTER SET utf8 /;
Query OK, 1 row affected (0.00 sec)
root@localhost[(none)]:15: >use querydb;
Database changed
root@localhost[querydb]:20: >create table t(id int auto_increment ,name varchar(50),primary key(id)) engine=innodb;
Query OK, 0 rows affected (0.02 sec)
root@localhost[querydb]:21: >insert into t values(NULL,'a');
Query OK, 1 row affected (0.00 sec)
root@localhost[querydb]:21: >insert into t values(NULL,'b');
Query OK, 1 row affected (0.00 sec)
root@localhost[querydb]:21: >insert into t values(NULL,'c');
Query OK, 1 row affected (0.01 sec)
Open MySQL's profile
root@localhost[querydb]:21: >set @@profiling=1;
Query OK, 0 rows affected, 1 warning (0.00 sec)
First we query a normal sql statement to see what MySQL internal operations.
root@localhost[querydb]:22: >select id,name from t;
+----+------+
| id | name |
+----+------+
| 1 | a |
| 2 | b |
| 3 | c |
+----+------+
3 rows in set (0.00 sec)
root@localhost[querydb]:24: >show profiles;
+----------+------------+-----------------------+
| Query_ID | Duration | Query |
+----------+------------+-----------------------+
| 1 | 0.00103500 | select id,name from t |
+----------+------------+-----------------------+
1 row in set, 1 warning (0.00 sec)
root@localhost[querydb]:24: >show profile for query 1;
+----------------------+----------+
| Status | Duration |
+----------------------+----------+
| starting | 0.000313 |
| checking permissions | 0.000029 |
| Opening tables | 0.000073 |
| init | 0.000058 |
| System lock | 0.000066 |
| optimizing | 0.000007 |
| statistics | 0.000044 |
| preparing | 0.000025 |
| executing | 0.000002 |
| Sending data | 0.000321 |
| end | 0.000007 |
| query end | 0.000018 |
| closing tables | 0.000018 |
| freeing items | 0.000017 |
| cleaning up | 0.000038 |
+----------------------+----------+
15 ows in set, 1 warning (0.00 sec)
Generally it can be seen from the above,
First check permission, permission checks open table after the operation, then the metadata lock operation and then optimized, pre-compiled, the final implementation, when the Sending data, this time has been pushed to the storage engine for pulling data layer. Finally release lock, close the table and clean up operations.
First talk about the characteristics of each stage:
starting: parsing and lexical analysis phase
checking permissions: check user rights
Opening tables: Table permission checks
Column privileges checklist: init
System lock: lock to get some information table
optimizing: logic optimization (optimization algebraic), the main optimization RBO
statistics: physical optimization (optimization non-algebraic), mainly optimized CBO
preparing and executing: generating a code and execution
Sending data: process may also include performing, extracting and transmitting data.
First, analysis
1. Analog keyword sql error
root@localhost[querydb]:31: >selectt id,name from t;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'selectt id,name from t' at line 1
root@localhost[querydb]:31: >show profiles;
+----------+------------+------------------------+
| Query_ID | Duration | Query |
+----------+------------+------------------------+
| 1 | 0.00103500 | select id,name from t |
| 2 | 0.00022225 | selectt id,name from t |
+----------+------------+------------------------+
2 rows in set, 1 warning (0.00 sec)
root@localhost[querydb]:32: >show profile for query 2;
+---------------+----------+
| Status | Duration |
+---------------+----------+
| starting | 0.000154 |
| freeing items | 0.000026 |
| cleaning up | 0.000043 |
+---------------+----------+
3 rows in set, 1 warning (0.00 sec)
another one
root@localhost[querydb]:45: >select id,name fr0m t;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 't' at line 1
root@localhost[querydb]:45: >show profiles;
+----------+------------+------------------------+
| Query_ID | Duration | Query |
+----------+------------+------------------------+
| 1 | 0.00034325 | select id,name from t |
| 2 | 0.00006925 | selectt id,name from t |
| 3 | 0.00018800 | select id,name fr0m t |
+----------+------------+------------------------+
3 rows in set, 1 warning (0.00 sec)
root@localhost[querydb]:45: >show profile for query 3;
+---------------+----------+
| Status | Duration |
+---------------+----------+
| starting | 0.000134 |
| freeing items | 0.000018 |
| cleaning up | 0.000036 |
+---------------+----------+
3 rows in set, 1 warning (0.00 sec)
By comparison, starting should be parsing and lexical analysis.
Why parsing and lexical analysis?
In fact, we usually native language with sentences like, for example:
I went to eat the words
According to our normal human logic sentences should be: I went to dinner
But the computer is not the same, there may be a lot:
I go to eat
I go to dinner
I am going to eat
I went to eat
.......
......
The computer will be based on certain rules, like our subject-predicate accusative type sentences like from the combination, but there is a possibility that a combination of syntax errors nor that there is no lexical errors but the meaning is not the same situation. It is assumed that: I want to eat and I eat, grammar and lexical are no errors, but different semantics, then the computer will be carried out semantic analysis.
This is inside the judge rules there automaton judge, also known as the Turing machine (the great father of computer science: the invention of Turing, Turing Award surely we are all familiar with it ^ _ ^), but the dragon book (compiler theory) in It will be introduced in detail
Second, check the query
root @ localhost [querydb]: 45:> select ida, name from t; # table exists, field does not exist
ERROR 1054 (42S22): Unknown column 'ida' in 'field list'
root@localhost[querydb]:19: >show profiles;
+----------+------------+------------------------+
| Query_ID | Duration | Query |
+----------+------------+------------------------+
| 1 | 0.00034325 | select id,name from t |
| 2 | 0.00006925 | selectt id,name from t |
| 3 | 0.00018800 | select id,name fr0m t |
| 4 | 0.00096275 | select ida,name from t |
+----------+------------+------------------------+
4 rows in set, 1 warning (0.00 sec)
root@localhost[querydb]:19: >show profile for query 4;
+----------------------+----------+
| Status | Duration |
+----------------------+----------+
| starting | 0.000531 |
| checking permissions | 0.000037 |
| Opening tables | 0.000133 |
| init | 0.000116 |
| end | 0.000017 |
| query end | 0.000018 |
| closing tables | 0.000027 |
| freeing items | 0.000032 |
| cleaning up | 0.000052 |
+----------------------+----------+
9 rows in set, 1 warning (0.00 sec)
root @ localhost [querydb]: 19:> select id, name from tab; # id, name field is the table t, and the tab table does not exist
ERROR 1146 (42S02): Table 'querydb.tab' doesn't exist
root@localhost[querydb]:23: >show profiles;
+----------+------------+-------------------------+
| Query_ID | Duration | Query |
+----------+------------+-------------------------+
| 1 | 0.00034325 | select id,name from t |
| 2 | 0.00006925 | selectt id,name from t |
| 3 | 0.00018800 | select id,name fr0m t |
| 4 | 0.00096275 | select ida,name from t |
| 5 | 0.00117675 | select id,name from tab |
+----------+------------+-------------------------+
5 rows in set, 1 warning (0.00 sec)
root@localhost[querydb]:23: >show profile for query 5;
+----------------------+----------+
| Status | Duration |
+----------------------+----------+
| starting | 0.000621 |
| checking permissions | 0.000039 |
| Opening tables | 0.000367 | feeling is not where
| Query end | 0.000023 | wrong, is not it
| Closing tables | 0.000006 | less init stage
| freeing items | 0.000055 |
| cleaning up | 0.000066 |
+----------------------+----------+
7 rows in set, 1 warning (0.00 sec)
It can be observed from above, where you can test checking permissions. I'm not here to be tested. [Give users permission to test what did not: Grant Usage ON . To test @ '%' IDENTIFIED by 'test'; and gives the user test2 t only authority to query a table id column: grant select (id) on querydb.t to test2 @ '%' identified by 'test2 '; ], a little test here
checking permissions: users connect to the MySQL permission checks.
Opening tables: Table for permission to be checked.
init stage: The columns in the table permission checking.
Third, check the optimization
Experimental sql statement:
select 1;
select id,name from t ;
select id,name from t where 0=1;
Three statements to see the changes.
root@localhost[querydb]:36: >select 1;
+---+
| 1 |
+---+
| 1 |
+---+
1 row in set (0.00 sec)
root@localhost[querydb]:36: >show profiles;
+----------+------------+----------------------------+
| Query_ID | Duration | Query |
+----------+------------+----------------------------+
| 1 | 0.00034325 | select id,name from t |
| 2 | 0.00006925 | selectt id,name from t |
| 3 | 0.00018800 | select id,name fr0m t |
| 4 | 0.00096275 | select ida,name from t |
| 5 | 0.00117675 | select id,name from tab |
| 6 | 0.00115800 | select id,name,abc from t |
| 7 | 0.00029450 | select 1 |
+----------+------------+----------------------------+
7 rows in set, 1 warning (0.00 sec)
root@localhost[querydb]:36: >show profile for query 7;
+----------------------+----------+
| Status | Duration |
+----------------------+----------+
| starting | 0.000196 |
| checking permissions | 0.000006 |
| Opening tables | 0.000007 |
| init | 0.000016 |
| optimizing | 0.000010 |
| executing | 0.000013 |
| end | 0.000008 |
| query end | 0.000006 |
| closing tables | 0.000001 |
| freeing items | 0.000014 |
| cleaning up | 0.000019 |
+----------------------+----------+
11 rows in set, 1 warning (0.00 sec)
root@localhost[querydb]:36: >select id,name from t ;
+----+------+
| id | name |
+----+------+
| 1 | a |
| 2 | b |
| 3 | c |
+----+------+
3 rows in set (0.00 sec)
root@localhost[querydb]:38: >show profiles;
+----------+------------+----------------------------+
| Query_ID | Duration | Query |
+----------+------------+----------------------------+
| 1 | 0.00034325 | select id,name from t |
| 2 | 0.00006925 | selectt id,name from t |
| 3 | 0.00018800 | select id,name fr0m t |
| 4 | 0.00096275 | select ida,name from t |
| 5 | 0.00117675 | select id,name from tab |
| 6 | 0.00115800 | select id,name,abc from t |
| 7 | 0.00029450 | select 1 |
| 8 | 0.00074025 | select id,name from t |
+----------+------------+----------------------------+
8 rows in set, 1 warning (0.00 sec)
root@localhost[querydb]:38: >show profile for query 8;
+----------------------+----------+
| Status | Duration |
+----------------------+----------+
| starting | 0.000274 |
| checking permissions | 0.000024 |
| Opening tables | 0.000059 |
| init | 0.000036 |
| System lock | 0.000029 |
| optimizing | 0.000008 |
| statistics | 0.000031 |
| preparing | 0.000021 |
| executing | 0.000002 |
| Sending data | 0.000172 |
| end | 0.000011 |
| query end | 0.000012 |
| closing tables | 0.000013 |
| freeing items | 0.000018 |
| cleaning up | 0.000031 |
+----------------------+----------+
15 rows in set, 1 warning (0.00 sec)
root@localhost[querydb]:38: >select id,name from t where 0=1;
Empty set (0.00 sec)
root@localhost[querydb]:41: >show profiles;
+----------+------------+---------------------------------+
| Query_ID | Duration | Query |
+----------+------------+---------------------------------+
| 1 | 0.00034325 | select id,name from t |
| 2 | 0.00006925 | selectt id,name from t |
| 3 | 0.00018800 | select id,name fr0m t |
| 4 | 0.00096275 | select ida,name from t |
| 5 | 0.00117675 | select id,name from tab |
| 6 | 0.00115800 | select id,name,abc from t |
| 7 | 0.00029450 | select 1 |
| 8 | 0.00074025 | select id,name from t |
| 9 | 0.00058500 | select id,name from t where 0=1 |
+----------+------------+---------------------------------+
9 rows in set, 1 warning (0.00 sec)
root@localhost[querydb]:41: >show profile for query 9;
+----------------------+----------+
| Status | Duration |
+----------------------+----------+
| starting | 0.000279 |
| checking permissions | 0.000015 |
| Opening tables | 0.000046 |
| init | 0.000057 |
| System lock | 0.000019 |
| optimizing | 0.000025 |
| executing | 0.000014 |
| end | 0.000005 |
| query end | 0.000008 |
| closing tables | 0.000010 |
| freeing items | 0.000020 |
| cleaning up | 0.000089 |
+----------------------+----------+
12 rows in set, 1 warning (0.00 sec)
In order to facilitate observation, I am here to view the execution plan statement and then look at the statement executed.
A little ugly output below, you can see the screenshot below.
root@localhost[querydb]:10: >explain extended select id,name from t where 0=1;
+----+-------------+-------+------+---------------+------+---------+------+------+----------+------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------+---------------+------+---------+------+------+----------+------------------+
| 1 | SIMPLE | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | Impossible WHERE |
+----+-------------+-------+------+---------------+------+---------+------+------+----------+------------------+
1 row in set, 1 warning (0.00 sec)
root@localhost[querydb]:11: >show warnings;
+-------+------+------------------------------------------------------------------------------------------------------------+
| Level | Code | Message |
+-------+------+------------------------------------------------------------------------------------------------------------+
| Note | 1003 | / select#1 / select querydb.t.id AS id,querydb.t.name AS name from querydb.t where 0 |
+-------+------+------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
Found that the implementation of the three sql statement.
optimizing stages: optimization stage is algebraic
statistics and preparing stages: physical optimization phase information from the literal point of view, it is understood that the collection of statistical information, execution plan, choose the best access path for execution.
While not found Sending data, it has been described to this point a storage engine to pull data layer, the above comparison and select 1 where 0 = 1 are not Sending data, but also well understood, but here there is a problem? MySQL storage engine layer and the layer is achieved through the interface, when the result of the query returns how to know which thread to return it? In fact Internally, each query is assigned a query thread ID, according to the thread ID number.
First, the query execution
root@localhost[querydb]:41: >select id,name from t;
+----+------+
| id | name |
+----+------+
| 1 | a |
| 2 | b |
| 3 | c |
+----+------+
3 rows in set (0.00 sec)
root@localhost[querydb]:15: >show profiles;
+----------+------------+---------------------------------+
| Query_ID | Duration | Query |
+----------+------------+---------------------------------+
| 1 | 0.00034325 | select id,name from t |
| 2 | 0.00006925 | selectt id,name from t |
| 3 | 0.00018800 | select id,name fr0m t |
| 4 | 0.00096275 | select ida,name from t |
| 5 | 0.00117675 | select id,name from tab |
| 6 | 0.00115800 | select id,name,abc from t |
| 7 | 0.00029450 | select 1 |
| 8 | 0.00074025 | select id,name from t |
| 9 | 0.00058500 | select id,name from t where 0=1 |
| 10 | 0.00194675 | select id,name from t |
+----------+------------+---------------------------------+
10 rows in set, 1 warning (0.00 sec)
root@localhost[querydb]:15: >show profile for query 10;
+----------------------+----------+
| Status | Duration |
+----------------------+----------+
| starting | 0.000763 |
| checking permissions | 0.000050 |
| Opening tables | 0.000184 |
| init | 0.000178 |
| System lock | 0.000076 |
| optimizing | 0.000025 |
| statistics | 0.000095 |
| preparing | 0.000055 |
| executing | 0.000002 |
| Sending data | 0.000335 |
| end | 0.000018 |
| query end | 0.000028 |
| closing tables | 0.000023 |
| freeing items | 0.000041 |
| cleaning up | 0.000076 |
+----------------------+----------+
15 rows in set, 1 warning (0.00 sec)
executing: for the implementation phase.
There is also a System lock phase: in fact, when executed DDL, DML operation, MySQL will be carried out inside the locking of the metadata table there are other locks, such as locks S, X lock, IX lock, IS lock, with to solve or to ensure consistency between DDL and DML operation operation. [>> can see inside or transaction and implementation of technology << InnoDB storage engine]
Reference books:
Compiler theory [] [US] Alfred V.Aho, [the United States] Monica S.Lam, [the United States] Ravi Sethi waiting; Zhao Jianhua, Zheng Tao translated
[Introduction to Database Systems] Wang Shan, the Sa Shi Xuan
[Art database query optimizer: Principles and resolve SQL performance optimization with Li Haixiang]
[InnoDB storage engine technology inside the Jiangcheng Yao]
[Affairs] and implementation process