Novel: I am a DQL

The SQL execution flow chart is as follows.

This article is adapted from "High Performance Mysql", and Yan Ge uses the form of a novel to talk about this content.

The prologue introduces
myself. I am a SQL, just a long string. Don't ask me what I look like, because I am more arrogant.

Um~~ It’s not that I won’t say it, because in detail, I can subdivide into DML (Update, Insert, Delete), DDL (table structure modification), DCL (permission operation), DQL (Select) operation, one by one Introduce, I'm afraid everyone will dislike me!
Well, everyone doesn't have any comments. I will continue to introduce myself~
Due to the many types, here I am just a query SQL, which is a DQL.
The client sends me to the server according to the Mysql communication protocol.
When I reach the server, I will execute it in a separate thread. The server must first...

I never expected that I was interrupted again~ well, because I am executing in a thread, there must be a way to see the execution status of the thread. Mysql provides the following commands for everyone to check

SHOW [FULL] PROCESSLIST

The result is the

Command column in the figure below , which reflects the current execution status of this thread. During the execution of this thread, the state will change many times.
In the picture, there is a Sleep, which tells you that the thread is waiting for the client to send a new request. There is also a Query, which represents that the thread is executing a query or sending the results to the client.
As for the others, there are Locked, Sending data, etc., which respectively represent...

Uh, okay, a lot of nagging, the big house is not annoying to me, um, as for the meaning of other states, you can go Query on Mysql official website.
Well, back to the topic just now. After I reach the server, Mysql has to judge whether my first 6 characters are select!
Moreover, the statement does not contain the SQL_NO_CACHE keyword, and if it meets the conditions, it enters the query cache.

In the first chapter, I
talked about the query cache, which is actually a hash table. The executed statements and their results will be directly cached in memory in the form of key-value pairs.
Its key is a hash value, which is generated by querying SQL (that is, me), the current database to be queried, client protocol version, etc., and its value is naturally the query result.

Of course, if I want to bypass the query cache, it is also very simple. I can write like this:

Select SQL_NO_CACHE * from table

You can also set the parameter query_cache_type to DEMAND to bypass the query cache.

However, one day the query cache sadly said to me: "You will never see me again in the future. I have been eliminated by history. Mysql 8.0 version will no longer have me!" After
hearing this news, I appeared on the surface Pretending to be strong and saying to the query cache: "Don't be square, everyone will miss you!"
However, what I actually thought was: "Hey hey, you cheat, finally no longer exists!" Don't think I am too evil , After all, query caching is really not easy to use. Next, let's talk about the parser... I

never thought that I wanted to fool it. The result... well, back to the topic, because

• As long as there is an update to a table, all query caches on this table will be emptied
• Any difference in SQL characters, such as spaces and comments, will cause cache misses

Therefore, I can think of querying the cached table in only one case, and that is the configuration table. Other business tables simply cannot take advantage of the features of query caching. Perhaps the Mysql team also felt that the use of query caching was too limited, so they ruthlessly removed it.

The second chapter of the love and hatred between me and the analyzer
(this article refers to the parser and the preprocessor as the analyzer)
said that after I leave the query cache, I enter the parser.
Parser: "Come on, let me analyze you lexically and tell me what you look like?"
I am like this

select username from userinfo

Parser: "Okay, okay, okay. I have two stages. I will perform lexical analysis on you first. I will input you character by character from left to right, and then recognize words according to the word formation rules. You will Generate 4 Tokens as shown below."

Keyword non-keyword keyword non-keyword
select username from userinfo
parser: "Next, perform grammatical analysis to determine whether the SQL statement you entered meets the MySQL grammar. Then generate the following syntax tree."

Me: "What if the grammar is wrong?"
Parser: "Then you will receive a prompt as follows!"

You have an error in your SQL syntax

Parser: "After successfully generating the grammar tree, I will send you to the preprocessor!"
Preprocessor: "Brother, you can pull it!"
Me: "Yeah!"
Preprocessor: "Brother, I'll help you See if your column name is correct, and if this column is actually in this table in the database. Check if the table name is correct, if not, you will see the following error!"

Unknown column xxx in ‘where clause’

Preprocessor: "Finally, I will send you to do permission verification. If you do not have the permission to operate this table, the following error will be reported!"

ERROR 1142 (42000): SELECT command denied to user 'root'@'localhost' for table 'xxx'

(This place, you may have questions, because some articles say that the authority verification is done by the executor, you can directly pull it to the bottom of this article to see the explanation)

Finally, this syntax tree will be passed to the optimizer.

Chapter 3 The moving past with the optimizer
After bidding farewell to the parser, I entered the optimizer.
The optimizer brother: "Tell me, what do you look like?"
I said: "Don't worry, I look like this~" (The optimization here should actually be the syntax tree. I only use SQL for the sake of explanation. As an example, it is actually optimized for the syntax tree)

select t1.*
from Table1 t1
inner join Table2 t2
on t1.CommonID = t2.CommonID

Optimizer brother: "My task is to help you determine how to perform faster, such as first checking Table1 and then Table2, or checking Table2 and then Table1? After judging how to execute, just generate an execution plan!"
I distrustfully said: "Huh, don't you make a mistake in judgment!" The
optimizer brother: "Then you have to rewrite the SQL. For example, if you bring the STRAIGHT_JOIN keyword, it looks like this."

select t1.*
from Table1 t1
STRAIGHT_JOIN  Table2 t2
on t1.CommonID = t2.CommonID

"Then I know that it is mandatory to find Table1 and then find Table2 in association. There are many similar examples, so I won't list them one by one!"
(STRAIGHT_JOIN function is similar to join, but the table on the left can drive the table on the right. Can change the execution order of table optimizer for join table query.)

I said: "Wow, how to write an efficient SQL is really a science!"
So, the optimizer brother turned me into an execution plan and gave it to the executor~

Chapter 4 My tragic experience with the executor
I: "Brother executor, what do you use for it?" The
executor:" executes the query according to the execution plan. I will call the bottom layer one by one according to your instructions. Storage engine, step by step."
MySQL defines a series of abstract storage engine APIs to support plug-in storage engine architecture. Mysql implements an abstract interface layer called handler(sql/handler.h), which defines interface functions, such as ha_open, ha_index_end, ha_create, etc. The storage engine needs to implement these interfaces to be used by the system.

Some emotions in the
last chapter In the last stage, Mysql will return the query results to the client.
The only thing that needs to be explained is that if it is a SELECT type of SQL, Mysql will cache the query results. As for the other SQL, it will be
the table related to the query cache emptied.

Some questions
, you may have some questions about at what stage the permission verification is performed.
There was an article by a big cow saying that permission verification is in the execution stage, and the permission is verified before execution. If you have read his article, you may have questions. I am not questioning people randomly, after all, I am just a small coffee. I'm just here to express my own argument, and everyone is welcome to comment on it.

Argument 1: Authorization verification in the executor does not make sense logically.
A query SQL goes through the query cache, analyzer, optimizer, and executor. If you find that the permissions are insufficient in the last stage of the executor, then the previous series of processes are not done for nothing, Mysql should not be so stupid~

Argument 2: It does not match the content of the
book "High Performance Mysql" . There is a sentence on page 209 of the book, as shown in the figure below.

The book also specifies that permission verification is performed in the preprocessor. In this article, preprocessing and parser are unified into the category of parser.

Argument 3: The source code does not match.
I looked at the source code of Mysql5.7.25. The core code for processing the query is as follows.
In the sql_parse.cc file, there is such a piece of code as follows

case SQLCOM_SELECT:
 {
    //省略
    res= select_precheck(thd, lex, all_tables, first_table);
    if (!res)
      res= execute_sqlcom_select(thd, all_tables);
    //省略
  }

Among them, select_precheck is for permission verification. The optimizer and executor are in the execute_sqlcom_select method.
Of course, everyone has new insights, please leave a message.