SELECT COUNT (*) FROM TABLE is a common thing to another SQL needs.
The number of rows in the use of MySQL specification, we generally use the transaction engine InnoDB as (General Business) storage engine table, in this context, the time complexity COUNT (*) operation degree of O (N), where N is a table .
And MyISAM tables can quickly access to table rows. Behind these practices is how the mechanism, and why you need / may be such that the article you want to explore.
First look at the overview, MySQL COUNT (*) part of the problem in two kinds of storage engine:
Here With these questions, mainly to the InnoDB storage engine to be discussed.
A, InnoDB table full COUNT (*)
main problem:
1, the implementation process like?
2, how to calculate the count? Factors affecting the count result of what?
3, where the count value exist? Data structures involved are what?
4. Why InnoDB can only be achieved count (*) by sweeping the table? (See last issue of this article)
5, the whole table COUNT (*) as a case table scan type of operation, What are the risks?
6, COUNT (*) if the operation would be like "SELECT *" overflow pages as possible to read large field involved?
1, performs frame - cycle: read count +?
1.1 The basic conclusion:
Full table scan, a circulation problem.
The inner loop: read a line first, and then decide whether the line included in the count.
The circulation is performed line by line counting process.
1.2 Description:
Simple SELELCT-SQL implementation framework, the analogy INSERT INTO ... SELECT is the same process.
The following will gradually refine how to read and count (count ++).
2, the implementation process?
Performing part of the process, it is divided into four parts:
(1) COUNT (*) Pre-process: Prior to MySQL-Server-side implementation of SQL SELECT statement sent from Client-side, to make a stage for some of the elaborate back.
(2) COUNT (*) process: gives a brief focus on the call stack part of the code-level process framework and two core steps.
(3) reads a line: Visibility and row_search_mvcc function, describes how it affects visibility COUNT (*) results.
(4) line count: Evaluate_join_record column is empty, and describes how they affect the counting process COUNT (*) results.
If readers want to see directly how the COUNT (*), then you can ignore (1), and jump directly to (2) beginning to see.
2.1, COUNT (*) Pre-Process memories - to send SQL from Client terminal function sub_select
In order to see the calling process is not unexpected, we first recall how to perform this function sub_select to come:
(1) MySQL-Client sends SQL statements transmitted in accordance with a communication protocol packet MySQL.
(2) Mysql-Server receives a data packet, parsing the command type (the QUERY) and the SQL statement (character string) by the protocol.
(. 3) through the SQL statement parser output JOIN object class, for expression of the structured SQL statement.
PS: Here JOIN structure, not just purely syntactic structure, semantic processing but has been, roughly speaking, a summary list of tables (table_list), list (target_list) of the target column, WHERE conditions, sub-query syntax structure.
In the whole table COUNT (*) -case in, table_list = [table "t" (alias is "t")], target_list = [Target Object column (column named "COUNT (*)")], of course, there is no WHERE conditions, sub-queries and other structures.
(4) JOIN object has two important methods: JOIN :: optimize (), JOIN :: exec (), respectively, for execution of the query optimization and query statements.
join-> optimize (), optimization phase (myisam full table at a later count (*) operation will involve a little content here).
join-> exec (), the implementation phase (focus), including the full implementation of the process InnoDB table count (*) operation.
(5) join-> exec () After several calls, the call to sub_select function to perform simple SQL, including COUNT (*).
（6）END of sub_select 。
2.2, COUNT (*) Process (in sub_select function)
The upper and the code is a relatively simple process, focusing on sub_select function, which functions are Class 2 and corresponds to a portion of the two steps "implementation framework" - The count. Given to the following conclusions:
(1) reads a line: from sub_select after some function of the relative top-level call, a call to all the branches finally row_search_mvcc function, the function is a B + -tree structure from InnoDB storage engine to read one line stored in the memory in a buf (uchar *), the subsequent processing to be used.
Here it will involve acquiring row locks, MVCC and the line of visibility problems. Of course, for this type of snapshot read SELECT COUNT (*), and will involve MVCC visibility, without involving the row lock. Details can skip to "visibility and row_search_mvcc functions."
(2) line count: the code level, will be evaluated in the read row evaluate_join_record function, to see whether it should be included in the count (i.e., whether or not to count ++).
Briefly, COUNT (arg) itself as a function of MySQL operation for one line, the values inside the parentheses if the argument arg (entire column or a row) if it is not NULL, the count ++, or not to count the rows. Details can skip to "Evaluate_join_record column is empty" section.
Effect of these two phases COUNT (*) The results are as follows: (two layers of filtration)
Process SQL layer frame-related code is summarized as follows:
Q:
Code level, the first step (read line) has two branches, and why?
A:
InnoDB From the interface level, divided into "read the first line," and "read the next line," is two different processes execute, read the first line need to find a (cursor) position and do some initial work to make follow-up the process can be recursive.
As if we use a script / program to progressive scan table operation, to achieve the two will involve the following SQL:
In particular this embodiment relates to the code, the SQL engine layer to the storage layer calling relationship, reads as follows :( call stack for phase reference)
We can see that, no matter what branch of reading, eventually achieving the same function in row_search_mvcc.
LOOP is above the code to do some brief description, let's look row_search_mvcc and evaluate_join_record how to output the final count results.
2.3, and the line visibility function row_search_mvcc
Here we look at the line affect visibility COUNT (*) by a set of case and a few questions.
Q:
For "SELECT COUNT (*) FROM t " or "SELECT MIN (id) FROM t " operation, the first read operation of the read line is the smallest table t (B + tree, the leftmost leaf node within the page) of record? (Ha_index_first why also call row_search_mvcc to get the minimum key value?)
A:
Not necessarily. Even MIN (id) do not necessarily read the smallest line that is id, because there is also a problem of visibility line, in fact index_read to take statements within the current transaction is visible to the minimum index record. It also reflects the previously mentioned join_read_first and join_read_next "the same thing" to row_search_mvcc is as it should be.
Q:
For FIG last asked, if the transaction X is RU (Read-Uncommitted) isolation level, and the complete C-Insert (100) is the X-count (*) during execution (only scan to 5 or 10 which records) is completed, then the X-count (*) after the transaction C-Insert (100) is completed, after the reading process can be seen in the 100 this record it?
A:
MySQL adopt strategy "is to read what what", that X-count (*) in the back can read 100 this record.
2.4, evaluate_join_record column is empty
Q:
How a line included in the count?
A:
In both cases we will read the lines included in count:
(1) If the COUNT parameter is the function of a column, will be determined by the read row and column define whether the value of the column Nullable whether NULL; if both are yes, COUNT is not counted, the count or into the count.
e.g. SELECT COUNT(col_name) FROM t
col_name may be a primary key, unique key, non-unique keys, non-indexed fields
(2) If in with COUNT *, will determine whether this part of the entire line is NULL, if it is determined parameter is NULL, it is ignored, otherwise count ++.
Q:
In particular, for SELECT COUNT (id) FROM t, where t is the table id field is a primary key, then how?
A:
equivalent to COUNT (*) in effect. Because both the COUNT (*), or COUNT (pk_col) because there is a primary key in order to fully determine the data request is not NULL, COUNT such expressions may be used to obtain the number of table rows currently visible.
Q:
optimize the user level for InnoDB COUNT (*) operational problems
A:
The problem is a familiar problem the industry, scanning non-null unique key to obtain the number of rows in the table, but the number of bytes involved may be a lot less (the President and the table's primary key, unique key length difference between the more time) , a relatively small expense of a lot of IO.
Related refer to the following call stack:
Second, the data structure
Q:
COUNT value which is stored in the memory variable in?
A:
After parsing SQL, stored in the expression COUNT (*) in this one, ((Item_sum_count *) item_sum) -> count
Recalling the structure shown in FIG JOIN following our previous "COUNT (*) Pre-Process" section above.
I.e. SQL parser for each SQL statement to structure, in which a target JOIN (join) are expressed. Create and populate a list result_field_list column for the result, each element of the list is a result of the column (Item_result_field *) objects (pointers) in the subject.
In COUNT (*) -case, the result list contains only one column element, (Item_sum_count: public Item_result_field) the type of object (name = "COUNT (*)"), wherein the specific class member variable count is also desired.
Three, MyISAM table full COUNT (*)
Since the MyISAM engine is not commonly used in actual business, only briefly described as follows:
1, MyISAM-COUNT (*) operation is an operation complexity O (1) time.
2, each table stores a meta information MyISAM -count value in memory with each copy of a file, in memory count variable is initialized by the value of count values of the file is read.
3, SELECT COUNT (*) FROM t t will count variable value table corresponding to the memory read directly.
4, count value and the count value in the file memory to be updated by the write operation, consistency of table-level lock to be guaranteed.
5, table lock to ensure that the write serialized, the same timing for all users of the thread is locked or read, or only see to a data state.
Q:
MyISAM and InnoDB during the execution COUNT (*) where split operation start?
Common: SQL layer is present in common, i.e., after the data structure is the same SQL parsing, the count variable is present in the object as a result Item_sum_count type column; the process is returned to the client is similar - the count variable assignment via MySQL communication protocol back to the client.
Difference: count value calculation InnoDB is carried out in the execution phase of the SQL; and MyISAM table itself have a contains meta information table row_count value in memory, the optimization phase by tag storage engine to optimizer a hint in the SQL, indicating that the table storage engine used to save the exact number of rows, you can get directly, without having to re-enter the actuator.
BLOG Address : www.liangsonghua.com
Focus on micro-channel public number: preserved egg blackboard, get more exciting!
Public No. Description: sharing technology insights to work in Jingdong, as well as JAVA technology and industry best practices, most of them are pragmatic, can understand, reproducible
![]()