Question: MySQL uses internal temporary tables when processing temporary result sets (UNION operations/aggregation operations, etc.). So how much memory will the internal temporary table use?
experiment:
We first create a database for testing,
Then prepare the data,
We use a child table with UNION, so that the execution plan will use the internal temporary table:
You can see that the execution plan does use a temporary table:
Let's start another session and use performance_schema to observe the memory:
In the main session, probe its connection number and find the thread number:
In performance_schema, confirm the initial state of its memory allocation statistics:
Execute SQL in the main session:
In performance_schema, check its memory allocation:
It can be seen that in the processing of this SQL, a total of more than 4M of memory is allocated for internal temporary tables:
We all know that the memory temporary table is a table in the memory (heap) engine format. Then we manually build an explicit memory table, which should use the same memory as the memory temporary table, let’s test it.
Create a memory table in the main session, and insert data into the memory table:
Observing the performance_schema, we can see that the number of bytes of the memory table residing in the memory is the same as the number of bytes used by the previous temporary table.
in conclusion
We observed the memory allocation of the memory engine through performance_schema, and calculated the memory usage of internal temporary tables.
MySQL does not display information about internal temporary tables in other metadata, such as information_schema.INNODB_TEMP_TABLE_INFO, as shown in the figure:
It is also worth noting that the memory engine will divide a lot of space. For example, in this example, our data is 300025 rows * 4 bytes = ~ 1.2M, and the engine separates more than 4M of memory for storage.
Therefore, if you make an estimate, you need to multiply the amount of data by a larger coefficient to accurately estimate.
This is the second time we have used dbdeployer, let’s introduce its life experience:
The predecessor of dbdeployer is the famous mysql-sandbox. It is the masterpiece of the famous blogger Giuseppe Maxia (http://datacharmer.blogspot.com). It is extremely convenient to build a test environment for MySQL with multiple architectures, and the commands are simple and elegant.
In future experiments, we will use dbdeployer many times, or use the MySQL container to quickly build and experiment.