1. InnoDB Storage Engine Architecture
www.weixiu3721.com
innoDB main storage engine architecture as shown in FIG.
First, the worker thread: Default seven background thread, namely four io thread (insert buffer, log, read, write), 1 Ge master thread (highest priority), a lock (lock) monitoring thread, an error monitoring thread. Can be viewed by show engine innodb status. The new version of the default read thread and the write thread respectively increased to four, can be viewed by show variables like 'innodb_io_thread%'.
With a large block of memory pool for each thread operations: the memory cell is divided into a plurality of regions, respectively, the buffer pool (buffer pool), pool redo log (redo log buffer) and additional memory pool (Additional memory pool). The memory pool is also responsible for the maintenance thread needs to access data structures, data cache disk file, fast and easy to read, while changes to the disk before the data file where the cache, and redo log caching. Specific configuration may show variables like 'innodb_buffer_pool_size', show variables like 'innodb_log_buffer_size', show variables like 'innodb_additional_mem_pool_size' View to view the three instructions.
The third part is the most basic InnoDB storage engine corresponding database table disk files, log files.
Worker thread 2. InnoDB storage engine
2.1 Master Thread
the thread is the core InnoDB background thread, most of the work done here, is responsible for the buffer pool asynchronous data flushed to disk to ensure data consistency, including dirty pages refresh combined insert buffer, recycling undo page.
Master Thread thread has the highest priority, its internal multiple cycles, including the main loop (Loop), the background loop (background loop), the refresh cycle (flush loop) and the cycle pause (suspend loop). The main thread switches in four cycles based on a database running.
1. Main Loop Loop
The main loop contains most of the operations, which is divided into two parts, and operations are operations performed every 10 seconds per second. Pseudo code
It can be seen in the circulation loop, for a first cycle that a total of 10 times, wherein the method implemented by the thread and sleep operation is performed once every 10 seconds, this way must be called every second imprecise certainly there will be delays, that is probably the most maintained at this frequency.
for operation (i.e., once per second operation) within a loop comprising:
(1) log buffer is flushed to disk (even if the transaction did not submit, will happen)
(2) combining the insert buffer (necessarily)
(3) up to 100 refresh dirty pages InnoDB buffer pool to disk (possible)
(4) determining whether the current user activity, if not then switch to the background loop.
for the outer loop (operation performed once every 10 seconds) comprising:
(1) Refresh 100 pages of dirty data to a disk file (possible)
(2) inserting a buffer pool up to 5 (constant)
(3) the log buffer is flushed to disk (certain)
(4) undo delete unwanted pages (certain)
(5) Refresh 100 or 10 dirty pages to disk file (certain)
2. Background cycle (background)
if the user is not currently active, that is, no client is connected or user login, it will switch to this cycle, mainly performs the following operations:
(1) undo delete unwanted pages (certain)
(2) inserting merge buffer 20 (constant)
(3) jumps back to the main loop (constant)
(4) continued to rack up 100 pages know qualifying (possible jump to flush loop is completed)
If the flush loop is also nothing to do in, InnoDB will switch to suspend loop, the main thread hangs waiting time of occurrence, if the user is enabled InnoDB engine, but not based on any database tables InnoDB, then the main thread will always be in a suspended state.
2.2 IO Thread
In MySQL, InnoDB uses a lot of AIO to handle write IO requests, which can greatly improve database performance, and IO Thread task is responsible for processing the callback request these IO (AIO based on asynchronous callback to add non obstruction), there are four types of IO Thread, respectively, write, read, insert buffer and log. And may be provided by innodb_read_io_threads innodb_write_io_threads parameters.
2.3 Purge Thread
affairs after being submitted, undolog they use may no longer need, and therefore need to purge Thread to recover has been used and allocated undo page, the thread can only exist one, before innoDB1.1 version of the original thread the task is on the main thread of execution, but from version 1.1, purge operation independent threads of execution to complete.
3. InnoDB engine memory structures
3.1 buffer pool buffer pool
buffer pool, in fact, is an area of memory, because the disk read and write performance compared to the processing speed of the CPU is quite different, it will be to improve the overall database performance through memory (buffer pool), database when data is read from the page, it will first read the data into the buffer pool, and then read the next page, will be available directly from the buffer pool, if this pool is not part of the data , then go to the disk to read the data file, the cache to the buffer pool; modify operation for page database in the first modified page buffer pool, then a certain frequency synchronization update the data to disk.
Pool of the cache data includes: index page, data pages, pages Use the undo, insert buffer, adaptive hash index, InnoDB lock information stored in the data dictionary information. Wherein the index and data pages to occupy a larger portion.
Pool allows multiple instances, each page is assigned to a different average buffer pool instances based on a hash value, the benefits of doing so is to reduce competition for resources inside the database, increasing the amount of database concurrency. Can view and configure innodb_buffer_pool_instances, the default value is 1, can also query the status of each pool by innodb_buffer_pool_stats table.
LRU List、Free List和Flush List
Buffer pool is a large area of memory, which store various types of data pages, however, the memory area is limited, generally speaking, we can not put all the files of all disk data cached in memory, but by each kinds of policies to manage data in memory. Database buffer pool by LRU (least recently used) algorithm to manage, in innoDB engine, the size of the buffer pool page defaults to 16kb, also uses an LRU algorithm to manage the buffer pool, but the LRU algorithm made some changes optimization.
In innoDB storage engine, LRU list also joined the midpoint position, a new page was read, although the latest data is read, but not on the forefront of LRU queue, but will be placed in the position of the midpoint, the default configuration, Midpoint queue length is set at 5/8, but can be controlled by innodb_old_blocks_pct. After the midpoint queue element is the old queue before the queue is new.
Has been adopted midpoint, rather than directly on the queue header, because some SQL operations can cause hot data buffer pool squeezed out common operations such as index or scan operation data, such operations will visit many pages in the table, even all the pages, and these pages and just this operation requires access operation, and can not be regarded as hot data, but after all, still have to scan into the buffer, if placed in the buffer queue head, then it may be a lot of really hot data will go out, leading to the hot data read in the next time and needs to be read from disk again, seriously affect performance.
Therefore, the two parameters used innoDB to solve the above problem, Midpoint addition, there is another innodb_old_blocks_time, for indicating to the mid position after the page read how long before the data is added to the hot end of the LRU queue. Thus, when performing the above mentioned SQL operations, the following method may make the hot data LRU list is not possible to brush out.
First, the data newly read is sure to be placed in mid position, then if the data in the mid position (ie old part) the number of surviving more than innodb_old_blocks_time times, then the new data will be placed in queue end (this operation is pages made young), and if the parameter results as set innodb_old_blocks_time page data is not moved from the old to the new queue, this operation is referred pages not made young, so as to ensure front mid position as hot data keep their survival, hot data will not be in position behind the mid quickly be eliminated.
You can be viewed by show engine innodb status command memory usage (certainly not the real-time display of the status of the command, but the state in the past a certain time frame, such as the status of the first 60 seconds):
Which, Total large memory represents the maximum amount of memory allocated, Buffer pool size to represent the data how many pages of the current total size is 512 * 16kb, and Free buffers represents the number of Free queue page, Database pages indicate LRU queue page number. There may be Free buffers + Database pages Buffer pool size is not equal, since Buffer pool size may also have adaptive hash index, Lock information and other data, which do not maintain LRU algorithm, it does not exist in the LRU queue.
Pages made young 0, not young 0: recording the number of times the page is moved to the new end of the LRU list.
0.00 youngs / s, 0.00 non-youngs / s: these represent two types of operations that occur per second.
There is another very important observation parameter buffer pool hit rate, the parameter indicates the cache hit rate, if the hit rate is 100%, it indicates that the buffer pool is very good, if less than 95%, then it must be checked whether the table due to total LRU list scanning due to contamination problems.
About unzip_LRU
In the image above data, we can see the penultimate line of the two parameters LRU len: 256, unzip_LRU len: 0, these two parameters are represented by the total number of pages LRU management, and in innoDB engine from version 1.0 after compression support page, the page will be the original 16kb of compressed 1kb, 2kb, 4kb and 8kb, but this part of the page after compression is managed by unzip_LRU, but the number of LRU len is the number of included unzip_LRU.
About Flush List
When the data in a LRU page is modified, then the page is called dirty pages, i.e., data buffer pool is inconsistent with the disk page on the page, which is part of a dirty page is copied to Flush List, the database will this time by checkpoint mechanism will refresh the dirty data pages to disk, that LRU List and Flush LIst have in a dirty pages, LRU in the page buffer to ensure the availability of, and the Flush List It is updated with the data in terms of dirty pages to disk, both independently of each other. That is, the number of dirty pages in db pages Modified data on the map.
Defines three page (page) in the innodb:
1) free page: This page has not been used (usually when the database has just started), this type of page is located free List, once a page is read data or operation, then the page will be from free List removed, moved to the LRU List
2) clean page: this page is used, the corresponding data file in a page, but the page is not modified (made only reading operation), this type of page located lru List
3) dirty page: this page is used, corresponding to a page in the data file, but the data page is modified, this type lru List page located in and flush List
3.2 redo log buffer (redo log buffer)
redo log buffer, or log buffer, innoDB engine first redo log information will be stored into this buffer, and then will be flushed to the redo log file in a certain frequency region, heavy log buffer generally do not need to set up a large, usually because log buffer data is refreshed once per second to a log file, it is only necessary to ensure that the amount of transactions per second can be generated in the size of this buffer. This value is controlled by the parameter innodb_log_buffer_size, the default is 8MB.