MySQL Series Articles
MySQL (1) Basic structure, SQL statement operation, try
MySQL (2) Index principle and optimization
MySQL (3) SQL optimization, Buffer pool, Change buffer
MySQL (4) Transaction principle and analysis
MySQL (5) Caching strategy
MySQL (6) Three paradigms of master-slave replication database
SQL optimization
MySQL can use B+ trees to reduce the number of disk IOs during index search, but if disk IO is required for each search and new addition, bottlenecks will still be encountered if frequent operations are performed. Therefore, there are Buffer pool and Change buffer.
Buffer pool
Purpose: The buffer pool is to reduce the number of disk IO reads and writes.
If there is no buffer pool, each query will read from the disk for IO operations.
Therefore, a large area in the memory will be specially selected as the Buffer pool to store some pages that have been read and the surrounding page data (spatial locality). In this way, the next time the data is queried, it will be queried in the Buffer pool to see if it exists . Required pages, reducing the number of read and write IOs.
When modifying data, the data in the buffer pool will also be modified first. The modified page is called a dirty page. Generally, the redo log persistence mechanism is used to uniformly write the dirty page to the disk.
buffer pool structure
The buffer pool involves three linked lists:
free list organizes unused cache pages in the buffer pool;
flush list organizes dirty pages in the buffer pool, that is, pages to be flushed;
lru list organizes hot and cold data in the buffer pool, when the buffer pool If there is no free page, the longest unused data in the lru list will be eliminated;
Buffer pool&LRU algorithm
The buffer pool optimizes the LRU method. Because MySQL has a read-ahead mechanism, each time a cache page is loaded into the Buffer pool, data pages near the target cache page are also added. It is the principle of spatial locality of disk reads.
This will cause a problem, the data pages that may be pre-read and added may not be accessed for a long time afterwards.
Optimization 1. Hot and cold data
5/8 of the area in the buffer pool is the new sublist, and 3/8 of the area is the old sublist.
- Therefore, when data is added to the pool buffer pool, it will first enter the old sublist, and when the page is accessed , it will enter the new sublist. When the free list
is insufficient, those old sublist areas will be first flushed into the disk, and then the pool area will be cleared.- Therefore, if the data pages added by pre-reading are not accessed, they will stay in the old area forever, and will be emptied in advance when there is insufficient space. This can solve the problem of read-ahead failure.
Optimization 2. Time threshold
When a page is accessed and stays in the old sublist for more than the configured threshold, it enters the new sublist to solve the problem of batch data access and elimination of a large amount of hot data. Usually the threshold is set to 1s.
Why 1s?
Because the data pages loaded by the read-ahead mechanism are usually accessed within 1s, and these loaded data pages are usually accessed within 1s, and may not be accessed again after 1s.
Therefore, it is not good to add these accessed cache pages to the new area at this time, so the time threshold is configured. After 1 second, the accessed data page enters the new area and puts it into the head of the new area, indicating that it may be accessed later. The data pages accessed before 1s remain unchanged.
Detailed content reference: https://www.jianshu.com/p/7cb6d7d59064
Change buffer
If a page data that needs to be modified is not in the buffer pool, what should we do:
- Load the data page into the buffer pool, a random disk IO
- Update the data in the buffer pool, a memory IO
- Write the modification to the redo log, and write IO to the disk sequentially
It seems that the operation is ok, but in the scenario of writing more and reading less, we have a better method, that is, the change buffer of innodb.
Change buffer working principle
Change buffer caches the data changes of the non-unique index (DML operation, only records the operation, not the result). When accessing this data page or regularly arriving at the data in the Change buffer, it will be asynchronously merged to the disk;
Therefore, for the page data that needs to be modified just now, which is not in the buffer pool, use the change buffer operation:
- Write the modified operation into the change buffer, a memory IO
- Write the log to the redolog, wait for the merge to be triggered, and write IO to the disk sequentially
It can be seen that using the change buffer will save the most time-consuming disk IO reads and writes in the scenario of more writes and less reads.
Scenarios where change buffer is not applicable
- Unique index scenario: A unique index needs to determine whether there is a conflict, that is, whether it is unique. This judgment requires a global scan and reads data from the disk, so the change buffer is useless.
- Scenarios with less writing and more reading: Because reading the page after modifying the data page will trigger a merge, and our purpose is to update multiple data page modifications to the disk through a merge. If there are many scenarios for reading data, then The number of merges is large, and the number of IO operations will not be reduced.
Summarize
When the data is not in the buffer pool, modify the page data and then read the data, understand how the buffer pool and change buffer work:
modification operation:
the page data cannot be matched in the buffer poo;
record the modification of the data in the change buffer Operation;
query operation:
the page data cannot be matched in the buffer pool;
the merge operation is read in the change buffer and put back into the buffer pool;
the page data is read from the disk;