Innodb database memory model

The beginning of the story has to start with an Innodb memory model diagram: the

buffer pool

     The InnoDB storage engine is based on disk storage and manages the records in it in pages. Buffer pools are often used to improve the overall performance of the database. The buffer pool is simply a piece of memory, which compensates for the impact of slow disk speed on database performance through the speed of memory.

 

     When a read operation is performed in the database, the page read from the disk is first stored in the buffer pool. When reading the same page next time, it is first determined whether it exists in the buffer pool. If there is, it is hit and read directly. just read from disk.

 

       When the database is modified, it is first written to the pages in the buffer pool (1) , and then flushed (2) to the disk at a certain frequency .

 

        The important parameters are roughly:

            1> innodb_buffer_pool_size ## buffer size

            2> innodb_buffer_pool_instances ## The number of cache pool instances, each page is evenly allocated to different instances according to the hash value to reduce the competition for internal resources in the database and increase the concurrent processing capability of the database.

 

       (1) A page is 16KB by default, and the unzip_LRU compression memory technology is used when allocating pages. The specific practice method is: for example, when a 4KB page needs to be allocated, it will first find out whether there is a 4KB free page, if not, then find out whether there is an 8KB free page, and if not, split a 16KB page into two 8KB pages. page, and then split one of the 8KB pages into two 4KB pages. Take one of the 4KB pages for allocation.

 

      (2) This is not triggered when the page changes. Instead, it is flushed to disk through a checkpoint mechanism. So what is the checkpoint mechanism:

          It was introduced to avoid flushing the disk every time a dirty page appears. The simple arrival is that Innodb uses this mechanism to decide when to flush dirty pages. There is no need to redo all the logs after the downtime. The previous checkpoint has been persisted to the disk, and only the data after the redo is needed . The timing can be roughly summed up as:

          1> When the buffer pool is not enough, buffer dirty pages to disk

          2> When redo logs are unavailable, flush dirty pages

 

LRU List、Free List和Flush List

 

       Although the meanings of these three lists are different, the data between LRU and Flush are partially overlapped. First, summarize what the three lists are in one sentence:

       1> LRU list: The page that stores cached data.

       2> Free list: Unused pages.

       3> Flush list: Pages that store dirty data.

 

       The buffer pool of the innoDB storage engine is managed by the LRU (Latest Recent Used) algorithm.

 

       The specific implementation method is as follows: the most frequently used page is at the front of the LRU list (managing the pages that have been read), and the least used page is at the end of the LRU list.

 

       Innodb made some optimizations to the LRU algorithm and added the midpoint position. What is midpoint:

 

       New data loaded into memory from a file will be placed in the midpoint position by default, not in the head of the LRU list. The purpose is to avoid operations such as full-scale scans (which are basically only used once) to squeeze hot data out of the cache pool.

 

       当需要从缓冲池中分配页时,首先从Free列表中查看是否有空闲的空闲页,若有则从Free列表中删除然后加入到LRU列表中。否则根据LRU算法,淘汰LRU列表末尾的页,将该内存空间分配给新的页。在LRU列表中的页被修改后,该页称为脏页(dirry page),这时数据库通过checkpoint机制将脏页刷回磁盘。而Flush列表中的页就是脏页列表。

 

重做日志缓冲(redo log buffer)

 

       innoDB存储引擎首先将重做日志信息放入这个缓冲区,然后按照一定的频率将其刷入重做日志文件中。重做日志缓冲区一般不需要很多,只要保证每秒产生的事务量在这个缓冲大小之内即可。可以通过innodb_log_buffer_size参数设置大小。

 

额外的内存池

 

       在innoDB存储引擎中,对内存的管理是通过一种称为内存堆(heap)的方式进行。在对一些数据结构本身的内存进行分配时,需要从额外的内 存池中进行申请,当该区域的内存不够时,需要从缓冲池中申请。

 

Innodb关键特性

       插入缓冲:物化到磁盘。优化非唯一辅助索引的插入操作,实现方式是B+树,和内存中索引页一起确保非聚集索引的性能。

       两次写:刷新脏页时,首先将脏页数据复制到内存的2M空间中,然后顺序写入磁盘,如果刷新脏页失败就拿它恢复。

       自适应hash索引:系统判断是否开启AHI,当访问模式相同时从缓冲池中的B+树构造而来。

       异步IO(AIO):多次发出IO请求,避免了中间因为等待IO完成所消耗的时间。 

       刷新邻接页:刷新一个脏页时,会检测该页所在区的所有页,如果存在脏页将一并刷新。

 

线程及作用

       Master Thread:1> 合并插入缓冲   2> 将日志缓冲刷新到磁盘

       IO Thread(Async IO):读写业务数据,读写插入缓冲,记录日志

       Purge Thread:回收undo页,离散读取undo页,可以进一步利用磁盘的随机读写性能 

       Page Cleaner Thread:刷新脏页

 

 

Guess you like

Origin http://10.200.1.11:23101/article/api/json?id=327090322&siteId=291194637