The architecture of the InnoDB storage engine is very complex and is specifically designed for high concurrency and complex transactional activities. It has many advanced features that should be prioritized before improving performance. We mainly focus on those features that can be monitored and improved, including indexes, buffer pools, log files, and tablespaces.
InnoDB tables use clustered indexes. Even if no index is specified, InnoDB assigns each row an internal value for use with a clustered index. A clustered index is a data structure that stores not only the index, but the data itself. That is, once a value in an index is located, the data can be retrieved directly without additional disk seeks. Of course, the primary key index or the first unique index of the table is created using a clustered index.
If a secondary index is created, the key (primary key, unique key or row ID) information of the clustered index will be stored in the secondary index. This allows for quick relocation and recall of the original data in the clustered index. This also means that if the secondary index is scanned using the primary key column, only the secondary index is needed to retrieve data.
The buffer pool is a caching mechanism used to manage transactions and read and write data to and from disk, and if configured properly, will increase the rate of disk access. The buffer pool is also an important part of crash recovery, because the information in the buffer pool will be written to disk periodically (for example, at shutdown). The buffer pool is a memory component whose validity must be monitored to ensure correct configuration.
InnoDB also uses a buffer pool to store data changes and transactions. InnoDB caches data changes by saving them to data pages (blocks) in the buffer pool. Every time a data page is referenced, it is placed in the buffer pool and marked as "dirty" when changed. This change is then written to disk to update the data, and a copy is written to the log. These log files are named ib_logfile or ib_logfile1. These files can be seen in the data directory of the MySQL server.
The InnoDB storage engine uses two disk-based mechanisms to store data: log file tablespaces. InnoDB also uses these logs to rebuild (or redo) data changes before shutting down or dying. At program startup, InnoDB reads the log and automatically writes dirty data to disk so that data changes in the buffer pool can be recovered before the system crashes.
A tablespace is an organizational tool used in InnoDB as a machine-independent file, including data, indexes, and a rollback mechanism (rollback transactions). By default, all tables share a tablespace. Tables can also be stored in their own tablespace. These tablespaces contain both data and indexes for the table. Tablespaces can automatically expand into multiple files, allowing you to store more data in the table than the operating system can handle. You can also divide the tablespace into multiple files and store them on different disks.
Use innodb_file_per_table to create separate tablespaces for each table. Tables created before this option is set will be stored in the shared tablespace. Using this command only affects newly created tables.
Monitor log files
InnoDB log files buffer data between your data and the operating system, and proper functioning of these files ensures good system performance. You can also monitor these log files directly by viewing the following system status variables.
mysql> SHOW STATUS LIKE 'innodb%log%';
+------------------------------+-------+
| Variable_name | Value |
+------------------------------+-------+
| Innodb_log_waits | 0 |
| Innodb_log_write_requests | 0 |
| Innodb_log_writes | 1 |
| Innodb_os_log_fsyncs | 3 |
| Innodb_os_log_pending_fsyncs | 0 |
| Innodb_os_log_pending_writes | 0 |
| Innodb_os_log_written | 512 |
| Innodb_available_undo_logs | 128 |
+------------------------------+-------+
Innodb_log_waits
Counter for the wait time the operation must wait for the log to flush when the log file is too small (that is, there is not enough space). If the value starts to increase and is longer than zero (except for bulk operations), you can increase the size of the log file.
Innodb_log_write_requests
The number of log write requests.
Innodb_log_writes
The number of times data was written to the log.
Innodb_os_log_fsyncs
The number of operating system file synchronizations (ie, fsync() method calls).
Innodb_os_log_pending_fsyncs
The number of file sync requests that are pending. If the value starts to increase and is greater than zero for a long time, you may need to check for disk access problems.
Innodb_os_log_pending_writes
The number of log write requests blocked (pend). If the value starts to increase and is greater than zero for a long time, you may need to check for disk access problems.
Innodb_os_log_written
The total amount of bytes written to the log.
All of these options display numerical information, and custom icons that describe this information can be created in MySQL Manager.
Monitor the buffer pool
The buffer pool is where the InnoDB cache frequently accesses data, and any updates to data in the buffer pool are also cached. The buffer pool also stores information about the current transaction. Therefore, the buffer pool is a critical mechanism for performance.
Use the SHOW ENGINE INNODB STATUS command to view information about the buffer pool, as shown in Example 9-5. For the convenience of viewing, we re-explain the knowledge of buffer pools and memory here.
----------------------
BUFFER POOL AND MEMORY
----------------------
Total memory allocated 137363456; in additional pool allocated 0
Dictionary memory allocated 94550
Buffer pool size 8192
Free buffers 7766
Database pages 426
Old database pages 0
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 426, created 0, written 1
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 426, unzip_LRU len: 0
I / O sum [0]: cur [0], unzip sum [0]: cur [0]
The information that needs to be noted in this report is given below. We'll discuss specific state variables in more detail later.
Free buffers
The number of buffer segments that are empty and can be used to buffer data.
Modified db pages
Changed (dirty) pages
Pending reads
The number of pending read requests, this value should be kept low
Pending writes
The number of pending read requests, this value should be kept low
Hit rate
The ratio between the number of requests that successfully access the buffer and the total number of requests should preferably be close to 1:1.
You can also view more information on state variables. The status variables for the InnoDB buffer pool are shown below:
mysql> SHOW STATUS LIKE 'innodb%buf%';
+---------------------------------------+-------------+
| Variable_name | Value |
+---------------------------------------+-------------+
| Innodb_buffer_pool_dump_status | not started |
| Innodb_buffer_pool_load_status | not started |
| Innodb_buffer_pool_pages_data | 426 |
| Innodb_buffer_pool_bytes_data | 6979584 |
| Innodb_buffer_pool_pages_dirty | 0 |
| Innodb_buffer_pool_bytes_dirty | 0 |
| Innodb_buffer_pool_pages_flushed | 1 |
| Innodb_buffer_pool_pages_free | 7766 |
| Innodb_buffer_pool_pages_misc | 0 |
| Innodb_buffer_pool_pages_total | 8192 |
| Innodb_buffer_pool_read_ahead_rnd | 0 |
| Innodb_buffer_pool_read_ahead | 0 |
| Innodb_buffer_pool_read_ahead_evicted | 0 |
| Innodb_buffer_pool_read_requests | 4168 |
| Innodb_buffer_pool_reads | 427 |
| Innodb_buffer_pool_wait_free | 0 |
| Innodb_buffer_pool_write_requests | 1 |
+---------------------------------------+-------------+
The buffer pool has many status variables that display key statistics about the performance of the buffer pool, such as the buffer pool page status, buffer pool read and write information, and the frequency of read and write waits in the buffer pool. The individual state variables are described in detail below:
Innodb_buffer_pool_pages_data
Number of pages containing data, including unchanged and changed pages (i.e. dirty pages)
Innodb_buffer_pool_pages_dirty
The number of changed pages (i.e. dirty pages)
Innodb_buffer_pool_pages_flushed
The number of times the buffer pool page was flushed
Innodb_buffer_pool_pages_free
number of empty pages
Innodb_buffer_pool_pages_misc
The number of pages the InnoDB engine uses to manage work, calculated as follows:
X = Innodb_buffer_pool_pages_total - Innodb_buffer_pool_pages_free - Innodb_buffer_pool_pages_data
Innodb_buffer_pool_pages_total
The total number of pages in the buffer pool
Innodb_buffer_pool_read_ahead_rnd
The number of random read heads that occur when InnoDB scans large blocks of data
Innodb_buffer_pool_read_ahead_seq
The number of sequential read headers that occurred during a sequential full table scan
Innodb_buffer_pool_read_requests
Number of logical read requests
Innodb_buffer_pool_reads
Number of logical reads directly from disk (rather than from the buffer pool)
Innodb_buffer_pool_wait_free
If the buffer pool is busy and has no empty pages, InnoDB may need to wait for a page flush. The value represents the number of times to wait. If this value increases and is always greater than 0, the buffer pool may be too small or there is a problem with disk access.
Innodb_buffer_pool_write_requests
The number of writes to the InnoDB buffer pool.
All of these options display numerical information, and custom charts that describe this information can be built in MySQL Manager.
monitor tablespace
If InnoDB can expand tablespaces when running slowly, then InnoDB's tablespaces can be largely self-sufficient. configure
--innodb_data_file_path=indata1:10M:autoextend
See the "InnoDB Configuration" chapter of the MySQL Online Reference Manual for details.
Use the SHOW ENGINE INNODB STATUS command to view the current tablespace configuration information. You can also view tablespace details by opening the InnoDB Tablespace Monitor.
Using the information_schema table
If your installed version of MySQL includes the InnoDB storage engine plugin. Seven special tables are also accessible in the information_schema database.
Technically, these tables are not really tables. Its data is not stored on disk, but is generated when the table is queried. These tables provide another way to monitor InnoDB and provide administrators with performance information. These tables are used to monitor compression, transactions, and locks, which we'll briefly cover in turn.
INNODB_CMP
Displays compressed table details and statistics.
INNODB_CMP_RESET
Displays the same information as INNODB_CMP, but with one feature: statistics are reset when the table is queried, which allows you to track statistics on a regular basis (eg hourly, daily, etc.).
INNODB_CMPMEM
Displays details and statistics about compression used in the buffer pool.
INNODB_CMPMEM_RESET
Displays the same information as INNODB_CMPMEM, but with the feature that statistics are reset when the table is queried, allowing you to track statistics on a regular basis (eg hourly, daily, etc.).
INNODB_TRX
Displays details and statistics for all transactions, including transaction status and currently running query information.
INNODB_LOCKS
Displays details and statistics of locks requested by a transaction. Describes the status, mode, type, etc. of each lock.
INNODB_LOCK_WAITS
Displays details and statistics about locks requested by blocking transactions, describing the status, mode, type, and blocking transactions of each lock.
Compression information for a table can be monitored using the compression table, which includes details such as page size, which pages are used, compression and decompression times, and more. This information is important to monitor if you are using compression and want the overhead of compression to not impact database server performance.
Transactions and locking tables can be used to monitor transactions. This is a very useful tool to keep transactional databases running smoothly. More importantly, it can precisely determine the status of individual transactions, and which transactions are blocked and which are locked. This information is also important for diagnosing complex transaction problems such as deadlocks or poor performance.
Other parameters to consider
There are many other parameters for monitoring and tuning the InnoDB storage engine. We've only discussed some of them earlier, focusing on monitoring the subsystem and improving performance. Of course, there are some other parameters that may need to be considered.
In some cases, thread performance can be improved by tuning the innodb_thread_concurrency option. The default value of this parameter is 0, in general, this value is sufficient. However, if you are running MySQL on a server with multiple processors and multiple independent disks (and use InnoDB frequently), then setting this value to the sum of the number of processors plus the number of independent disks will improve system performance and ensure that enough InnoDB usage is used. Threads maximize concurrent operations. If the value is larger than what the system can support, it will have little or no effect - such a setting will not reach its maximum value if there are no threads available.
If the system is shut down frequently or even periodically (for example, running MySQL on Linux system startup), it may take a long time to shut down InnoDB. Fortunately, InnoDB can be shut down quickly by setting the innodb_fast_shutdown option. This does not affect data integrity, nor does it cause memory (buffer) management losses. A fast shutdown of InnoDB simply skips the potentially expensive operations of clearing the internal cache and merging the insert buffer, still performing a controlled shutdown, and storing the buffer pool on disk.
Early releases of MySQL had concurrency control and locking issues. For these earlier versions, you can control how InnoDB handles deadlocks by setting the innodb_wait_timeout variable. This variable has global and session scope and controls how long InnoDB allows a transaction to wait for a row lock before terminating. The default value is 50 seconds. If there are many lock wait timeouts, you can increase the value of this variable to alleviate some concurrency problems.
If you're importing large amounts of data, make sure to sort the incoming data files by primary key, which can improve load times. also. Setting AUTOCOMMIT to 0 turns off autocommit and ensures that the entire load is committed only once. Bulk loading can also be improved by turning off foreign keys and unique constraints.