Hello everyone, I'm Snow Hide, as one of the "MySQL real" students of this column, this is my first punch 20 days, and also my 75th this operation.
Today I review in this column called "infrastructure: how a SQL query statement is executed? "" Log system: how to update a SQL statement is executed? "" Transaction isolation: Why I can not see you changed? "" Layman index (on) "," in layman's language index (down) "," Global and table locks: add a field to the table how there are so many hinder? "" Row locks merits and demerits: how to reduce the impact on the performance of the line lock? "Article.
Keywords summary: Server layer (connectors, query cache, analyzer, optimizer, actuators, built-in functions (date, time, math, encryption function), across storage engine features (stored procedures, triggers, views)), engine storage layer (store, retrieve, the InnoDB), important log module: redo log (belonging-layer, WAL technology, Redo log, Write Pos, Checkpoint, Crash-Safe), important log module: binlog (belonging-layer two the difference between the three log points), two-phase commit (to restore a specified seconds, the first to write binlog after redo log write, write write redo log after the first binlog), isolation and isolation level (isolation level (read uncommitted , read committed, repeatable read, serialization), the effect of the transaction isolation level (read uncommitted, read committed, repeatable read, serialized)), transaction isolation implementation, a transaction start mode (explicit start, implicitly started), a common model index (hash table, ordered array, the search tree), the InnoDB model index (primary key index / clustered index, the non-primary key index / two indexes, primary cable Query the difference between ordinary index), index maintenance (space), covering index (query performance), the most left-prefix principles (establishment of a joint and single-field index), index pushed down, lock classification, global lock (lock method, scenarios, risk, we recommend two reasons for using FTWRL way), table-level locking (table-level locking species (table lock, lock metadata, MDL write lock)), lock engine support line, two-phase locking (row lock release times), deadlocks and deadlock detection (two strategies to solve the deadlock, solve performance problems caused by hot-line update).
The study summary:
MySQL Infrastructure
Server layer
Connector
Responsible for establishing the connection with the client, access permissions, maintain and manage the connection.
Query Cache
Table does not change often only suitable for use query cache, such as system configuration table.
Analyzer
What MySQL need to identify which strings are what representatives.
Optimizer
Optimizer when a plurality of index table, determines which index is used; or related phrase in a multi-table determines the connection order of the table.
Actuator
When started, the fragments corresponding to whether the user has permission to the operation type of the operation table, an error message is returned without permission.
MySQL log system
Important log modules: redo log / redo logs
Belongs layer
redo log at InnoDB engine layer. Unique crash-safe capability.
WAL Technology
WAL stands for Write-Ahead Logging. Its principle is to write the log, write to disk.
Redo Log
When there is need to update the record, InnoDB engine will first recorded in the redo log (staging) and updates the memory, and then recorded to disk to update the system is idle.
Write Pos
The current record is not known, after finishing moves to the next position. Write cycle.
Checkpoint
Between the empty and the write pos portion is used to record a new checkpoint operation.
Crash-Safe
redo log to ensure that abnormal occurrence database records previously submitted after the restart still exists, this is known as crash-safe (Ben collapse security).
Important log modules: binlog / archive log
Belongs layer
binlog at the Server layer. No crash-safe capability.
The three-point difference between both logs
- redo log for the InnoDB engine unique; binlog implemented by the MySQL Server layer, all engines can be used;
- redo log belongs to the physical log, recording changes made to the statement; the binlog is logical log, records the original logical statements;
- redo log is a write cycle, the space may be exhausted; the binlog is additionally written, may switch to the new file after the specified amount of data, will not overwrite the original log.
Two-phase commit
Because redo log with binlog are two separate logic, not two-phase commit, it can only be the first to write redo log write binlog, or reverse operation.
Return to a specified seconds
- Find the most recent full backup, restore it to its temporary library;
- Starting from the point in time backup, binlog sequentially removed and the backup to the moment before the reproduction mistakenly deleted tables.
After the first to write redo log write binlog
Assume redo log written, but not yet finished binlog, MySQL process exception occurs. Since the redo log has finished, even if the system Ben collapse, we are still able to recover data. But the logic of the statement recorded binlog would not exist.
After-Write binlog write redo log
If binlog finished, but not yet finished when the redo log, Ben collapse after the restart, the transaction does not hold. At this point, binlog in the statement logical record is new, and redo log in physical data is old.
And implementation of the transaction isolation level
Isolation and isolation level
To solve the dirty read (dirty read), non-repeatable read (non-repeatable read), phantom read (phantom read) and so on.
Isolation Levels
SQL standard transaction isolation levels are: Uncommitted Read (read uncommitted), read-committed (read committed), Repeatable read (repeatable read) and serialization (serializable).
Uncommitted Read
Before the transaction commits its changes can be made visible to other transactions.
READ COMMITTED
After the transaction commits changes made before it is seen by other transactions.
Repeatable read
During a transaction always see the data and transaction data to see when you start the same. Before the transaction commits changes made to it are not visible to other transactions.
Serialization
For the same record, plus write locks when writing, plus read lock when reading. And other matters required to perform read and write locks before continuing conflict.
Transaction isolation level effect
Uncommitted Read
Although the transaction has not been submitted to B, but A transaction can see the results.
READ COMMITTED
B's update transaction in order to be seen after the transaction is committed A.
Repeatable read
During the execution of a transaction to see the data must be consistent before and after submission.
Serialization
Transaction B can not manipulate the data transaction A is in operation, etc. A submission after the transaction and release the lock, transaction B to continue.
Transaction Isolation implementation
Each record while recording a rollback in the update is going to be. By rollback operation, the value of the latest recorded values before a state can be obtained.
Starting method affairs
Explicit start
By begin or start transaction statements begin, ending commit or rollback exception to the end.
Implicit start
The set autocommit set to 0 speaks thread closed automatically submitted when the implementation of the select statement triggered the transaction, which will always exist until it encounters commit or rollback statement or disconnect from the database.
This is commonly referred to as a long connection. Recommended to set autocommit is set to 1, and explicitly start a transaction when the CUD. Frequent use of a transaction where you can use commit work and chain syntax, the current transaction will be submitted to its action and immediately open the next transaction, eliminating the overhead of calling begin statement brought again.
MySQL indexes
Index common model
Index can improve query efficiency.
Hash table
Key data structure, according to the find value key. By hashing function key converted into an index, and the value in the array is located at the position marked.
Hash table applies only to the equivalent query scenarios, such as Memcached or other NoSQL storage.
Ordered array
In the scope of the query or the equivalent parity can have a good performance.
An ordered array of storage is only suitable for static, such as some will not make changes to the data.
Search tree
Highest efficiency binary tree, but in fact most of the database storage is not applicable. Because the index is more than exist in the memory, but also simultaneously written to disk.
To minimize IO operations, to make inquiries as little as possible to access or not access data block, so we do not use a binary tree.
InnoDB index model
B + tree index using the model, the B + tree in the data storage.
Each index corresponds to a B + tree.
Primary key index / clustered index / Clustered Index
Leaf node is stored in a full line of data.
Non-primary key index / secondary index / Secondary Index
Leaf node stored value is gradual.
Discover the difference with ordinary primary key index index
- Primary key index : simply search the corresponding primary key field B + tree;
- Common Index : index tree needs to search the normal field to obtain their primary key value, and then by means of the primary key values corresponding to the primary key field in the index tree query again. This process is called back to the table.
- Non-primary key index queries need more scan the index tree, therefore, should try to use the primary key query.
Index Maintenance
take up space
The smaller the length of the primary key, the smaller the leaf nodes of the normal index, the smaller the space occupied by the general index.
To performance and space utilization as a starting point, then increment the primary key is a more reasonable choice.
Covering index
Improve query performance
Covering indexes can reduce the number of searches of the tree, can obviously improve query performance, so use a covering index is a common optimization approach.
The most left-prefix principles
It may be utilized B + "leftmost prefix" index tree structure to locate the record.
The establishment of joint and single-field index
The first principle is that when the adjusted order, if we can maintain an index less, that the order is likely to be given priority.
Index pushdown
During its traversal index field contained do first determination, the recording does not match the filter to reduce the number back to the table.
MySQL's global locks, table locks, row lock
Lock Category
Global Lock, table-level locks, row lock.
Global Lock
Lock method
FTWRL (Flush Tables With Read Lock), so that the whole library in read-only state, then some statements, other threads running will be blocked: data update (additions and deletions), data definition (construction of the table, modify table structure), update class transaction commits.
Scenarios
Full database logical backup.
risk
- Can not be performed during the main library locked update, suspend business operations;
- During the lock can not be performed from the main library database synchronization over binlog, leading to a master-slave delay.
The reason for using two recommended ways of FTWRL
- readonly In some systems, other logic is used, for example, determines a master database or library equipment. Therefore modify global variables affect relatively large;
- After performing FTWRL client if an exception is disconnected, library global lock automatically released and back to normal occurrence. If the client is set to take place during the readonly abnormal, the library will remain readonly state, will not return to normal state.
Table-level locking
Table-level locking species
Table lock, lock metadata (MDL, Meta Data Lock)
Table lock
lock tables ... read / write. And FTWRL Similarly, the lock can be released or locked automatically released when the client is disconnected by means of unlock tables. In addition to limiting write lock tables other threads, the threads will limit the next operation target.
Metadata lock
No need to explicitly declare, automatically lock when accessing the table, and to ensure the accuracy of reading and writing.
MDL write lock
- Read locks are not mutually exclusive, can be multi-threaded on the table CRUD;
- Read-write lock exclusive write locks to ensure the safety of the table structure changes. Multithreaded operating sequence table structure.
The engine supports row lock
InnoDB supports row lock.
Two-phase locking
Row lock release time
Wait until the end of the transaction release.
Deadlock and deadlock detection
Two strategies to solve deadlock
- Waiting for the timeout. Timeout parameter settings: innodb_lock_wait_timeout;
- Initiate deadlock detection, roll back a transaction deadlock chain, so that other transactions can continue. Innodb_deadlock_detech to set the parameters on.
Solve performance problems caused by hot-line update
- In ensuring the business will not be a deadlock situation temporarily turn off deadlock detection, turn off deadlock detection may occur a lot of time out;
- ? Control the degree of concurrency. Update Peer queued before entering the engine, in order to reduce the deadlock detection work inside the InnoDB;
- To reduce conflicts into a row of multiple rows of logic, in order to reduce the number of wait lock.
Finally
Re summed up the contents mentioned in the text: MySQL logical framework, SQL statements, complete the implementation process, redo log physical log, binlog logical log, parameter redo log to ensure that crash-safe feature requires configuration, logging system two-phase commit, cross-system maintain data consistency logic solutions, and implement transaction isolation level phenomenon, there is the risk of long transaction, to avoid long transaction by the right way, the concept of database indexes, covering indexes, prefix index, the index pushdown, access and reduce resource consumption, global lock , table lock, lock global scenario, a table lock scenario, the MDL metadata locks, row locks, two-phase locking protocol, deadlock detection and deadlock, transaction control of the same number of concurrent access to resources.
