Why should there be a lock?
We all know that locks in the database design is to solve concurrency issues when multiple users simultaneous access to shared resources. When access to shared resources, meaning the locked user access rules. The locked range, MySQL the lock may be roughly divided into global lock table level three and row locks. In this article, we will in turn introduce three types of locks. After reading this article, you should have the following contents:
- Why use the global lock when the backup?
- Why it is recommended to use InnoDB engine as a backup?
- The method of setting a global read-only
- Two types of table-level locking
- MDL cause problems hang database
- How to use two-phase locking protocol to reduce lock conflicts
- How to resolve a deadlock
- For hot table, how to avoid detection of loss shown?
Global Lock
What is the global lock?
Global lock will make the entire library is read-only, other threads statements (DML, DDL, update transaction class) statement would have been blocked.
Use the global lock scene
When doing a full database logical backup, will select the entire library and then save it as text.
Why use the global lock?
Imagine a scenario where you want to buy a backup system, in which the purchase operation designed to update account balances and users curriculum.
Now the logical backup, the backup process, a user purchased a course, then need to subtract the balance in the balance sheet, then add a course in the purchase of the course. Certainly the correct order to make a purchase operation, reducing the balance of the course and then increase during the backup. But there may be such a problem:
If the order in time to back up the balance sheet (u_account), then the user purchases (operating two tables), and then back up users curriculum (u_course)?
Then do to restore data backed up, users will find that they did not spend money to buy a lesson. The reason is that you back up the balance sheet, indicating that the user balance unchanged. After the operation was to make a purchase, minus the money balances, an increase in curriculum courses. Then backup curriculum, curriculum courses plus one. Purchase operation performed after the backup has been completed balance sheet.
If in order to back up user time curriculum (u_course), then the user purchases (operating two tables), and then back up the balance sheet (u_account)?
Similarly, if you back up the curriculum, course not increased because no purchase. Followed by after-purchase operations, balance sheet minus the money, then it is backed up. Appeared, users spend money but did not buy is successful.
In other words, not locked, then the library is not a logical point in time to get the backup system, this view is a logical inconsistency.
How to solve the problem view logical inconsistency?
For the engine does not support transactions, such as MyISAM. By using Flush tables with read lockto open the global lock (FTWRL) command.
But there is the use of FTWRL question is:
- When backing up in the main library, the update can not be performed during backup, basic business suspended.
- In the backup from the library, main library from the library can not be performed during backup synchronization over binlog, leading to a master-slave delay.
With the engine function of MVCC (InnoDB) to support the business and turned a consistent view (Repeatable Read), the backup is very simple.
Use the official mysqldumptool, plus --single-transactionoptions, and then export the data before it will start a transaction, to ensure to get a consistent view. And because of MVCC support, and can be updated operation.
Comparison of the whole library read-only setting method
Why is not recommended set global readonly=trueto use FTWRL:
In some systems, readonly values will be used to do other logic, such as a library is used to determine the main library or a library equipment. Therefore, modify global variables affect the way a larger surface, it is not recommended.
There are differences in the exception handling mechanism.
After executing the command because the client FTWRL abnormal disconnect, then MySQL automatically releases the global lock, the entire library may return to normal status updates.
After the entire library is set to readonly, an exception if the client fails, the database will remain readonly state, this will cause the entire library for a long time in a non-writable state, the higher the risk.
Table-level locking
What is the table-level lock?
Table lock is scoped to a particular table for locking, in two-level locks MySQL table A table lock is a lock metadata (meta data lock, MDL).
Table lock
And FTWRL similar, can be used lock tables … read/writeto lock a particular table. Upon release, may be used unlock tableswhen the lock is released or disconnected, active release.
It should be noted that the lock table this way, not only will limit other threads to read and write, but also defines the object's own operating threads.
If thread A executed lock tables t1 read, t2 write;operation.
For tables this time t1, the other threads can only be read-only, thread A can only be read-only, can not write.
For table t2, the thread A read only, read and write other threads are blocked.
Metadata lock
Table different manual locking latch, lock metadata is automatically added.
Why should MDL?
MDL is to ensure the correctness of reading and writing, such as when a query data in this case another thread to change the table structure, table structure and query results are inconsistent certainly not. In simple terms, ** MDL is to solve the problem while operating between DML and DDL's .
In MySQL 5.5 introduces MDL, at the time of one of DML, will add DML read lock. When DDL, will add MDL write lock.
Not mutually exclusive between the read lock allows multiple threads to the same table were DML.
Between the read-write lock, write lock between mutually exclusive, to ensure the safety of the operation of the change table structure.
- If there are two threads to simultaneously add fields to a table, to wait for another one to finish execution started.
- If a thread to read, to write another thread. The following table access time, a completion operation, it may be another.
MDL issues raised?
Add fields to the table, has led to the library hung up?
Because the MDL is automatic, and added to the table when the field or fields to modify or add an index, the data necessary to scan the whole table. So when large operating table, to be very careful so as not to affect the service line. But in fact, when operating small tables, or may go wrong. T is assumed small table. Shown in the illustration, four open session.
MySQL 5.7.27
Say you have a table called sync_test:
mysql> desc sync_test;
+-------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(255) | NO | | NULL | |
+-------+--------------+------+-----+---------+----------------+
2 rows in set (0.00 sec)Open transaction 1, insert the data. For transaction 1, the automatic application of the table sync_test of MDL read lock:
Open transaction 2, insert data. For transaction 2, the automatic application of the table sync_test of MDL read lock:
3 open affairs, change the table structure. For transaction 3, the application form will MDL sync_test write lock, this time due to the read-write lock mutex is blocked:
4 open affairs, insert data. For transaction 4, it will apply for sync_test of MDL read lock, due before the transaction 3 in advance applied for a write lock, so mutex is blocked:
Then if the query on this table is very frequent, and the client has reconnection mechanism, a new session will be renewed request after the timeout, the thread of this library will soon be full up.
How to secure resources to the table plus
Can also be seen by the above example, MDL will not release until the transaction commits, do the table structure changes, they must be careful not to lock the line leading to query and update. After opening the transaction, it did not end in a short time, which is due to the so-called long transactions caused. Is there a long-running transaction (innodb_trx table under information_schema) next time if you want to operate on a table DDL, you can first query, you can kill the transaction, then do DDL operations.
But sometimes kill may not be able, when the table is frequently used, new transactions may be here soon. Ideally, the alter table set in the waiting time, if in time to get the best, or else give up, do not block the statement. After this operation was repeated.
MariaDB has consolidated AliSQL this feature, so these two open source branch currently supported DDL NOWAIT / WAIT n this syntax.
ALTER TABLE tbl_name NOWAIT add column ...
ALTER TABLE tbl_name WAIT N add column ...Row-level locking
What is row-level locking?
MySQL's row lock is achieved by the engine itself invited all not all engines perform row lock, such as locking in MyISAM does not support line engine. Does not support row lock concurrency control means can only use table locks, which resulted in only one update at the same time in the implementation, it affects the degree of concurrency business. InnoDB supports row lock is to be replaced by MyISAM important reason.
Row lock is locked rows in the database table records. Such transaction A, B want to update the row of data simultaneously, will be performed in a certain order when updating, but can not be updated simultaneously.
The purpose is to reduce the row lock lock conflicts like the table level, to enhance the degree of concurrency business.
Two-phase locking protocol
In the InnoDB transaction, the row lock is plus, but not used up on the release when needed, but released after the end of the transaction, which is the two-phase locking protocol.
Suppose there is a table, the transaction process t A, B operating table t as follows:
在事务 A 的两条语句更新后，事务 B 更新操作会被阻塞。直到事务 A 中执行 commit 操作后才能执行。
两阶段锁在事务上的帮助
由于两阶段锁的特点，在事务结束时才会释放锁，所以需要遵循的一个原则是事务中需要锁多个行时，把有可能造成锁冲突，最可能影响并发度的锁尽量向后放。
比如购买课程的例子，顾客 A 购买培训机构 B 一门课程。涉及到操作：
- 顾客 A 的余额减少
- 培训机构 B 所在的余额增加。
- 插入一条交易信息的操作。
对于第二个操作，当有许多人同时购买时并发度就较高，出现锁冲突的情况也较高。所以将操作 2 放置一个事务的最后就更好。
当有时并发度过大时，我们会发现一种现象 CPU 的使用率接近 100%，但事务执行数量却很少。这就可能出现了死锁。
死锁的检查
当并发系统中不同的线程出现循环的资源依赖，等待别的线程释放资源时，就会让涉及的线程处于一直等待的情况。这就称为死锁。
如上图中，事务 A 对id =1 的所在行，加入了行锁。等待 id=2 的行锁。事务 B 对 id = 2 的行，加入了行锁。等待 id=1 的行锁。事务 A，B 等待对方资源的释放。
如何解决死锁
方式 一： 设置死锁的等待时间 innodb_lock_wait_timeout
还是 sync_test 这张表，模拟简单的锁等待情况，注意这里并不是死锁。开启两个事务 A，B. 同时对 id=1 这行进行更新。
事务 A 更新操作：
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> update sync_test set name="dead_lock_test" where id = 1;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0事务 B 更新操作：
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> update sync_test set name="dead_lock_test2" where id = 1;
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction可以看到事务 B 抛出了死锁等待的错误。
设置等待时间的问题
在 InnoDB 中，MySQL 默认的死锁等待时间是 50s. 意味着在出现死锁后，被锁住的线程要过 50s 被能退出，这对于在线服务说，等待时间过长。但如果把值设置的过小，如果是像上述例子这样是简单的锁等待呢，并不是死锁怎么办，就会出现误伤的情况。
方式二：发起死锁检测，发现死锁后，主动回滚某个事务，让其他事务继续执行。
MySQL 中默认就是打开状态，能够快速发现死锁的情况。
set innodb_deadlock_detect=on事务 A，B 互相依赖，造成死锁的例子：
开启事务 A:
mysql> begin;
mysql> update sync_test set name="dead_lock_test1" where id = 1;开启事务 A:
mysql> begin;
mysql> update sync_test set name="dead_lock_test3" where id = 3;Continue Transaction A:
mysql> update sync_test set name="dead_lock_test3_1" where id = 3;
# 会出现阻塞的情况Continue Transaction B:
mysql> update sync_test set name="dead_lock_test1_2" where id = 1;
ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transactionA blocked at this time to cancel the transaction, executed successfully.
However, deadlock detection is also an additional burden, whenever a transaction is locked, it is necessary to see it depends thread has not locked the others, and so on, and finally determine whether there is a waiting loop, which is deadlock . If all transactions will be updated the same row scene it? Each new thread is blocked, and will not be judged because of their added led to a deadlock, this is a time complexity is O (n) operation. Suppose there are 1000 concurrent threads to simultaneously update the same row, then the deadlock detection operation is 1000 * 1000 = 1 million this magnitude.
So, for updated frequently complicated by the large table, deadlock detection will lead to consume a lot of CPU.
How to avoid deadlock detection loss
Method One: If the business will not guarantee deadlock, the deadlock can temporarily turn off the check.
But this there is a certain risk, because there is no serious problem as the deadlock business design, rollback after a deadlock, and try again no problem occurs. But turn off deadlock detection, a lot of time out of the situation that may arise.
Method two: concurrency control.
If you can control for the amount of concurrency, such as the same line at the same time only a maximum of 10 threads in the update, then a very low cost deadlock detection, they will not have this problem. Specifically, the client doing concurrency control, but application client more, can not control. So concurrency control in database server, if the middleware can also be considered implemented in middleware.
Method three: reduce the probability of deadlock
The statistical line structure, split into multiple lines accumulated structure. For example, the amount of a teaching institution before the demolition of the line to 10 lines, the total revenue is equivalent to a total of the 10 rows of data. So that the original lock collision probability becomes 1/10, thus reducing the consumption of CPU deadlock detection. But becomes 0 in a portion of the rows, the code require special handling.
to sum up
This article, in turn introduced the global lock, the concept of table-level locks and row locks.
For global lock, using the InnoDB engine with the help of RR level and MVCC, you can let colleagues update the data in the backup.
For table-level locking, the time to update the table structure hot table, pay attention to the situation MDL write mutex lock, resulting in the database hang.
For row-level locking, the rational use of two-phase locking protocol, reducing lock conflicts. And pay attention to the situation of deadlock, deadlock detection to take appropriate means.