[I'm learning technology in Lagou training camp] How much do I know about mysql transactions and locks

Preface

Source of article content output: Lagou Education Java High Salary Training Camp;

I learned mysql transaction characteristics, isolation level, transaction control, etc. in the Lagou training camp, and I am no longer afraid of it during the interview.

ACID characteristics

In a relational database management system, for a logical unit of work to become a transaction, it must meet these four characteristics, the so-called ACID: Atomicity, Consistency, Isolation, and Durability ).

Atomicity

A transaction is an atomic operation unit, and all modifications to data are either executed or not executed.

Each write transaction will modify the BufferPool to generate the corresponding Redo/Undo log. Before the pages in the Buffer Pool are flushed to the disk, the log information will be written to the log file first. If there are no dirty pages in the Buffer Pool If the flashing is successful, the database is hung up at this time. After the database is restarted, it can be recovered through the Redo log to ensure that the data written in the dirty pages will not be lost. If the dirty page refresh is successful, and the database is hung up at this time, it needs to be realized through Undo.

Endurance

It means that once a transaction is committed, it will be permanent when it modifies the data in the database, and subsequent operations will not have any impact on it and will not be lost.

As shown in the figure below, the operations triggered by a "commit" action include: binlog landing, sending binlog, storage engine submission, flush_logs, check_point, transaction commit flag, etc. These are the means for the database to ensure the integrity and durability of its data.

The durability of MySQL is also related to WAL technology. Redo log can repair data when the system crashes and restarts, thereby ensuring the durability of the transaction. The atomicity can ensure logical persistence, and the data flashing of the storage engine can ensure physical persistence.

Isolation

It means that the execution of a transaction will not be interfered by other transactions. The internal operations and data used by a transaction are isolated from other concurrent transactions.

consistency

It means that the integrity limit of the database has not been destroyed before the transaction starts and after the transaction ends. Consistency is divided into constraint consistency and data consistency.

• Constraint consistency: the foreign key, check, unique index and other constraints specified when creating the table structure
• Data consistency: A result that is guaranteed by atomicity, durability, and isolation.

ACID and the relationship between them are shown in the figure below. Three of the four features are related to WAL, and they all need to be guaranteed by Redo and Undo logs.

The full name of WAL is Write-Ahead Logging, which writes the log first, and then writes to the disk.

Transaction control

Transaction concurrency

Concurrent transaction processing may bring some problems, such as: lost update, dirty read, non-repeatable read, phantom read, etc.

• Update loss
  When two or more transactions update the same row of records, there will be update loss. Can be divided into rollback coverage and submission coverage.

  Rollback coverage: A transaction rollback operation overwrites the submitted data of other transactions.

  Commit coverage: A transaction commit operation overwrites the submitted data of other transactions.

• Dirty read

  One transaction reads data modified but not committed by another transaction.

• Non-repeatable

  Reading the same row multiple times in a transaction is inconsistent, and the later read is inconsistent with the previous read.

• Phantom reading

  In a transaction, the results are inconsistent when the same conditions are used for multiple queries. The results of subsequent queries are different from the results of previous queries, with a few more or fewer rows of records.

Exclusive lock

After the introduction of locks, concurrent processing of transactions can be supported. If the same data items are involved between transactions, exclusive locks, or mutex locks, will be used. After the first entry transaction monopolizes the data item, other transactions are blocked and wait for the previous The transaction releases the lock.

The lock will not be released before the entire transaction 1 ends, so transaction 2 must wait until transaction 1 ends.

Read-write lock

Read and write operations: read and write, write and write, read and write, write and read.

The read-write lock is to further refine the granularity of the lock, distinguish between read operations and write operations, so that there is no lock between read and read, so that the following two transactions can be executed at the same time.

Read and write locks can make reading and reading parallel, and there is still an exclusive lock between reading and writing, writing and reading, and writing and writing.

MVCC

Multi-version control MVCC, which is the idea of ​​Copy on Write. In addition to supporting read and read parallelism, MVCC also supports read and write, write and read parallelism, but in order to ensure consistency, writing and writing cannot be parallelized.

When transaction 1 starts the write operation, a copy of the record will be copied, and other transaction read operations will read this copy of the record, so it will not affect the reading of this record by other transactions, achieving parallel writing and reading.

concept

MVCC (Multi Version Concurrency Control) is called multi-version control, which refers to the multi-version processing of data in order to achieve high concurrent data access in the database, and the visibility of the transaction to ensure that the transaction can see what it should see Data version. Multi-version control cleverly converts the exclusive mutual exclusion of scarce resources into concurrency, which greatly improves the throughput and read-write performance of the database. How to generate multiple versions? Before each transaction modification operation, the data status and transaction number before the modification will be recorded in the Undo log. The backup record can be used for reading other transactions, and data rollback when necessary.

MVCC realization principle

The biggest advantage of MVCC is that reads are not locked, and reads and writes do not conflict. In system applications with more reads and less writes, it is very important that reads and writes do not conflict, which greatly improves the concurrent performance of the system. This is why almost all relational databases support MVCC at this stage, but currently MVCC is only available in Read Commited and Repeatable Read work under two isolation levels.

In MVCC concurrency control, read operations can be divided into two categories: Snapshot Read and Current Read

• Snapshot read: Read the recorded snapshot version (maybe historical version) without any lock. (Select)
• Current read: The latest version of the record is read, and the record returned by the current read will be locked to ensure that other transactions will not concurrently modify this record. (Select... for update or lock in share mode, insert/delete/update)

In order to let everyone more intuitively understand the implementation principle of MVCC, let's take a record update case to explain the implementation of multiple versions in MVCC. Suppose F1～F6 are the names of the fields in the table, and 1～6 are the corresponding data. The following three hidden fields correspond to the hidden ID, transaction number and rollback pointer of the row, as shown in the following figure

The specific update process is as follows:

1. Lock the row with an exclusive lock; record the Redo log

2. Copy the value of the row before modification to Undo log, which is the lower row in the figure;

3. Modify the value of the current row, fill in the transaction number, and make the rollback pointer point to the row before the modification in the Undo log.

MVCC has implemented concurrent processing of reading, reading, and writing. If you want to further resolve write-write conflicts, you can use the following two solutions: optimistic locking and pessimistic locking.

Transaction isolation level

MySQL database is solved through transaction isolation level. The database system provides the following 4 transaction isolation levels for users to choose.

• Read uncommitted

  Read Uncommitted: It solves the loss of update of the rollback coverage type, but the phenomenon of dirty read may occur, that is, the data modified by the uncommitted transaction in other sessions may be read.

• Submitted

  Read Committed: Only the submitted data in other sessions can be read, which solves dirty reads. However, non-repeatable reads may occur, that is, two query results may be inconsistent in a transaction.

• Repeatability

  Repeatable Read: It solves the problem of non-repeatable read. It ensures that multiple instances of the same transaction will see the same data row when reading data concurrently. But in theory, there will be phantom reading. Simply put, phantom reading refers to when the user reads a range of data rows, another transaction inserts a new row in the range, and when the user is reading the data in the range You will find a new phantom line.

• Serializable

  Serializable: Serial execution of all additions, deletions, and modifications. It solves the problem of illusion degree by forcing the order of affairs to resolve conflicts. This level may lead to a large number of timeouts and lock contention, which is inefficient.

The higher the transaction isolation level of the database, the smaller the concurrency problem, but the worse the concurrent processing capability (cost). The read uncommitted isolation level is the lowest, and there are many concurrency problems, but the concurrency processing capability is good. In the future, you can choose an appropriate isolation level according to the characteristics of the system. For example, you are not sensitive to non-repeatable reads and phantom reads, and you are more concerned about the concurrent processing capabilities of the database. At this time, you can use the Read Commited isolation level.

The relationship between transaction isolation level and lock

1. The transaction isolation level is a standard customized by SQL92, which is equivalent to the overall solution for transaction concurrency control. It is essentially a encapsulation of the use of locks and MVCC, which hides the underlying details.

2. Locking is the basis for concurrency control of the database. Transaction isolation is achieved by using locks. Adding different locks to the corresponding operations can prevent other transactions from reading and writing data at the same time.

3. For the user, first choose to use the isolation level. When the selected isolation level cannot solve the concurrency problem or demand, it is necessary to manually set the lock during development.

MySQL default isolation level: repeatable read. Oracle, SQLServer default isolation level: read submitted

In general use, it is recommended to use the default isolation level, and then some concurrency problems can be handled through pessimistic locking, optimistic locking, etc.

MySQL isolation level control

View the isolation level of the database:

show variables like '%transaction_isolation%';

You can set the transaction isolation level with the following commands:

 set global transaction_isolation ='READ-UNCOMMITTED'; 
 set global transaction_isolation ='READ-COMMITTED'; 
 set global transaction_isolation ='REPEATABLE-READ'; 
 set global transaction_isolation ='SERIALIZABLE';

Lock mechanism

Lock classification

There are many different categories of locks in MySQL

• The granularity of operations can be divided into table-level locks, row-level locks, and page-level locks.
  • Table-level lock: lock the entire table for each operation. Large locking granularity, the highest probability of lock conflicts, and the lowest concurrency. Used in storage engines such as MyISAM, InnoDB, and BDB.
  • Row-level lock: lock a row of data for each operation. The locking granularity is the smallest, the probability of lock conflicts is the lowest, and the concurrency is the highest. Used in the InnoDB storage engine.
  • Page-level locks: lock an adjacent set of records each time, the locking granularity is between table locks and row locks, overhead and locking time are between table locks and row locks, and the concurrency is average. Used in the BDB storage engine.
• The type of slave operation can be divided into read lock and write lock
  • Read lock (S lock): Shared lock, for the same data, multiple read operations can be performed at the same time without affecting each other.
  • Write lock (X lock): Exclusive lock, it will block other write locks and read locks before the current write operation is completed.

IS locks and IX locks: Intentional read locks and intentional write locks are table-level locks. S and X are mainly for row-level locks. Before adding S or X locks to table records, IS or IX locks are added to the table first.

S lock: Transaction A adds an S lock to the record, which can read the record but cannot modify it. Other transactions can add an S lock to the record, but cannot add an X lock. You need to add an X lock, and you need to wait for the record's S lock Release all.

X lock: Transaction A adds X lock to the record, which can read and modify the record, while other transactions cannot read and modify the record.

• The performance of the operation can be divided into optimistic locking and pessimistic locking.

  • Optimistic lock: The general implementation is to compare the recorded data version, and conflict detection will be performed when the data update is submitted. If a conflict is found, an error message will be displayed.

  • Pessimistic lock: When modifying a piece of data, in order to avoid being modified by others at the same time, it is a control method that locks the data before modifying it and then modifies it. Shared locks and exclusive locks are different implementations of pessimistic locks, but both belong to the category of pessimistic locks.

Row lock principle

In the InnoDB engine, we can use row locks and table locks. Row locks are divided into shared locks and exclusive locks. InnoDB row lock is achieved by locking the records on the index data page. There are three main implementation algorithms: Record Lock, Gap Lock and Next-key Lock.

• RecordLock lock: lock a single row record. (Record lock, RC, RR isolation levels are supported)

• GapLock lock: gap lock, lock the gap of index records, to ensure that the gap of index records remains unchanged. (Range lock, RR isolation level support)

• Next-key Lock: Combination of record lock and gap lock, lock data at the same time, and lock the front and back range of data. (Record lock + range lock, RR isolation level support)

At the RR isolation level, InnoDB first uses Next-Key Lock for record locking behavior, but when SQL operations contain unique indexes, Innodb will optimize Next-Key Lock and downgrade to RecordLock, which only locks the index itself instead of range.

 1）select ... from 语句：InnoDB引擎采用MVCC机制实现非阻塞读，所以对于普通的select语句，InnoDB不加锁

 2）select ... from lock in share mode语句：追加了共享锁，InnoDB会使用Next-Key Lock锁进行处理，如果扫描发现唯一索引，可以降级为RecordLock锁。

 3）select ... from for update语句：追加了排他锁，InnoDB会使用Next-Key Lock锁进行处理，如果扫描发现唯一索引，可以降级为RecordLock锁。

 4）update ... where 语句：InnoDB会使用Next-Key Lock锁进行处理，如果扫描发现唯一索引，可以降级为RecordLock锁。

 5）delete ... where 语句：InnoDB会使用Next-Key Lock锁进行处理，如果扫描发现唯一索引，可以降级为RecordLock锁。

 6）insert语句：InnoDB会在将要插入的那一行设置一个排他的RecordLock锁。

Pessimistic lock

Pessimistic locking (Pessimistic Locking) refers to the process of data processing, the data is locked, generally using the database lock mechanism to achieve. Broadly speaking, the aforementioned row locks, table locks, read locks, write locks, shared locks, exclusive locks, etc., all belong to the category of pessimistic locks.

• Table-level lock

  Table-level locks lock the entire table for each operation, with the lowest concurrency. Common commands are as follows:

  Manually add table lock

  lock table 表名称 read|write,表名称2 read|write;
  

  View the locks added on the table

  show open tables;
  

  Drop table lock

  unlock tables;
  

  Table-level read lock: The read lock is added to the current table, and the current connection and other connections can be read; but the current connection addition, deletion, and modification operation will report an error, and other connection additions, deletions, and modifications will be blocked.

  Table-level write lock: Write lock is added to the current table. The current connection can perform addition, deletion, modification, and check operations on the table. All other connections on the table are blocked (including queries).

  Summary: Table-level read locks will block write operations, but not read operations. The write lock will block both read and write operations.

• Shared lock (row-level lock-read lock)

  Shared locks are also called read locks, or S locks for short. Shared lock means that multiple transactions can share a lock for the same data and can access the data, but can only be read but not modified. Method of using shared lock

  select ... lock in share mode;
  

  Only applicable to query statements.

  Summary: The transaction uses a shared lock (read lock), which can only be read but cannot be modified, and the modification operation is blocked.

• Exclusive lock (row-level lock-write lock)

  Exclusive locks are also called write locks, or X locks for short. An exclusive lock cannot coexist with other locks. For example, if a transaction acquires an exclusive lock on a data row, other transactions cannot perform other operations on the row record, nor can it acquire the lock on the row. The way to use exclusive locks is to add at the end of SQL

  select ... for update;
  

  By default, the innodb engine will add for update to the update and delete statements. The realization of row-level lock actually relies on its corresponding index, so if the operation does not use index query, then the entire table record will be locked.

  Summary: The transaction uses an exclusive lock (write lock), the current transaction can read and modify, and other transactions cannot modify or obtain records

  Lock (select... for update). If the query does not use the index, the entire table record will be locked.

Optimistic lock

Optimistic lock is relative to pessimistic lock. It is not a function provided by the database and needs to be implemented by the developer. In the database operation, the idea is very optimistic, I think this operation will not cause conflicts, so no special processing is done during the database operation, that is, no locks, but to judge whether there is a conflict when the transaction is committed Up.

The key point of optimistic locking is the detection of conflicts.

Both pessimistic locks and optimistic locks can solve transaction write and write concurrency. In applications, you can choose to distinguish based on concurrent processing capabilities, such as optimistic locks for high concurrency requirements; pessimistic locks for low concurrency requirements.

Principles of Optimistic Locking

• Use the version field (version)

  First add a version field to the data table, and add 1 to the version number of that record for each operation. version

  It is used to check whether the read record has changed, and the function is to prevent the record from being modified by other transactions during business processing.

• Use Timestamp (Timestamp)

  Similar to using the version field, you also need to add a field to the data table, and the field type uses timestamp. It is also when the update is submitted to check the timestamp of the data in the current database and compare it with the timestamp taken before the update, if they are consistent, submit the update, otherwise it is a version conflict and cancel the operation.

Deadlock

Four necessary conditions for deadlock:

Mutually exclusive conditions: A resource can only be occupied by one process at a time.

Inalienable: A resource cannot be deprived unless the occupied process actively releases it

Request hold: When a resource is requested, the request is kept until the resource is not obtained

Circular waiting: process a waits for resources held by process b, and process b waits for resources held by process a

to sum up

Remember all knowledge points such as transaction characteristics, isolation levels, exclusive locks, shared locks, read-write locks, MVVC, etc. are almost the same. Remember to collect if you can’t remember, don’t get lost haha