On the PostgreSQL lock mechanism

Lock mechanism is very important in PostgreSQL (The same is true for other modern RDBMS). For database application developers (programmers, especially those related to highly concurrent code), need to be very familiar with the lock. For some problems, lock focus and needs to be checked. In most cases, these problems with deadlocks or data inconsistencies are related, basically due to the locking mechanism of the Postgres do not know the cause. Although the lock mechanism is very important within the Postgres, but the lack of documentation is very lacking, or sometimes even wrong, and the result of the document pointed out inconsistencies. I will tell you everything proficient Postgres locking mechanism need to know to know to lock more understanding to solve problems associated with the lock will be faster.

Documentation said what?

Postgres has three lock mechanisms: table-level locking, row-level locking and advisory locks. Table and row-level locks may be explicit or implicit type. Advisory lock is generally explicit. Explicit locks acquired by explicit user request (through a special query), implicit lock is acquired by the standard SQL commands.

In addition to table and row level locking, as well as page-level share / exclusion locks to control access to shared cache pool table pages. After a row of data to be read or updated, the lock is released immediately. Application developers typically do not need to concern page-level locking.

Lock mechanism will change from time to time, it is only for Postgres 9.x version here. 9.1 and 9.2 are essentially similar, they are 9.3 and 9.4 with some differences, mainly related to row-level locking.

 

 

Table-level locking

Most of the table-level locking is obtained by a built-in SQL commands, but they can also be obtained by explicitly lock command. Using table lock comprising:

• Access to a shared (ACCESS SHARE) - get the lock on the table SELECT command can be referenced in the query. The general rule is that all inquiries only meter reading only acquire this lock.

• Line Sharing (ROW SHARE) - SELECT FOR UPDATE and SELECT FOR SHARE commands available (access to a shared lock, and query all references to the table) the lock on the target table.

• Exclusive row (ROW EXCLUSIVE) - UPDATE, INSERT, and DELETE commands to get (access to a shared lock, and query all references to the table) the lock on the target table. The general rule is to modify the query to get all the lock table.

• Exclusive shared update (SHARE UPDATE EXCLUSIVE) - VACUUM (excluding FULL), ANALYZE, CREATE INDEX CONCURRENTLY, ALTER TABLE, and some commands get the lock.

• Shared (SHARE) - get the lock on the table CREATE INDEX command referenced in the query.

• Shared row exclusive (SHARE ROW EXCLUSIVE) - not get any implicit command.

• Exclusive (EXCLUSIVE) - This lock mode is obtained when this lock transaction can only read operations in parallel. It can not be acquired by any command implicitly.

• Exclusive access (ACCESS EXCLUSIVE) - get the lock on the table ALTER TABLE, DROP TABLE, TRUNCATE, REINDEX, CLUSTER, and VACUUM FULL commands referenced in the query. This lock mode is the default mode LOCK command.

It is important to know that all these locks are table-level locks, even if the name, there are lines (ROW) word.

The most important information for each lock mode is the mode of conflicting lists. At the same time the same table, the two transactions can not lock mode while maintaining the conflict. Transaction never conflicts with itself occurs. Non-locking conflicts can support multiple concurrent transactions. It is also important to know that some patterns and their own conflicts. Some lock mode after obtaining lasts until the end of the transaction. But if the lock is in the establishment of a point to get saved, save point after the rollback lock will be released immediately. The following table shows which mode is conflicting:

 

Row-level locking

There are two row-level locking mode in Postgres 9.1 and 9.2, but there are four row-level locking mode in Postgres 9.3 and 9.4.

Any information modified rows in memory of Postgres will not remember, the number of rows locked at one time is not limited. However, locking a row may cause a disk write, for example, SELECT FOR UPDATE Edit selected rows and mark them locked, it will result in disk writes.

Postgres 9.1 and 9.2 row-level locking

In both versions, only two kinds of row-level locking: exclusive or shared locks. When the row updated or deleted automatically obtain an exclusive row-level locking. Row-level locks do not prevent data query, they only prevent the same line as written. Exclusive row lock by SELECT FOR UPDATE command to obtain clear, even if the row is not actually changed.

Shared row-level locking by SELECT FOR SHARE commands available. A shared lock does not prevent other transactions from acquiring the same shared lock. However, when any other transaction holds a shared lock, update transactions, delete, or exclusive locks are not allowed.

 

Postgres 9.3 and 9.4 row-level locking

There are four types of row-level locking in Postgres 9.3 and 9.4:

• Update (FOR UPDATE) - This mode reads result in updating the SELECT line is locked. This prevents them from being locked by another transaction, modified or deleted. That attempt UPDATE, DELETE, SELECT FOR UPDATE, SELECT FOR NO KEY UPDATE, SELECT FOR SHARE other transactions or SELECT FOR KEY SHARE will be blocked. Delete a row, update some of the columns can also get to this lock mode (the current set of columns are those that have a unique index, and can be used as a foreign key - but this may change in the future).

• Keyless Update (FOR NO KEY UPDATE) - This mode is similar FOR UPDATE, but weaker - it does not block SELECT FOR KEY SHARE lock mode. It was adopted does not get updated lock UPDATE command.

• Share (FOR SHARE) - This mode locks with keyless update is similar, except that it can acquire a shared lock (non-exclusive). A shared lock prevents other transactions UPDATE, DELETE, SELECT FOR UPDATE or SELECT FOR NO KEY UPDATE operations on these lines, but does not prevent them or SELECT FOR SHARE SELECT FOR KEY SHARE.

• Shared key (FOR KEY SHARE) - share similar behavior, but the lock is weak: to prevent the SELECT FOR UPDATE, but does not prevent SELECT FOR NO KEY UPDATE. A shared key lock prevents other transactions or any changes to the DELETE keys UPDATE, without prejudice to any other UPDATE, SELECT FOR NO KEY UPDATE, SELECT FOR SHARE or SELECT FOR KEY SHARE.

Row-level locking conflict:

 

Page-level locking

In addition to table-level and row-level locking other than page-level share / exclusive locks are used to control read / write to the shared buffer pool page table. These locks are released immediately after the row is fetched or updated. Application developers typically do not care about page-level locking, we mention them here only for completeness.

Lock advice

Postgres provides lock has created an application-defined method, which is known as advised lock (advisory locks), because the system does not support its use, depending on the application proper use of the lock.

Postgres There are two ways to get a lock advice: Session level or transaction level. Once advised lock in session level, it will be maintained until the end of the session or explicit release. Unlike standard lock requests, session level lock request does not comply with the advice of the transaction semantics: the transaction is rolled back after the lock will be maintained with the rollback, the same way even after calling lock transaction fails, the unlock request is still valid of. A lock can be its own process of multiple acquisitions; lock request for each completed before the lock is actually released there must be a corresponding unlock request.

On the other hand, transaction-level lock requests behave more like a normal lock requests: they are automatically released at the end of the transaction, and there is no explicit unlock operation. For short-term use of advice lock, this feature is usually more convenient than the session level. It is conceivable that a session-level and transaction-level lock identifier request will block each other the same advice. If a session has been advised a lock, then it will always be requested when successful, even if another session is waiting for the lock; whether to maintain the existing lock and the new request is a session-level or transaction level, are like this. Documentation can be found in the full list of functions that operate advised lock.

Here are a few examples of transaction-level advice to obtain the lock (pg_locks is a system view, it will be explained later article there information and advice table-level locking locks held by the transaction.):

The first start psql session, begin a transaction and get a lock advice:

-- Transaction 1
BEGIN;
SELECT pg_advisory_xact_lock(1);
-- Some work here

    Psql now start a second session and in the same lock advised to perform a new transaction:

-- Transaction 2
BEGIN;
SELECT pg_advisory_xact_lock(1);
-- This transaction is now blocked

In the third psql session where we can look at the lock current situation:

SELECT * FROM pg_locks;-- Only relevant parts of output
   locktype    | database | relation | page | tuple | virtualxid | transactionid | classid | objid | objsubid | virtualtransaction |  pid  |        mode         | granted |fastpath---------------+----------+----------+------+-------+------------+---------------+---------+-------+----------+--------------------+-------+---------------------+---------+----------
    advisory   |    16393 |          |      |       |            |               |       0 |     1 |        1 | 4/36               |  1360 | ExclusiveLock       | f       | f
    advisory   |    16393 |          |      |       |            |               |       0 |     1 |        1 | 3/186              | 14340 | ExclusiveLock       | t       | f
-- Transaction 1
COMMIT;
-- This transaction now released lock, so Transaction 2 can continue

 

We can also call the non-blocking method to get the lock, these methods will attempt to acquire a lock, and returns true (if successful) or false (if you can not get a lock).

-- Transaction 1
BEGIN;
SELECT pg_advisory_xact_lock(1);
-- Some work here
-- Transaction 2
BEGIN;
SELECT pg_try_advisory_xact_lock(1) INTO vLockAcquired;
IF vLockAcquired THEN
-- Some work
ELSE
-- Lock not acquired
END IF;
-- Transaction 1
COMMIT;

 

Monitoring locks

The basic configuration of all the activities held by the transaction monitoring system lock is the view pg_locks. The view for each object can be locked, the lock mode has been requested and the associated transaction contains a row. Very important point is that, pg_locks hold information key memory is tracked, so it does not display row-level locking! (Translation: According to the investigation the previous document, information on row-level locking is on disk rather than memory) This view shows the table-level locks and lock advice. If a transaction ID for the transaction is waiting for the fixed row-level locks current owner of waiting for a row-level locks, which usually appear in the view. This makes it more difficult to debug row-level locking. In fact, anywhere you can not see row-level locking, blocking until someone holding this lock transaction (then you can see a tuple is locked in pg_locks table). pg_locks is poor readability of view (not very user-friendly), so we have to make the view display lock information well taken:

-- View with readable locks info and filtered out locks on system tables
CREATE VIEW active_locks AS
SELECT clock_timestamp(), pg_class.relname, pg_locks.locktype, pg_locks.database,
       pg_locks.relation, pg_locks.page, pg_locks.tuple, pg_locks.virtualtransaction,
       pg_locks.pid, pg_locks.mode, pg_locks.granted
FROM pg_locks JOIN pg_class ON pg_locks.relation = pg_class.oid
WHERE relname !~ '^pg_' and relname <> 'active_locks';
-- Now when we want to see locks just type
SELECT * FROM active_locks;

 

- View the session the session 
the SELECT pg_backend_pid (); 

- Check Lock session held by 
the SELECT  *  from the pg_locks the WHERE pid = 3797 ; 

- 1, view the database 

the SELECT   pg_database.datname, pg_database_size (pg_database.datname) AS size from pg_database; / / query the database for all, and the amount of space 

- 2. query latched in the data table 

the SELECT . a.locktype, a database , a.pid, a.mode, a.relation, b.relname - , SA. * 
from the pg_locks A
 the Join the pg_class entry that B ON a.relation =b.oid 
 Inner  the Join   the pg_stat_activity SA ON a.pid = sa.procpid 

- 3. queries within a table, the lock status and the associated lock query 

the SELECT a.locktype, A. Database , a.pid, A. MODE, a.relation, b.relname - ., * SA 
from the pg_locks A
 the Join the pg_class entry that B ON a.relation = b.oid 
 Inner  the Join   the pg_stat_activity SA ON a.pid = sa.procpid
 WHERE . A Database = 382 790 774   and SA. waiting_reason = ' Lock' 
Order  by sa.query_start
 - 4. Check database table size 

SELECT pg_database_size ( ' Playboy ' );

 

Deadlock

The possibility of the use of explicit locking can increase deadlock, deadlock is when two (or more) transactions each hold locks the other side wants.
For example, if the transaction is 1 in table A an exclusive lock while trying to get one on the table B exclusive lock and Transaction 2 already hold in Table B of the exclusive lock while it is requesting a row on the A table locks, so neither one can proceed. PostgreSQL automatically detects deadlock situations and will to solve this problem by interrupting one of the transactions to allow other transactions to complete (Exactly which transaction will be interrupted is difficult to predict, and should not rely on such predictions).
Deadlocks may also to be noted that as a result of the row-level locks occur (and therefore, they can also occur even without the use of an explicit lock). Consider a case in which two concurrent transactions modify a table. The first transaction execution:

       
       


        
        
UPDATE accounts SET balance = balance + 100.00 WHERE acctnum = 11111;

       
       

Thus obtaining a row-level lock on the specified account row. Then, the second transaction execution:

UPDATE accounts SET balance = balance + 100.00 WHERE acctnum = 22222;
UPDATE accounts SET balance = balance - 100.00 WHERE acctnum = 11111;

第一个UPDATE语句成功地在指定行上获得了一个行级锁，因此它成功更新了该行。 但是第
二个UPDATE语句发现它试图更新的行已经被锁住了，因此它等待持有该锁的事务结束。事
务二现在就在等待事务一结束，然后再继续执行。现在，事务一执行：

UPDATE accounts SET balance = balance - 100.00 WHERE acctnum = 22222;

事务一试图在指定行上获得一个行级锁，但是它得不到：事务二已经持有了这样的锁。所以
它要等待事务二完成。因此，事务一被事务二阻塞，而事务二也被事务一阻塞：一个死锁。
PostgreSQL将检测这样的情况并中断其中一个事务。

防止死锁的最好方法通常是保证所有使用一个数据库的应用都以一致的顺序在多个对象上获得锁。在上面的例子里，如果两个事务以同样的顺序更新那些行，那么就不会发生死锁。 我们也应该保证一个事务中在一个对象上获得的第一个锁是该对象需要的最严格的锁模式。如果我们无法提前验证这些，那么可以通过重试因死锁而中断的事务来即时处理死锁。
只要没有检测到死锁情况，寻求一个表级或行级锁的事务将无限等待冲突锁被释放。这意味着一个应用长时间保持事务开启不是什么好事（例如等待用户输入）。

咨询锁

• PostgreSQL提供了一种方法创建由应用定义其含义的锁。这种锁被称为咨询锁，因为系统并不强迫其使用 — 而是由应用来保证其正确的使用。咨询锁可用于 MVCC 模型不适用的锁定策略。例如，咨询锁的一种常用用法是模拟所谓“平面文件”数据管理系统典型的悲观锁策略。虽然一个存储在表中的标志可以被用于相同目的，但咨询锁更快、可以避免表膨胀并且会由服务器在会话结束时自动清理。
• 有两种方法在PostgreSQL中获取一个咨询锁：在会话级别或在事务级别。一旦在会话级别获得了咨询锁，它将被保持直到被显式释放或会话结束。不同于标准锁请求，会话级咨询锁请求不尊重事务语义：在一个后来被回滚的事务中得到的锁在回滚后仍然被保持，并且同样即使调用它的事务后来失败一个解锁也是有效的。一个锁在它所属的进程中可以被获取多次；对于每一个完成的锁请求必须有一个相应的解锁请求，直至锁被真正释放。在另一方面，事务级锁请求的行为更像普通锁请求：在事务结束时会自动释放它们，并且没有显式的解锁操作。这种行为通常比会话级别的行为更方便，因为它使用一个咨询锁的时间更短。对于同一咨询锁标识符的会话级别和事务级别的锁请求按照期望将彼此阻塞。如果一个会话已经持有了一个给定的咨询锁，由它发出的附加请求将总是成功，即使有其他会话在等待该锁；不管现有的锁和新请求是处在会话级别还是事务级别，这种说法都是真的。
• 和所有PostgreSQL中的锁一样，当前被任何会话所持有的咨询锁的完整列表可以在pg_locks系统视图中找到
• 咨询锁和普通锁都被存储在一个共享内存池中，它的尺寸由max_locks_per_transaction和max_connections配置变量定义。 必须当心不要耗尽这些内存，否则服务器将不能再授予任何锁。这对服务器可以授予的咨询锁数量设置了一个上限，根据服务器的配置不同，这个限制通常是数万到数十万。
• 在使用咨询锁方法的特定情况下，特别是查询中涉及显式排序和LIMIT子句时，由于 SQL 表达式被计算的顺序，必须小心控制锁的获取。例如：

SELECT pg_advisory_lock(id) FROM foo WHERE id = 12345; -- ok
SELECT pg_advisory_lock(id) FROM foo WHERE id > 12345 LIMIT 100; -- danger!
SELECT pg_advisory_lock(q.id) FROM
(
 SELECT id FROM foo WHERE id > 12345 LIMIT 100
) q; -- ok

 

在上述查询中，第二种形式是危险的，因为不能保证在锁定函数被执行之前应用LIMIT。这
可能导致获得某些应用不期望的锁，并因此在会话结束之前无法释放。 从应用的角度来看，
这样的锁将被挂起，虽然它们仍然在pg_locks中可见。

 

Reference: https://www.oschina.net/translate/postgresql-locking-revealed

        https://www.jianshu.com/p/ee73c17f2d71