python deadlock learning

Examples

This article summarizes Python deadlock articles and explains some of the incomprehensible content.
First enumerate a situation where a deadlock will occur

死锁的一个原因是互斥锁。假设银行系统中，用户a试图转账100块给用户b，
与此同时用户b试图转账200块给用户a，则可能产生死锁。
2个线程互相等待对方的锁，互相占用着资源不释放。
#coding=utf-8
import time
import threading
class Account:
  def __init__(self, _id, balance, lock):
    self.id = _id
    self.balance = balance
    self.lock = lock

  def withdraw(self, amount):
    self.balance -= amount

  def deposit(self, amount):
    self.balance += amount


def transfer(_from, to, amount):
  if _from.lock.acquire():#锁住自己的账户
    _from.withdraw(amount)
    time.sleep(1)#让交易时间变长，2个交易线程时间上重叠，有足够时间来产生死锁
    print('wait for lock...')
    if to.lock.acquire():#锁住对方的账户
      to.deposit(amount)
      to.lock.release()
    _from.lock.release()
  print( 'finish...')

a = Account('a',1000, threading.Lock())
b = Account('b',1000, threading.Lock())
threading.Thread(target = transfer, args = (a, b, 100)).start()
threading.Thread(target = transfer, args = (b, a, 200)).start()

In multithreaded programs, a large part of the deadlock problem is caused by threads acquiring multiple locks at the same time. For example: a thread acquires the first lock, and then blocks when acquiring the second lock, then this thread may block the execution of other threads, resulting in the death of the entire program. One solution to the deadlock problem is to assign a unique id to each lock in the program, and then only allow multiple locks to be used in ascending order . This rule is very easy to implement using a context manager

First look at what is the context manager @contextmanager
The function decorated by the decorator is divided into three parts:

1. The code block in the with statement executes the code before yield in the function
2. The content returned by yield is copied to the variable after as
3. After the with code block is executed, the code after the yield in the function is executed

solution

import threading
from contextlib import contextmanager

# Thread-local state to stored information on locks already acquired
_local = threading.local()


@contextmanager
def acquire(*locks):
  # Sort locks by object identifier
  locks = sorted(locks, key=lambda x: id(x))

  # Make sure lock order of previously acquired locks is not violated
  acquired = getattr(_local, 'acquired', [])
  #如果出现新锁id大于旧锁id 报错
  if acquired and max(id(lock) for lock in acquired) >= id(locks[0]):
    raise RuntimeError('Lock Order Violation')
  # Acquire all of the locks
  acquired.extend(locks)
  _local.acquired = acquired
  try:
    for lock in locks:
      lock.acquire()
    yield		#无返回值 yield为空
  finally:
    # Release locks in reverse order of acquisition
    for lock in reversed(locks):
      lock.release()
    del _local.acquired[-len(locks):]

Of course, there are certain problems in acquiring locks in ascending order, see the following code

import threading
x_lock = threading.Lock()
y_lock = threading.Lock()
 
def thread_1():
  while True:
        with acquire(x_lock):
            print("acquire x_lock")
            time.sleep(1)
            with acquire(y_lock):
                print('Thread-1')
                time.sleep(1)


def thread_2():
  while True:
        with acquire(y_lock):
            print('acquire y_lock')
            time.sleep(1)
            with acquire(x_lock):
                print('Thread-2')
 
t1 = threading.Thread(target=thread_1)
t1.daemon = False
t1.start()
 
t2 = threading.Thread(target=thread_2)
t2.daemon = False
t2.start()

daemon:
(1)如果某个子线程的daemon属性为False，主线程结束时会检测该子线程是否结束，
如果该子线程还在运行，则主线程会等待它完成后再退出；
(2)如果某个子线程的daemon属性为True，主线程运行结束时不对这个子线程进行检
查而直接退出，同时所有daemon值为True的子线程将随主线程一起结束，而不论
是否运行完成。

The result of the code running is an exception

. The reason for the crash is that each thread records the lock it has acquired. The acquire () function checks the list of previously acquired locks. Since the locks are acquired in ascending order, the function assumes that the id of the previously acquired lock must be less than the newly acquired lock, and an exception is triggered.
The reason for the above result is that the exception first appeared in Thread-2, causing the thread to be interrupted. After that, there will be no deadlock, because only one thread Thread-1 is running, and through the context manager, the lock will be automatically released after each internal acquire call, so there will be no deadlock.

to sum up:

Avoiding deadlock is a way to solve the deadlock problem. When the process acquires the lock, it will be arranged in strict ascending order of the object id. After mathematical proof, this ensures that the program will not enter the deadlock state. The main idea of ​​avoiding deadlock is that simply adding locks in the order of increasing object id will not create a circular dependency, and circular dependency is a necessary condition for deadlock, so as to avoid the program from entering a deadlock state.