Lock & RLock: mutex to ensure multithreaded access to shared variables
Semaphore objects: Lock mutex enhanced version, can also have multiple threads, and Lock can only be one thread simultaneously owns.
Event Object: It is the means of communication between threads, it corresponds to the signal, after a thread to another thread can send a signal allowed to perform an operation.
Condition objects: Conditions that can trigger certain events or achieve specific data after processing
First, thread synchronization lock (mutex)
concept
-
Thread synchronization, inter-thread coordination, through some kind of technology that allows one thread to access certain data, other threads can not access the data until the thread to complete the operation on the data.
-
Lock (Lock): Once a thread to acquire a lock, other threads trying to acquire the lock will be blocked to wait. Lock: Where there is competition for places shared support can use locks to ensure that only one user can fully use the resources.
Lock application scenarios
When the lock is suitable for access and modify the same shared resource, that resource to read and write the same time.
Use lock Notes:
1, less lock, with a lock, if necessary. Use the lock is locked when multiple threads access a resource, it becomes a serial or queued for execution, either fighting for execution. For example, the car on the highway running parallel, but to the provincial boundary is only open a toll gate, after this opening, the vehicle can still run together on multi-lane. When crossing the toll gate, if a vehicle line up too, plus an unlocked quite the same efficiency, but once the competition, a vehicle must be locked too. Note that regardless of the increase is not locked, as long as a vehicle, efficiency declined.
2, lock the shorter the better, you do not need to immediately release the lock.
3, be sure to avoid deadlock.
Thread lock and GIL (Global Interpreter Lock)
Note points: 1. grab the thread is GIL lock, lock GIL equivalent to execute permissions, got to get the mutex lock Lock after execute permissions, other threads can grab GIL, but if you find Lock still has not been released the obstruction, even to get permission to perform GIL have to pay out immediately
2.join is waiting for all that serial whole, but only lock to lock modify some shared data, that is part of the serial, in order to ensure data security is a fundamental principle is to allow concurrent become serial, join with the mutex are It can be achieved, no doubt, part of the serial mutex to higher efficiency
Python has a GIL same time to ensure that only one thread to execute, why there is also need to lock?
First, we need to reach a consensus: The purpose of the lock is to protect the shared data, at the same time only one thread to modify the shared data,
Then, we can conclude that: different data protection should be added to different locks.
Finally, it is clear the problem, GIL and Lock are two locks, protection of data is not the same, the former interpreter level (of course, is to protect the interpreter-level data, such as data garbage collection), which is the protection of data of the user's own application development, it is clear that GIL is not responsible for this matter, can only handle user-defined lock that Lock.
Process Analysis: grab all the threads that GIL lock, or grab all threads are executing authority
Thread 1 grab GIL lock, get execute permissions, execution, and then add a handful Lock, has not yet finished, that thread 1 has not been released Lock, thread 2 is likely to grab GIL lock, begin the process of execution Lock has not been found to release the thread 1, thread 2 then enters the blocked execute permissions are taken away, it is possible to get the thread 1 GIL, and then perform normal to release Lock. . . This leads to the effect of serial operation
Since it is serial, then we execute
t1.start()
t1.join
t2.start()
t2.join()
This is the serial execution ah, why add Lock it, Know join t1 is waiting for all of the code executed, the equivalent of all the code lock t1, and Lock code is only part of the operation to lock shared data.
deeper reason
Because the Python interpreter to help you to automatically recover memory regularly, you can understand python interpreter has a separate thread, every once in a while it did wake up from a global poll to see which memory data can be cleared, at this point in the program of your own threads and py interpreter own thread is running concurrently, if you delete a variable thread, the thread garbage collection py interpreter clearing moment in the process of emptying this variable, it may be a other threads that just has not come yet again to clear the memory space and was assigned a result it is possible to assign new data has been deleted, in order to solve similar problems, python interpreter simple and crude, locking that is, when a thread running, other people do not move, this would resolve the above problems, it can be said that the legacy of earlier versions of Python.
example
Locking ago
import time
from threading import Lock,Thread
# Lock 互斥锁加锁前
def func(lock):
global n
temp = n
time.sleep(0.2)
n = temp - 1
if __name__ == '__main__':
n = 10
t_lst = []
lock = Lock()
for i in range(10):
t = Thread(target=func, args=(lock,))
t.start()
t_lst.append(t)
for t in t_lst: t.join()
print(n) #结果很大可能为9
Analysis: a plurality of threads 10 obtained value of n, then perform a final value of 9 n-1 obtained at the same time i.e. 10-1.
After locking
Time Import
from Import Threading Lock, the Thread
# Lock Mutex
DEF FUNC (Lock):
Global n-
lock.acquire ()
TEMP = n-
the time.sleep (0.2)
n-TEMP = -. 1
lock.release ()
IF the __name__ == '__main__':
n-10 =
t_lst = []
Lock Lock = ()
for I in Range (10):
T = the Thread (target = FUNC, args = (Lock,))
t.start ()
t_lst.append (T)
for T in t_lst: t.join ()
Print (n-) the result is 0 #
# when a plurality of threads modify a shared data, the synchronization control is required.
# Thread synchronization ensures secure access to multiple threads compete for resources, the most simple synchronization mechanism is introduced mutex.
#
# Mutex for the introduction of a resource status: locked / unlocked. When you want to change a thread to share data, it first locked,
this time the state of the resource # is "locked", other threads can not be changed; until the thread to release resources, state resources into a "non-locking"
# others thread can lock the resource again.
# Mutex to ensure that only one thread is written, so as to ensure the correctness of the data of multiple threads.
Second, thread deadlocks and lock recursion
Deadlock: The so-called deadlock: refers to the phenomenon of two or more processes or threads in the implementation process, a result of competition for resources caused by waiting for each other, in the absence of external force, they will not be able to promote it. At this time, say the system is in deadlock state or system to produce a deadlock, which is always in the process of waiting for another process called the deadlock. In one thread, we use part of the code lock lock.acquire () when the lock has not been released, calls another method, a method in turn uses the lock, so if you use the same common lock is prone to deadlock .
example
from threading import Thread,Lock
import time
mutexA=Lock()
mutexB=Lock()
class MyThread(Thread):
def run(self):
self.func1()
self.func2()
def func1(self):
mutexA.acquire()
print('\033[41m%s 拿到A锁\033[0m' %self.name)
mutexB.acquire()
print('\033[42m%s 拿到B锁\033[0m' %self.name)
mutexB.release()
mutexA.release()
def func2(self):
mutexB.acquire()
print('\033[43m%s 拿到B锁\033[0m' %self.name)
time.sleep(2)
mutexA.acquire()
print ( '\ 033 [44m% s get A lock \ 033 [0m' the self.name%)
mutexA.release ()
mutexB.release ()
IF the __name__ == '__main__':
for I in Range (10) :
T = the MyThread ()
t.start ()
'' '
Thread-1 to get A lock
Thread-1 B get locked
Thread-1 B get lock
Thread-2 A lock get
then stuck deadlock the
'' '
Solution:
Recursive locks, in order to support Python, in the same thread multiple requests for the same resource, python provides a "reentrant lock (recursive lock)": threading.RLock. Internal RLock maintains a Lock and a counter variable, counter records the number of times acquire, so that resources can be repeatedly acquire. Acquire a thread until all have been release, other threads to get resources.
Examples of surface if used in place of RLock Lock, the deadlock does not occur:
mutexA = mutexB = threading.RLock () # get a thread lock, counter plus 1, in this thread ran into a locked case, the counter continues to add 1, during which all the other threads can only wait, wait for the thread release all locks that counter is decremented until 0
example
Threading the Thread Import from, RLOCK, Lock
Import Time
# mutexA = Lock ()
# mutexB = Lock ()
mutexA = mutexB = RLOCK ()
class MyThread (the Thread):
DEF RUN (Self):
self.func1 ()
Self .func2 ()
DEF func1 (Self):
mutexA.acquire () counts count # lock. 1 =
Print ( '\ 033 [% S 41M get a lock \ 033 [0m' the self.name%)
mutexB.acquire ( ) lock count count # +. 1. 1 = 2 =
Print ( '\ 033 [42m% S B get locked \ 033 [0m' the self.name%)
mutexB.release () # lock count count = 2-1 = . 1
mutexA.release () # 1-1 of the lock = 0 count = cOUNT
DEF func2 (Self):
mutexB.acquire () # lock count. 1 = COUNT
Print ( '\ 033 [43m% S B get locked \ 033 [0m' the self.name%)
the time.sleep (2)
mutexA.acquire () # lock = +. 1. 1 count = cOUNT 2
Print ( '\ 033 [% S 44M get a lock \ 033 [0m' the self.name%)
mutexA.release () # lock. 1 = 2-1 = count cOUNT
mutexB. release () lock count count # 1-1 of = 0 =
IF the __name__ == '__main__':
for I in Range (10):
T = the MyThread ()
t.start ()
Third, the signal amount Semaphore
Lock and the like, internal semaphore object maintains a down counter, every time we call acquire () built-in counter -1, call release (built-in counter +1). When the count is found to acquire method to block the request of the thread 0, the other threads until after the Release semaphore, the count is greater than 0, the recovery blocked thread.
example
Threading the Thread Import from, Semaphore
Import Time
DEF FUNC (SEM, I):
sem.acquire ()
Print (. "is executed once, acquire a semaphore = {} the _value" the format (sem._value))
Print ( I)
the time.sleep (. 1)
Print ( "release a semaphore")
sem.release ()
SEM = semaphore (. 5)
for I in Range (20 is):
the Thread (target = FUNC, args = (SEM, I )). start ()
Example 2
Threading Import, Time
class myThread (of the threading.Thread):
DEF run (Self): # After starting, run method
if semaphore.acquire (): # Adding a lock to be put to a plurality (corresponding to the lock 5 , five keys at the same time there are five threads)
Print (self.name)
the time.sleep (5)
semaphore.release ()
IF __name__ == "__main__":
semaphore = threading.Semaphore (5) at the same time to have a # several threads into (5 is set to go in the first five threads), similar to parking with a few cars can park
thrs = [] # empty list
for i in range (100): # 100 threads
thrs.append ( myThread ()) # plus thread object
for t in THRS:
t.start () # respectively start
Output
. 1-the Thread the Thread-2 the Thread. 3- the Thread. 4- the Thread # 5. 5-threads out of the Thread. 8- the Thread. 6- the Thread. 9- the Thread. 7- the Thread-10 every 3 seconds # 5 then print out partially omitted .......
Pool and process are completely different concepts, process pool Pool (4), the maximum can only produce 4 process, and the process from start to finish just four, no new, and semaphores is to generate a bunch of threads / process
Reference material
[1]https://blog.csdn.net/u013008795/article/details/91357383
[2]