Coroutine
Coroutine: Concurrency under single thread, also known as microthread, fiber. The English name is Coroutine.
One sentence explains what a thread is: a coroutine is a lightweight thread in a user mode, that is, a coroutine is controlled and scheduled by the user program itself.
- Python threads belong to the kernel level, that is, the operating system controls the scheduling (such as a single thread encounters io or the execution time is too long, it will be forced to hand over the CPU execution authority, switch other threads to run)
- The coroutine is started in a single thread. Once io is encountered, the switching will be controlled from the application level (not the operating system) to improve efficiency (!!! The switching of non-io operations has nothing to do with efficiency)
Compared with the operating system control thread switch, the user controls the switch of the coroutine in a single thread, the
advantages are as follows:
- The switching overhead of the coroutine is smaller, which belongs to the program-level switching, and the operating system is completely unaware of it, so it is more lightweight
- The effect of concurrency can be achieved in a single thread, and the cpu can be used to the maximum
The disadvantages are as follows:
- The essence of the coroutine is that it is single-threaded and cannot use multiple cores. It can be that one program opens multiple processes, each process opens multiple threads, and each thread opens the coroutine
- The coroutine refers to a single thread, so once the coroutine is blocked, the entire thread will be blocked
Summarize the characteristics of the coroutine:
Concurrency must be implemented in only one single thread. There is
no need to lock the shared data. The
user program saves the context stack of multiple control flows.
Additional: A coroutine automatically switches to other coroutines when encountering an IO operation (how to To achieve detection IO, yield and greenlet cannot be achieved, so the gevent module (select mechanism) is used)
signal
信号量代码演示（信号量指的是同一时间有多把锁并发运行）
Semaphore 导入信号量模块
from threading import Thread, Semaphore, current_thread
import time
def func():
sm.acquire() # 抢锁
print("%s 干饭 " % current_thread().getName())
time.sleep(3)
sm.release() # 释放锁
if __name__ == '__main__':
sm = Semaphore(5) # 一共有5把锁
for i in range(21):
t = Thread(target=func)
t.start()
Deadlock phenomenon and recursive lock
There will be no error prompt when deadlock phenomenon occurs, and the lock sleeve lock will appear deadlock phenomenon
模拟死锁代码演示
from threading import Thread, Lock
mutexA = Lock()
mutexB = Lock()
class Mythread(Thread):
def __init__(self, name):
super().__init__()
self.name = name
def run(self):
self.f1()
self.f2()
def f1(self):
mutexA.acquire()
print("%s 抢到了A锁" % self.name) # 线程1先抢到了a锁，继续抢了b锁。这时候其他三个线程都并发抢a锁，
# 线程1释放了b锁，再释放了a锁，其他三个并发抢a锁的同时，线程1又抢了b锁并且睡了0.1秒
mutexB.acquire() # 这个时候b锁被线程2抢到了，a锁在线程1的手里。线程1和线程2互相锁死。线程3和线程4还在等着抢a锁
print("%s 抢到了B锁" % self.name)
mutexB.release()
mutexA.release()
def f2(self):
mutexB.acquire()
print("%s 抢到了B锁" % self.name)
time.sleep(0.1)
mutexA.acquire()
print("%s 抢到了A锁" % self.name)
mutexA.release()
mutexB.release()
if __name__ == '__main__':
t1 = Mythread("线程1")
t2 = Mythread("线程2")
t3 = Mythread("线程3")
t4 = Mythread("线程4")
t1.start()
t2.start()
t3.start()
t4.start()
print("主线程")
死锁现象发生打印的结果：
线程1 抢到了A锁
线程1 抢到了B锁
线程1 抢到了B锁
线程2 抢到了A锁
主线程
Recursive lock
递归锁同一把锁可以连续acquire，以计数的方式来抢锁和释放锁
from threading import Thread, RLock
mutexA = mutexB = RLock() # 递归锁，一个进程可以连续抢锁，抢一次锁技计数1次，第一次计数1，第二次计数2。。。。
# 其他的线程如果发现递归锁的计数不为0就直接堵塞在原地，等这把锁计数为0的时候才能抢锁
class Mythread(Thread):
def __init__(self, name):
super().__init__()
self.name = name
def run(self):
self.f1()
self.f2()
def f1(self):
mutexA.acquire()
print("%s 抢到了A锁" % self.name)
mutexB.acquire()
print("%s 抢到了B锁" % self.name)
mutexB.release()
mutexA.release()
def f2(self):
mutexB.acquire()
print("%s 抢到了B锁" % self.name)
time.sleep(0.1)
mutexA.acquire()
print("%s 抢到了A锁" % self.name)
mutexA.release()
mutexB.release()
if __name__ == '__main__':
t1 = Mythread("线程1")
t2 = Mythread("线程2")
t3 = Mythread("线程3")
t4 = Mythread("线程4")
t1.start()
t2.start()
t3.start()
t4.start()
print("主线程")
Event event When
one thread finishes, another thread starts to run. It does not involve the transfer of data. It sends a signal (data) to remind another thread.
代码演示
from threading import Event, Thread, current_thread
e = Event() # 默认全局变量为False
def f1():
print("%s 运行" % current_thread().name) # current_thread().name获取线程名字
time.sleep(3)
e.set() # 把全局变量变成了True
# e.clear() # 把全局变量变成False
# e.is_set() # 判断有没有设置过，设置过就是True，没有设置过就是False
def f2():
e.wait() # 在原地等待，等到全局变量变为True。e.wait(n)可以设置等待时间，超时如果全局变量还没有变成True那么会自动执行代码
print("%s 运行" % current_thread().name)
if __name__ == '__main__':
t1 = Thread(target=f1)
t2 = Thread(target=f2)
t1.start()
t2.start()
Case study
from threading import Event, Thread, current_thread
import time
import random
e = Event() # e = Ture
def task1():
while True:
e.clear() # e = False
print("红灯亮")
time.sleep(2)
e.set() # e = True
print("绿灯亮")
time.sleep(3)
# task1：2秒亮一次红灯，三秒亮一次绿灯。不停的在循环
def task2():
while True:
if e.is_set(): # 判断e有没有被设置过，如果设置过就是True，没有设置过就是False
print("%s 走你" % current_thread().name)
break # 线程只有进入“走你”才会结束循环
else:
print("%s 等灯" % current_thread().name)
e.wait() # 等待全局变量变为True
if __name__ == '__main__':
Thread(target=task1).start()
while True:
time.sleep(random.randint(1, 5)) # 每1-5秒之间产生循环
Thread(target=task2).start()
Timer and thread queue
Timer
from threading import Timer
def hello(x):
print("hello,world", x)
t = Timer(0.2, hello, args=(10,)) # Timer(n,"xx")可以在定时器第一个逗号前面写上时间，表示过多长时间运行代码
t.start()
Thread queue (there are queues, stacks, priority queues)
import queue
队列(先进先出)
q = queue.Queue(3)
q.put(111)
q.put("aaa")
q.put((1, 2, 3))
print(q.get())
print(q.get())
print(q.get())
堆栈（后进先出）
q = queue.LifoQueue(3)
q.put(111)
q.put("aaa")
q.put((1, 2, 3))
print(q.get())
print(q.get())
print(q.get())
优先级队列（元组，第一个整型数字是优先级排序）
q = queue.PriorityQueue(3)
q.put((2, (111)))
q.put((3, "aaa"))
q.put((1, (1, 2, 3)))
print(q.get())
print(q.get())
print(q.get())
GIL interpreter lock (the lock that comes with the cpython interpreter)
In the python interpreter, multiple threads opened in the same process can only be executed by one thread at the same time, and cannot take advantage of multi-core.
Python threads need to get the gil of the interpreter The lock can run the code, which is equivalent to an interpreter execution permission. With the execution permission, the operating system will allocate the cpu to the python code. Once the cpu is transferred, the gil lock will directly release the
gil lock mainly to protect the python interpreter The data security of memory management is not able to protect the data security of the python code you wrote yourself. The
gil lock can make threads concurrent, but cannot make threads parallel (python can only use one cpu to run a process, if you want to use multiple cpus at the same time To improve computing efficiency, multiple processes need to be opened)
from threading import Thread, Lock
import time
mutex = Lock()
n = 100
def task():
global n
mutex.acquire() # 互斥锁
temp = n
time.sleep(0.1)
n = temp - 1
mutex.release()
if __name__ == '__main__':
l = []
for i in range(100):
t = Thread(target=task)
l.append(t)
t.start()
for obj in l:
obj.join()
print(n)
Multi-threaded performance test
Pure computing behavior can use multiple processes, using multi-core advantages to improve operating efficiency.
Pure io behavior can open multiple threads in one process, because the multi-core advantages of io behavior will not be reflected.
Open multiple processes. Open up a new memory space, copy the program of the parent process to the child process, and start the process. Once the io behavior occurs, the operating system will automatically transfer the cpu away, and the operating system will transfer the cup again after the io is over.
Open multiple threads, a cpu switch back and forth, because the cost of opening threads is far less than the cost of opening processes, so the efficiency will be higher than that of multiple processes.
代码演示
from multiprocessing import Process
from threading import Thread
import os, time
def work():
# res = 0
# for i in range(100000000):
# res *= i
time.sleep(5)
if __name__ == '__main__':
l = []
# print(os.cpu_count()) # 本机为8核
start = time.time()
for i in range(8):
p = Process(target=work) # 纯计算 多进程花费的时间12.8s |纯io行为多进程运行时间为5.335s
# p = Thread(target=work) # 纯计算 多线程运行的时间46.45s |纯io行为多线程运行时间为5.003s
l.append(p)
p.start()
for p in l:
p.join()
stop = time.time()
print("run time is %s" % (stop - start))