Process lock
lock.acquire () lock
lock.release () to release the lock
Process lock: the lock code becomes serial
join: the entire code becomes serial
To ensure data security, serial sacrifice efficiency
Queue (Queue)
Queue: pipe + Lock Queue (not suitable for large file transfer, typically a number of message transfer)
ipc mechanism: Process Communications
Pipeline: pipe-based shared memory space
具体用法:
from multiprocessing import Process,Queue
q = Queue() # 空队列，可放数字控制队列大小，
q.put('zhang')
q.put('jian') # 放入，超过队列大小是会阻塞
# q.put(对象，block=True,timeout=) 第一个参数为放入的数据，第二个参数默认为True,表示如果放入的时候满了等待，timeout控制阻塞时间，单位秒，超过时间没有执行会报错
q.get()
q.get()
q.get() # 取值，没有的时候回等待，会阻塞
q.get(对象，block=True,timeout=) 第一参数数据，第二参数表示没有东西会等待，timeout等待时间，超过没有去处东西会报错
block = False,当放入是队列满了，或者去时队列空了直接报错
q.get_nowait() 表示不等待，相当于q.get(block = False,)
q.put_nowait() 表示不等待，相当于q.put(block = False,)
Producer consumer model
Producer: Production Data
Consumers: data processing
Producer --- --- consumer queue
Producers can not stop production reached their maximum production efficiency
Producer consumer model to improve the efficiency of production and producers
第一代
from multiprocessing import Process, Queue
def producer(q,name,food):
for i in range(10):
print(f'{name}生产了{food}{i}')
res = f'{food}{i}'
q.put(res)
q.put(None)
def consumer(q,name):
while True:
res = q.get(timeout=5)
if res is None:
break
print(f'{name}吃了{res}')
if __name__ == '__main__':
q = Queue()
p1 = Process(target=producer, args=(q, 'zjh', '包子'))
c1 = Process(target=consumer,args=(q,'zj'))
p1.start()
c1.start()
q.put(None)
第二代
from multiprocessing import Process,Queue
import time,random
def producer(q,name,food):
for i in range(3):
print(f'{name}生产了{food}{i}')
time.sleep(random.randint(1,3))
res = f'{food}{i}'
q.put(res)
def consumer(q,name):
while True:
res = q.get(timeout = 5)
if res is None:
break
time.sleep(random.randint(1,3))
print(f'{name}吃了{res}')
if __name__ == '__main__':
q = Queue()
p1 = Process(target=producer(q,'lqz','baozi'))
p2 = Process(target=producer(q,'nick','xie'))
p3 = Process(target=producer(q,'rockey','dabing'))
c1 = Process(target=consumer(q,'qxp'))
c2 = Process(target=consumer(q,'dgq'))
p1.start()
p2.start()
p3.start()
c1.start()
c2.start()
p1.join()
p2.join()
p3.join()
q.put(None)
q.put(None)
第三代
from multiprocessing import Process,Queue,JoinableQueue
import time,random
def producer(q,name,food):
for i in range(3):
print(f'{name}生产了{food}{i}')
time.sleep(random.randint(1,3))
res = f'{food}{i}'
q.put(res)
def consumer(q,name):
while True:
res = q.get()
time.sleep(random.randint(1,3))
print(f'{name}吃了{res}')
q.task_done()
if __name__ == '__main__':
q = JoinableQueue()
p1 = Process(target=producer, args=(q, 'rocky', '包子'))
p2 = Process(target=producer, args=(q, 'mac', '韭菜'))
p3 = Process(target=producer, args=(q, 'nick', '蒜泥'))
c1 = Process(target=consumer, args=(q, '成哥'))
c2 = Process(target=consumer, args=(q, '浩南哥'))
p1.start()
p2.start()
p3.start()
c1.daemon = True
c2.daemon = True
c1.start()
c2.start()
p1.join()
p2.join()
p3.join()
q.join()
JoinableQueue usage
from multiprocessing import Process,Queue,JoinableQueue
q = JoinableQueue()
q.put('zhang') # 放入一次任务
q.put('jian')
print(q.get())
q.task_done() # 完成了一次任务
print(q.get())
q.task_done()
q.join() # 计数器不为0时，阻塞等待计数器为0
Acquaintance thread
Thread: Running procedural code, cpu smallest unit of execution
Traditional operating systems, each process has an address space, but by default there is a thread of control
In the factory, each plant has a house, each shop will have a default line
Operating System --- factory
Process --- workshop (plus thread resource unit)
Thread --- (pipeline)
--- CPU (power supply)
= Thread run code execution
Process: collection, resource units
Processes running thread = resources +