1. zombie process and orphaned
Unix-based environment (linux, macOS)
After the main process needs to wait for the end of the child process, the main process until the end
Main operating state detecting process time of the child process, the child process ends when, within a period of time, the child process is recovered.
Why is not the primary process child process immediately after the end of its recycle it?
- The main process and sub-process is asynchronous relationship, the primary process can not capture the child process immediately when to stop
- If the child immediately after the end of the process to release resources in memory, the main process there is no way to detect the state of the child process.
Unix provides for a mechanism to the above question
- After the end of all the sub-processes, will be freed immediately operational links and most of the data memory files, but will retain some elements: process ID, end time, running, waiting for the main course detection and recovery.
Zombie process: in the life cycle of the parent process, after the end of all the sub-processes, the main process before being recycled, will enter zombie state
Zombie process whether the harm ???
- If the parent does not zombie process for recycling (wait / waitpid), generate a lot of zombie process, which would take up the contents, taking the process pid No.
Orphaned
- For some reason the parent process is over, but the child process is still running, so that the process becomes orphaned when parent process is over, took over after the init process became orphans parent process, all orphans the process will be init process (process 1) recycling.
How to solve the zombie process ???
- The parent process produces a large number of child processes, but not recovered, which would form a large number of zombie process, solution is to directly kill the parent process, all the zombies orphaned process for recycling by the init.
2. mutex (to ensure data security, locked himself prone to deadlock.)
(Hereinafter collectively referred to as the process is a concept, can refer to processes or threads)
互斥锁: 指散布在不同进程之间的若干程序片断，当某个进程运行其中一个程序片段时，其它进程就不能运行它们之中的任一程序片段，只能等到该进程运行完这个程序片段后才可以运行的一种类似于"锁"的机制Mutex role: that of ensuring the child serial, but also to ensure the safety of the execution order of the randomness of the child, as well as data
Suppose there are three co-workers, while for a printer to print content
# 三个进程模拟三个同事,输出平台模拟打印机
# 版本一
from multiprocessing import Process
from multiprocessing import Lock
import time
import random
import os
def task():
print(f"{os.getpid()}开始打印了")
time.sleep(random.randint(1, 3))
print(f"{os.getpid()}打印结束了")
if __name__ == '__main__':
p1 = Process(target=task)
p2 = Process(target=task)
p3 = Process(target=task)
p1.start()
p2.start()
p3.start()
# 结果是打印混乱
# 现在是所有的进程都并发的抢占打印机,
# 并发是以效率优先的,但是目前我们的需求: 顺序优先.
# 多个进程共强一个资源时, 要保证顺序优先: 串行,一个一个来.# 版本二
from multiprocessing import Process
from multiprocessing import Lock
import time
import random
import os
def task():
print(f"{os.getpid()}开始打印了")
time.sleep(random.randint(1, 3))
print(f"{os.getpid()}打印结束了")
if __name__ == '__main__':
p1 = Process(target=task)
p2 = Process(target=task)
p3 = Process(target=task)
p1.start()
p1.join()
p2.start()
p1.join()
p3.start()
p3.join()
# 我们利用join 解决串行的问题,保证了顺序优先,但是这个谁先谁后是固定的.
# 这样不合理. 你在争抢同一个资源的时候,应该是先到先得,保证公平.# 版本三
from multiprocessing import Process
from multiprocessing import Lock
import time
import random
import os
def task(p,lock):
'''
一把锁不能连续锁两次
lock.acquire()
lock.acquire()
lock.release()
lock.release()
'''
lock.acquire()
print(f"{p}开始打印了")
time.sleep(random.randint(1, 3))
print(f"{p}打印结束了")
lock.release()
if __name__ == '__main__':
lock = Lock()
p1 = Process(target=task,args=("p1",lock))
p2 = Process(target=task,args=("p2",lock))
p3 = Process(target=task,args=("p3",lock))
p1.start()
p2.start()
p3.start()
The difference between lock and join in.
Common: it can become complicated by the serial to ensure order.
Different points: join artificial setting procedure, let Lock scramble sequence to ensure fairness.
3. Communication between processes
Level process in memory is isolated, but on the disk file.
1. file-based communication.
# 抢票系统.
# 1. 先可以查票.查询余票数. 并发
# 2. 进行购买,向服务端发送请求,服务端接收请求,在后端将票数-1,返回到前端. 串行.
from multiprocessing import Process
from multiprocessing import Lock
import json
import time
import random
import os
def search():
time.sleep(random.randint(1,3)) # 模仿网络延迟(查询环节)
with open("db",encoding="utf-8") as f:
dic = json.load(f)
print(f"{os.getpid()} 查看了票数,剩余{dic['count']}")
def paid():
with open("db","r",encoding="utf-8") as f:
dic = json.load(f)
if dic['count'] > 0:
dic['count'] -= 1
time.sleep(random.randint(1,3)) # 模拟网络延迟(购买环节)
with open("db","w",encoding="utf-8") as f:
json.dump(dic,f)
print(f"{os.getpid()} 购买成功!")
def task(lock):
search()
paid()
if __name__ == '__main__':
mutex = Lock()
for i in range(10):
p = Process(target=task,args=(mutex,))
p.start()
# 当多个进程共抢一个数据时,如果要保证数据的安全,必须要串行.
# 要想让购买环节进行串行,我们必须要加锁处理.
# 抢票系统.
# 1. 先可以查票.查询余票数. 并发
# 2. 进行购买,向服务端发送请求,服务端接收请求,在后端将票数-1,返回到前端. 串行.
from multiprocessing import Process
from multiprocessing import Lock
import json
import time
import random
import os
def search():
time.sleep(random.randint(1,3)) # 模仿网络延迟(查询环节)
with open("db",encoding="utf-8") as f:
dic = json.load(f)
print(f"{os.getpid()} 查看了票数,剩余{dic['count']}")
def paid():
with open("db","r",encoding="utf-8") as f:
dic = json.load(f)
if dic['count'] > 0:
dic['count'] -= 1
time.sleep(random.randint(1,3)) # 模拟网络延迟(购买环节)
with open("db","w",encoding="utf-8") as f:
json.dump(dic,f)
print(f"{os.getpid()} 购买成功!")
def task(lock):
search()
lock.acquire()
paid()
lock.release()
if __name__ == '__main__':
mutex = Lock()
for i in range(10):
p = Process(target=task,args=(mutex,))
p.start()
# 当很多进程共强一个资源(数据)时, 你要保证顺序(数据的安全),一定要串行.
# 互斥锁: 可以公平性的保证顺序以及数据的安全.
# 基于文件的进程之间的通信:
# 效率低.
# 自己加锁麻烦而且很容易出现死锁.2. queue-based communication. (Real will be used)
Isolated from each other processes, to achieve inter-process communication (IPC) (inter-Process Communication), multiprocessing module supports two forms: the queue and the pipeline, these two methods are the use of the message passing
Create a class queue (that is, to the bottom of the pipe and binding manner) :
Queue([maxsize]):创建共享的进程队列，Queue是多进程安全的队列，可以使用Queue实现多进程之间的数据传递。Parameter Description
maxsize是队列中允许最大项数,省略则无大小限制Methods Introduction:
Main methods:
1 q.put方法用以插入数据到队列中，put方法还有两个可选参数：blocked和timeout。如果blocked为True（默认值），并且timeout为正值，该方法会阻塞timeout指定的时间，直到该队列有剩余的空间。如果超时，会抛出Queue.Full异常。如果blocked为False，但该Queue已满，会立即抛出Queue.Full异常。
2 q.get方法可以从队列读取并且删除一个元素。同样，get方法有两个可选参数：blocked和timeout。如果blocked为True（默认值），并且timeout为正值，那么在等待时间内没有取到任何元素，会抛出Queue.Empty异常。如果blocked为False，有两种情况存在，如果Queue有一个值可用，则立即返回该值，否则，如果队列为空，则立即抛出Queue.Empty异常.
3
4 q.get_nowait():同q.get(False)
5 q.put_nowait():同q.put(False)
6
7 q.empty():调用此方法时q为空则返回True，该结果不可靠，比如在返回True的过程中，如果队列中又加入了项目。
8 q.full()：调用此方法时q已满则返回True，该结果不可靠，比如在返回True的过程中，如果队列中的项目被取走。
9 q.qsize():返回队列中目前项目的正确数量，结果也不可靠，理由同q.empty()和q.full()一样Other methods (to know):
1 q.cancel_join_thread():不会在进程退出时自动连接后台线程。可以防止join_thread()方法阻塞
2 q.close():关闭队列，防止队列中加入更多数据。调用此方法，后台线程将继续写入那些已经入队列但尚未写入的数据，但将在此方法完成时马上关闭。如果q被垃圾收集，将调用此方法。关闭队列不会在队列使用者中产生任何类型的数据结束信号或异常。例如，如果某个使用者正在被阻塞在get()操作上，关闭生产者中的队列不会导致get()方法返回错误。
3 q.join_thread()：连接队列的后台线程。此方法用于在调用q.close()方法之后，等待所有队列项被消耗。默认情况下，此方法由不是q的原始创建者的所有进程调用。调用q.cancel_join_thread方法可以禁止这种行为
application:
from multiprocessing import Process
import time
q = Queue(3)
q.put(3)
q.put(3)
q.put(3)
print(q.full()) # 满了
print(q.get())
print(q.get())
print(q.get())
print(q.empty()) # 空了
q.put(3)
q.put(3)
q.put(3)
q.put(3) # 在这阻塞住了,下面的方法不会执行
q.get(3)
3. Based on the communication pipe
Introduction:
#创建管道的类：
Pipe([duplex]):在进程之间创建一条管道，并返回元组（conn1,conn2）,其中conn1，conn2表示管道两端的连接对象，强调一点：必须在产生Process对象之前产生管道
#参数介绍：
dumplex:默认管道是全双工的，如果将duplex射成False，conn1只能用于接收，conn2只能用于发送。
#主要方法：
conn1.recv():接收conn2.send(obj)发送的对象。如果没有消息可接收，recv方法会一直阻塞。如果连接的另外一端已经关闭，那么recv方法会抛出EOFError。
conn1.send(obj):通过连接发送对象。obj是与序列化兼容的任意对象
#其他方法：
conn1.close():关闭连接。如果conn1被垃圾回收，将自动调用此方法
conn1.fileno():返回连接使用的整数文件描述符
conn1.poll([timeout]):如果连接上的数据可用，返回True。timeout指定等待的最长时限。如果省略此参数，方法将立即返回结果。如果将timeout射成None，操作将无限期地等待数据到达。
conn1.recv_bytes([maxlength]):接收c.send_bytes()方法发送的一条完整的字节消息。maxlength指定要接收的最大字节数。如果进入的消息，超过了这个最大值，将引发IOError异常，并且在连接上无法进行进一步读取。如果连接的另外一端已经关闭，再也不存在任何数据，将引发EOFError异常。
conn.send_bytes(buffer [, offset [, size]])：通过连接发送字节数据缓冲区，buffer是支持缓冲区接口的任意对象，offset是缓冲区中的字节偏移量，而size是要发送字节数。结果数据以单条消息的形式发出，然后调用c.recv_bytes()函数进行接收
conn1.recv_bytes_into(buffer [, offset]):接收一条完整的字节消息，并把它保存在buffer对象中，该对象支持可写入的缓冲区接口（即bytearray对象或类似的对象）。offset指定缓冲区中放置消息处的字节位移。返回值是收到的字节数。如果消息长度大于可用的缓冲区空间，将引发BufferTooShort异常。
from multiprocessing import Process,Pipe
import time,os
def consumer(p,name):
left,right=p
left.close()
while True:
try:
baozi=right.recv()
print('%s 收到包子:%s' %(name,baozi))
except EOFError:
right.close()
break
def producer(seq,p):
left,right=p
right.close()
for i in seq:
left.send(i)
# time.sleep(1)
else:
left.close()
if __name__ == '__main__':
left,right=Pipe()
c1=Process(target=consumer,args=((left,right),'c1'))
c1.start()
seq=(i for i in range(10))
producer(seq,(left,right))
right.close()
left.close()
c1.join()
print('主进程')
基于管道实现进程间通信（与队列的方式是类似的，队列就是管道加锁实现的）
However, the pipeline is problematic, the pipeline will cause unsafe data, the official explanation given is that the pipeline may cause data corruption.