python并发问题

在python写进程并发,进程之间通信定义队列时,一般会写成:

queue = multiprocessing.Queue()
    queue = multiprocessing.Queue()
    from concurrent.futures import ProcessPoolExecutor
    from tool.utils import random_k
    import random
    futures = []
    strs = ['I am',' a ','good student','and smart','boyy.','Believe?']
    with ProcessPoolExecutor(max_workers=4) as executor:
        # If these processs use same function, use [executor.map]
        for i in range(4):
            future = executor.submit(commonucate, 
            queue,random_k(random.randint(0,len(strs)-1),strs))  # executor.submit(func, 
                                                               #     *args, **kwargs)
            futures.append(future)

这时候程序会报错,如下图,显示Queue对象只能使用继承(inheritance)的方式共享。这是因为Queue本身基于unix的Pipe对象实现,而Pipe对象的共享需要通过继承。

因此,在一个典型的应用实现模型当中,应该是父进程创建Queue,然后创建子进程共享该Queue,由父进程和子进程分别读写。另一种实现方式是父进程创建Queue,创建多个子进程,有的子进程读Queue,有的子进程写Queue

进程间共享队列正确的写法应该是:

manager = multiprocessing.Manager()
queue = manager.Queue()

参考:

http://stackoverflow.com/questions/3217002/how-do-you-pass-a-queue-reference-to-a-function-managed-by-pool-map-async

http://blog.ftofficer.com/2009/11/using-python-multiprocessing-1-process-model/

http://blog.ftofficer.com/2009/12/python-multiprocessing-2-object-sharing-across-process/

http://blog.ftofficer.com/2009/12/python-multiprocessing-3-about-queue/

猜你喜欢

转载自blog.csdn.net/Poo_Chai/article/details/88878219