笔记｜Python 的 multiprocess.Pool 的报错在主进程打印方法

如果直接使用 multiprocess.Pool 多进程执行任务函数，若任务函数中抛出异常，则主进程中无法获得异常信息。例如：

import multiprocessing


def task():
    print("test print")
    raise Exception()


if __name__ == "__main__":
    pool = multiprocessing.Pool()
    pool.apply_async(task)
    pool.close()
    pool.join()
    print("finish")

运行结果如下：

test print
finish

Process finished with exit code 0

1. 在主进程中获取 apply_async() 方法返回的 ApplyResult 实例，并调用其 get() 获取异常

我们可以在 task 函数中嵌套异常捕获逻辑，但是无法捕获 task 函数的实参和形参不匹配的问题。因此，为了在主进程中获取各种异常信息，我们可以将上述逻辑改写如下：

import multiprocessing


def task():
    print("test print")
    raise Exception()


if __name__ == "__main__":
    pool = multiprocessing.Pool()
    t = pool.apply_async(task)
    t.get()
    pool.close()
    pool.join()

运行结果如下：

test print
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "test.py", line 6, in task
    raise Exception()
Exception
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "test.py", line 12, in <module>
    t.get()
  File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 771, in get
    raise self._value
Exception

Process finished with exit code 1

这种处理导致主进程抛出异常被停止。

2. 在主进程捕获异常并打印信息

于是在主进程捕获异常，并打印异常信息即可：

import multiprocessing
import traceback


def task():
    print("test print")
    raise Exception()


if __name__ == "__main__":
    pool = multiprocessing.Pool()
    t = pool.apply_async(task)
    try:
        t.get()
    except Exception as e:
        traceback.print_exc()
    pool.close()
    pool.join()
    print("finish")

运行结果如下：

test print
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "E:\brick\test\多进程报错样例.py", line 7, in task
    raise Exception()
Exception
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "E:\brick\test\多进程报错样例.py", line 14, in <module>
    t.get()
  File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 771, in get
    raise self._value
Exception
finish