[Python advanced] multi-process programming in Python

Process is a system concept, which also corresponds to the underlying hardware concept of the CPU.

First of all, you can check how many threads your CPU can support:

from multiprocessing import cpu_count
print(cpu_count())

When enabling multiple processes, it is best not to approach or exceed the number of hardware processes of the CPU.

from multiprocessing import Process
from time import sleep


def run(n):
    sleep(n)
    print(n)


if __name__ == '__main__':
    p1 = Process(target=run, args=(1,))
    p2 = Process(target=run, args=(2,))
    p3 = Process(target=run, args=(3,))
    p4 = Process(target=run, args=(4,))

    p1.start()
    p2.start()
    p3.start()
    p4.start()

Multi-process startup is very convenient, just specify the name of the target function and input parameters for startup (note that there is a comma after it).

Multi-processes can achieve the effect of accelerating programs to a certain extent through the clever use of CPU multi-cores and parallel computing. See experiment 2:

from multiprocessing import Process
from time import time


def run():
    start = time()
    res = 0
    for i in range(10000):
        for j in range(10000):
            res += 1
            res -= 1
    print('time cost:', time() - start)


if __name__ == '__main__':
    t0 = time()
    run()
    p1 = Process(target=run, args=())
    p2 = Process(target=run, args=())
    p3 = Process(target=run, args=())
    p4 = Process(target=run, args=())

    p1.start()
    p2.start()
    p3.start()
    p4.start()
    p1.join()
    p2.join()
    p3.join()
    p4.join()

    print('all time cost:', time() - t0)

This experiment compares the running time of multi-processes not enabled and multi-processes enabled. The output is:

time cost: 3.343838691711426
time cost: 3.4023547172546387
time cost: 3.4059526920318604
time cost: 3.4068567752838135
time cost: 3.409113883972168
all time cost: 6.755127668380737

It can be seen that it takes almost the same time to complete 4 calculations with 4 processes and to complete 1 calculation without multiple processes .