Python multiprocessing multi-process

In multiporcessing, create a process by creating a new Process object, and then call it with the start() method. Process is similar to threading.Thread. Here is a simple example:

from multiprocessing import Process

def f(name):
    print('hello' ,name)

if __name__ == '__main__':
    p = Process(target=f, args('bob',))
    p.start()
    p.join()

Tails

from multiprocessing import Process, Queue

def f(q):
    q.put([42, None, 'hello'])

if __name__ == '__main__':
    q = Queue()
    p = Process(target=f, args=(q,))
    p.start()
    print(q.get())    # prints "[42, None, 'hello']"
    p.join()

Queues are thread and process safe.

Pipes

Pipe can be unidirectional (half-duplex) or bidirectional (duplex). We create unidirectional pipes via mutiprocessing.Pipe(duplex=False) (default is bidirectional). A process inputs an object from one end of the PIPE, and is then received by the process at the other end of the PIPE. A unidirectional pipe only allows input from a process at one end of the pipe, while a bidirectional pipe allows input from both ends. 

import multiprocessing as mul


def proc1(pipe):
    pipe.send('hello')
    print('proc1 rec:', pipe.recv())


def proc2(pipe):
    print('proc2 rec:', pipe.recv())
    pipe.send('hello, too')


# Build a pipe
pipe = mul.Pipe()
if __name__ == '__main__':
    # Pass an end of the pipe to process 1
    p1 = mul.Process(target=proc1, args=(pipe[0],))
    # Pass the other end of the pipe to process 2
    p2 = mul.Process(target=proc2, args=(pipe[1],))
    p1.start()
    p2.start()
    p1.join()
    p2.join()

proc2 rec: hello
proc1 rec: hello, too

The Pipe() function returns a pair of objects connected together with a pipe, which is bidirectional by default. For example:

from multiprocessing import Process, Pipe

def f(conn):
    conn.send([42, None, 'hello'])
    conn.close()

if __name__ == '__main__':
    parent_conn, child_conn = Pipe()
    p = Process(target=f, args=(child_conn,))
    p.start()
    print(parent_conn.recv())   # prints "[42, None, 'hello']"
    p.join()

Use Pipe() to represent the two ends of the pipeline, which is used to connect two objects. Each connected object has send() and recv() methods. Note that if two processes (or threads) try to read from or write to one end of the pipe at the same time, there will be problems with the data in the pipe. Of course, there is no problem with different ports using different pipes at the same time.

reference:

https://www.cnblogs.com/gmhappy/p/11863960.html

https://hiteshmishra708.medium.com/multiprocessing-in-python-c6735fa70f3f

Python multi-process parallel operation - multiprocessing is simple to use - Programmer Sought