Python multiprocessing multiprocessing (b)

Immediately above

In the above Python multiprocessing multiprocessing (a) we introduced a multi-part process multiprocessing operating basis, in this article, we will continue to introduce some knowledge about the multi-process, such as process pool Pool this useful stuff. Get started now!

Use Case

Example 1

import multiprocessing as mp


def job(x):
    return x*x

def multicore():
    pool = mp.Pool(processes=2)
    res = pool.map(job,range(10))
    print('map res:',res)

    multi_res = [pool.apply_async(job,(i,)) for i in range(10)]
    print('apply_async res:',[res.get() for res in multi_res])

if __name__ == '__main__':
    multicore()

operation result:

map res: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
apply_async res: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

explain:

This example demonstrates the use of process pool pool, Pool create an object and assign 2 cores (using all default) to use.
Before we define the task when the return is not there, map pool method can be applied to the data up tasks sequentially.
Another approach is similar map apply_async, but need to pay attention to the parameters passed here is iterative.

Example 2

In a multithreaded threading, we can use global variables to share a global variable, but in a multi-process it does not work, we need to use in the form of shared memory shared memory, the specific practices are as follows:

import multiprocessing as mp

value = mp.Value('d', 3.14)
array = mp.Array('i', [1,2,3])

explain:

Here show two forms of shared memory, is a Value, is a Array (actually a list)
It is noted that, in the need to specify the value of the type defined values simultaneously, such as the integer "i", double floating point "d", detailed in the table below:

Type code	C Type	Python Type	Minimum size in bytes
'b'	signed char	int	1
'B'	unsigned char	int	1
'In'	Py_UNICODE	Unicode character	2
'h'	signed short	int	2
'H'	unsigned short	int	2
'i'	signed int	int	2
'I'	unsigned int	int	2
'l'	signed long	int	4
'L'	unsigned long	int	4
'q'	signed long long	int	8
'Q'	unsigned long long	int	8
'f'	float	float	4
'd'	double	float	8

(More detailed sources)

Example 3

import multiprocessing as mp

def job(v,num):
    v.value += num
    print(v.value)

def multicore():
    v = mp.Value('i', 0)
    p1 = mp.Process(target=job,args=(v,1))
    p2 = mp.Process(target=job,args=(v,10))
    p1.start()
    p2.start()
    p1.join()
    p2.join()

if __name__ == '__main__':
    multicore()

operation result:

1
11

explain:

This is the use of shared memory effect, two task execution are additive
.Value need to use when calling the shared memory value

Example 4

import multiprocessing as mp

def job(v,num,l):
    l.acquire()
    for i in range(10):
        v.value += num
        print(v.value)
    l.release()

def multicore():
    l = mp.Lock()
    v = mp.Value('i', 0)
    p1 = mp.Process(target=job,args=(v,1,l))
    p2 = mp.Process(target=job,args=(v,10,l))
    p1.start()
    p2.start()
    p1.join()
    p2.join()


if __name__ == '__main__':
    multicore()

operation result:

explain:

Here demonstrates the use of Lock, in fact, usage and multi-threaded lock, just as the use of methods to acquire and release control.
Note that you need to lock an object as a parameter in the process together.

summary

Multiprocessing multiprocessing part on here, plus share before the multithread threading, we should have some intuitive impression, when using multi-threaded multi-process, and which one to use depends on the specific task or use all together . Well, the rest is to go to practice, and stepped pit filled pit, master!