1. Introduction to multiprocessing module

Multi-threading in Python cannot take advantage of multi-core. If you want to make full use of multi-core CPU resources (os.cpu_count () view), in most cases in Python, you need to use multi-process. Python provides multiprocessing. The multiprocessing module is used to start a subprocess and execute our customized tasks (such as functions) in the subprocess. This module is similar to the programming interface of the multithreading module threading.

The multiprocessing module has many functions: it supports sub-processes, communication and shared data, performs different forms of synchronization, and provides components such as Process, Queue, Pipe, and Lock.

One thing that needs to be emphasized again is that, unlike threads, processes do not have any shared state, and the data modified by a process is limited to that process.

2. Introduction of Process

Create process class:

Process ([group [, target [, name [, args [, kwargs]]]]]), an object instantiated by this class, represents a task in a child process (not yet started)

Emphasize:
1. Need to use keywords to specify the parameters
 2. args specifies the positional parameters passed to the target function, is a tuple form, must have a comma

Parameter introduction:

The group parameter is not used, the value is always None
target indicates the calling object, that is, the task to be performed by the child process
args represents the tuple of position parameters of the calling object, args = (1,2, ' egon ' ,)
kwargs represents a dictionary of calling objects, kwargs = { ' name ' : ' egon ' , ' age ' : 18 }
name is the name of the child process

Method introduction:

p.start (): Start the process and call p.run () in the subprocess 
p.run (): The method to run when the process starts, it is precisely it to call the function specified by the target, we must implement this method in the class of our custom class  
p.terminate (): Forcibly terminate the process p without any cleanup operation. If p creates a child process, the child process becomes a zombie process. This method requires special care. If p still holds a lock, it will not be released, which will cause a deadlock
p.is_alive (): If p is still running, return True
p.join ([timeout]): The main thread waits for p to terminate (emphasis: the main thread is in a waiting state, and p is in a running state). timeout is an optional timeout, it should be emphasized that p.join can only join processes started by start, but not processes started by run.

Property introduction:

p.daemon: The default value is False. If set to True, it means that p is a daemon running in the background. When p's parent process terminates, p also terminates. After setting to True, p cannot create its own new Process, must be set before p.start ()

p.name: the name of the process

p.pid: the pid of the process

p.exitcode: The process is None when running, if it is -N, it means that it is ended by the signal N (you can understand it)

p.authkey: process authentication key, the default is a 32-character string randomly generated by os.urandom (). The purpose of this key is to provide security for low-level inter-process communication involving network connections. Such connections can only succeed if they have the same authentication key (understand)

Three, the use of Process class

Note: In Windows, Process () must be placed under: if __name__ == "__main__"

Detailed explanation:

Since Windows has no fork, the multiprocessing module starts a new Python process and imports the calling module. 
If Process() gets called upon import, then this sets off an infinite succession of new processes (or until your machine runs out of resources). 
This is the reason for hiding calls to Process() inside

if __name__ == "__main__"
since statements inside this if-statement will not get called upon import.
Since there is no fork in Windows, the multiprocessing module starts a new Python process and imports the calling module. 
If Process () is called during import, then this will start a new process with infinite inheritance (or until the machine runs out of resources). 
This is to hide the internal call to Process (). If if __name__ == " __main __", the statement in this if statement will not be called during import.

Two ways to create and start child processes:

method one:

import time
# import random
from multiprocessing import Process
def piao(name):
    print('%s piaoing' %name)
    # time.sleep(random.randrange(1,5))
    time.sleep(1)
    print('%s piao end' %name)


if __name__ == '__main__':

    P1 = Process (target = Piao, args = ( ' Egon ' ,)) # must be added, number 
    P2 = Process (target = Piao, args = ( ' Alex ' ,))
    p3=Process(target=piao,args=('wupeqi',))
    p4=Process(target=piao,args=('yuanhao',))

    p1.start()
    p2.start()
    p3.start()
    p4.start()
    print ( ' main thread ' )

'''
Main thread
be piaoing
alex piaoing
wupeqi piaoing
yuanhao piaoing
egon piao end
alex piao end
wupeqi piao end
yuanhao piao end #Note 

: 
1. p1.start () is to tell the operating system to apply for memory space, and then run the code, then p1.start () line of code is finished, continue to run print () 
2, apply for memory Space and then re-run the code takes a long time, the speed of the print code must be fast, so print "main thread" 

'' '

Method Two:

# Method to open two processes: 
Import Time
 # Import Random 
from multiprocessing Import Process


class Piao(Process):
    def __init__(self,name):
        super().__init__()
        self.name=name
    def run(self):
        print('%s piaoing' %self.name)

        # time.sleep(random.randrange(1,5))
        time.sleep(1)
        print('%s piao end' %self.name)
if __name__ == '__main__':

    p1 = Piao ( ' egon ' )
    p2=Piao('alex')
    p3 = Piao ( ' wupeiqi ' )
    p4 = Piao ( ' yuanhao ' )

    p1.start () # start will automatically call run 
    p2.start ()
    p3.start()
    p4.start()
    print ( ' main thread ' )

'''
Main thread
be piaoing
alex piaoing
wupeiqi piaoing
yuanhao piaoing
egon piao end
alex piao end
wupeiqi piao end
yuanhao piao end
'''

to sum up:

"""
The creation process is to apply for a piece of memory space in memory and throw the code that needs to be run into it.
A process corresponds to a separate memory space in memory
Multiple processes correspond to multiple independent memory spaces in memory
The data between processes cannot be directly interacted with by default.If you want to interact, you can use third-party tools and modules.
"""

Use of join method

Join is to let the main process wait for the sub-process code to finish running before continuing. Does not affect the execution of other child processes

from multiprocessing import Process
import time


def task(name, n):
    print('%s is running'%name)
    time.sleep(n)
    print('%s is over'%name)


if  __name__ == ' __main__ ' :
     # p1 = Process (target = task, args = ('jason', 1)) 
    # p2 = Process (target = task, args = ('egon', 2)) 
    # p3 = Process (target = task, args = ('tank', 3)) 
    # start_time = time.time () 
    # p1.start () 
    # p2.start () 
    # p3.start () # just tell the operating system to create a process 
    # # time.sleep (50000000000000000000) 
    # # p.join () # The main process waits for the child process p to finish running before continuing to execute 
    # p1.join () 
    # p2.join () 
    # p3.join () 
    start_time = time.time ()
    p_list = []
    for i in range(1, 4):
        p = Process(target=task, args=('子进程%s'%i, i))
        p.start()
        p_list.append(p)
    for p in p_list:
        p.join()
    print('主', time.time() - start_time)

Results of the:

'''
Child process 1 is running
Child process 2 is running
Child process 3 is running
Child process 1 is over
Child process 2 is over
Child process 3 is over
Main 3.0982506275177
'''

The memory space between processes is isolated

from multiprocessing import Process
n- = 100 # should be defined in a global variable in the windows system '__main__' on it if __name__ == 
DEF Work ():
     Global n-
    n = 0
     print ( ' inside child process: ' , n)


if __name__ == '__main__':
    p=Process(target=work)
    p.start()
    print ( ' Inside main process: ' , n)


"""
In main process: 100
In child process: 0
"""

Multiple processes of concurrent Python programming (operation)

table of Contents: