Python program in the process of operation - open multi-process (multiprocess.process)

Python program in the process of operation - open multi-process (multiprocess.process)

Before we have learned a lot of processes relevant theoretical knowledge and understanding of what the process should not be difficult, just that we already know, the running program is a process. All the processes are by its parent to create. Therefore, running the python program is a process, then we can re-create the process in the program. Multiple concurrent processes can achieve the effect, that is, when there are multiple processes of our program, at some point, it will allow faster execution speed of the program. Knowledge before we have learned, does not create a process to achieve this functionality, so we need the help of the powerful python module.

A, multiprocess module

Careful to say, multiprocess module but not a python in one operation, package management processes. It is called multi taken from multiple multi-functional meaning in this package includes almost all sub-modules and related processes. Because so many sub-module, in order to facilitate classify remember, this part of me roughly divided into four parts: part creation process, process synchronization part, the data pool portion between processes, process sharing.

Two, multiprocess.process module

process module is a module process of creating, by means of this module, you can complete the creation process.

Three, process module introduction

Process([group [, target [, name [, args [, kwargs]]]]]), The object of this class is instantiated obtained indicates that the task a sub-process (not yet started)
stressed:

• You need to use the keyword way to specify parameters
• args specified target position for the transfer function parameters, is in the form of a tuple, there must be a comma

Parameter Description:

• group parameter is not used, the values ​​are always None
• target indicates that the call target, that is the task to be executed child process
• args parameter indicates the position of the tuple call the object,args=(1,2,'egon',)
• kwargs indicates that the call object dictionary,kwargs={'name':'egon','age':18}
• name is the name of the child process

3.1 Method Introduction

• p.start(): Start the process and call the child process p.run ()

• p.run(): Method of operation, when the process starts, it is the function to call the specified target, we custom classes in this method must be achieved
  

• p.terminate(): P forced to terminate the process, will not carry out any cleaning operation, if p create a child process, the child process to become a zombie process, this method requires special care this situation. If p then also saved a lock will not be released, leading to a deadlock
• p.is_alive(): If p is still running and returns True

• p.join([timeout]): P wait for the main thread to terminate (stressed: the state is the main thread in the other, and p is in a state of operation). timeout is optional timeout it should be emphasized that, p.join can join live start open process, and can not join the live run open process

3.2 Introduction property

• p.daemon: The default value is False, if set to True, the representative of p is a daemon running in the background, when the termination of the parent process p, p also will be terminated, and after setting True, p can not create their own new process must in the p.start()before setting
• p.nameName of the process:
• p.pid: Pid process
• p.exitcode: The process at runtime None, if -N, representing the end of the signal N (to understand)
• p.authkey: Identity verification process key, default by os.urandom()a 32-character randomly generated string. To succeed (to understand) when the key purpose is to provide security for the network connection between the underlying processes involved in communication, this type of connection only with the same authentication key

3.3 Precautions process module in the windows

In the Windows operating system, because there is no fork (the process of creating a mechanism for the linux operating system), when creating the child process will automatically import this file to start it, but when they perform in the import of the entire file. So if the process () directly written in the file will create a child process infinite recursion error. It is necessary to create a child process of partial use if __name__ =='__main__'judgment protected, import time, it will not run recursively.

Fourth, create a process using a process module

Open a child process, start method and concurrent effect of a python process.

4.1 Start of the first child process in Python

import time
from multiprocessing import Process

def f(name):
    print('hello', name)
    print('我是子进程')

if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()
    time.sleep(1)
    print('执行主进程的内容了')

4.2 join method

import time
from multiprocessing import Process

def f(name):
    print('hello', name)
    time.sleep(1)
    print('我是子进程')


if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()
    # p.join()
    print('我是父进程')

4.3 Check the main and the child's process ID

import os
from multiprocessing import Process

def f(x):
    print('子进程id ：',os.getpid(),'父进程id ：',os.getppid())
    return x*x

if __name__ == '__main__':
    print('主进程id ：', os.getpid())
    p_lst = []
    for i in range(5):
        p = Process(target=f, args=(i,))
        p.start()

Advanced, multiple processes running simultaneously (note that the order of execution of the child process is not determined in accordance with the startup sequence)

4.4 Multiple processes running at the same time

import time
from multiprocessing import Process


def f(name):
    print('hello', name)
    time.sleep(1)


if __name__ == '__main__':
    p_lst = []
    for i in range(5):
        p = Process(target=f, args=('bob',))
        p.start()
        p_lst.append(p)

4.5 multiple processes running at the same time, talk about the join method (1)

import time
from multiprocessing import Process


def f(name):
    print('hello', name)
    time.sleep(1)


if __name__ == '__main__':
    p_lst = []
    for i in range(5):
        p = Process(target=f, args=('bob',))
        p.start()
        p_lst.append(p)
        p.join()
    # [p.join() for p in p_lst]
    print('父进程在执行')

4.6 multiple processes running at the same time, talk about the join method (2)

import time
from multiprocessing import Process

def f(name):
    print('hello', name)
    time.sleep(1)

if __name__ == '__main__':
    p_lst = []
    for i in range(5):
        p = Process(target=f, args=('bob',))
        p.start()
        p_lst.append(p)
    # [p.join() for p in p_lst]
    print('父进程在执行')

In addition to the above method to open these processes, there is a way to open the process in the form of inheritance Process class

4.7 open process through inheritance Process class

import os
from multiprocessing import Process


class MyProcess(Process):
    def __init__(self,name):
        super().__init__()
        self.name=name
    def run(self):
        print(os.getpid())
        print('%s 正在和女主播聊天' %self.name)

p1=MyProcess('wupeiqi')
p2=MyProcess('yuanhao')
p3=MyProcess('nezha')

p1.start() # start会自动调用run
p2.start()
# p2.run()
p3.start()


p1.join()
p2.join()
p3.join()

print('主线程')

Data between processes isolate the problem

Data between 4.8 process isolation problem

from multiprocessing import Process

def work():
    global n
    n=0
    print('子进程内: ',n)


if __name__ == '__main__':
    n = 100
    p=Process(target=work)
    p.start()
    print('主进程内: ',n)

V. daemon

With the end of the primary process will be ended. Create a master daemon process

First: the daemon will terminate after the end of the main process code execution
Second: Could not open a sub-process within the daemon, or throw an exception: AssertionError: daemonic processes are not allowed
to have children Note: the process is independent from each other the main process code to run over, daemon terminated.

Start 5.1 daemon

import os
import time
from multiprocessing import Process

class Myprocess(Process):
    def __init__(self,person):
        super().__init__()
        self.person = person
    def run(self):
        print(os.getpid(),self.name)
        print('%s正在和女主播聊天' %self.person)


p=Myprocess('哪吒')
p.daemon=True # 一定要在p.start()前设置,设置p为守护进程,禁止p创建子进程,
             #  并且父进程代码执行结束,p即终止运行
p.start()
time.sleep(10)  # 在sleep时查看进程id对应的进程ps -ef|grep id
print('主')

5.2 The main process code execution daemon immediate end to end

from multiprocessing import Process

def foo():
    print(123)
    time.sleep(1)
    print("end123")

def bar():
    print(456)
    time.sleep(3)
    print("end456")


p1=Process(target=foo)
p2=Process(target=bar)

p1.daemon=True
p1.start()
p2.start()
time.sleep(0.1)
print("main-------") 
# 打印该行则主进程代码结束,则守护进程p1应该被终止.
#可能会有p1任务执行的打印信息123,因为主进程打印main----时,
# p1也执行了,但是随即被终止.

Six, socket chat concurrent instances

6.1 Use multiple processes to achieve concurrent chat socket -server end

from socket import *
from multiprocessing import Process

server=socket(AF_INET,SOCK_STREAM)
server.setsockopt(SOL_SOCKET,SO_REUSEADDR,1)
server.bind(('127.0.0.1',8080))
server.listen(5)

def talk(conn,client_addr):
    while True:
        try:
            msg=conn.recv(1024)
            if not msg:break
            conn.send(msg.upper())
        except Exception:
            break

if __name__ == '__main__': # windows下start进程一定要写到这下面
    while True:
        conn,client_addr=server.accept()
        p=Process(target=talk,args=(conn,client_addr))
        p.start()

6.2 Use socket chat multiple processes to achieve concurrent -client end

from socket import *

client=socket(AF_INET,SOCK_STREAM)
client.connect(('127.0.0.1',8080))


while True:
    msg=input('>>: ').strip()
    if not msg:continue

    client.send(msg.encode('utf-8'))
    msg=client.recv(1024)
    print(msg.decode('utf-8'))

Seven multi-process in other ways

Other methods 7.1 process objects: terminate and is_alive

from multiprocessing import Process
import time
import random

class Myprocess(Process):
    def __init__(self,person):
        self.name=person
        super().__init__()

    def run(self):
        print('%s正在和网红脸聊天' %self.name)
        time.sleep(random.randrange(1,5))
        print('%s还在和网红脸聊天' %self.name)


p1=Myprocess('哪吒')
p1.start()

p1.terminate() #关闭进程,不会立即关闭,所以is_alive立刻查看的结果可能还是存活
print(p1.is_alive()) #结果为True

print('开始')
print(p1.is_alive()) #结果为False

7.2 Other properties of process objects: pid and name

class Myprocess(Process):
    def __init__(self,person):
        self.name=person   # name属性是Process中的属性，标示进程的名字
        super().__init__()  # 执行父类的初始化方法会覆盖name属性
        # self.name = person  # 在这里设置就可以修改进程名字了
        # self.person = person  # 如果不想覆盖进程名，就修改属性名称就可以了
    def run(self):
        print('%s正在和网红脸聊天' %self.name)
        # print('%s正在和网红脸聊天' %self.person)
        time.sleep(random.randrange(1,5))
        print('%s正在和网红脸聊天' %self.name)
        # print('%s正在和网红脸聊天' %self.person)


p1=Myprocess('哪吒')
p1.start()
print(p1.pid)    #可以查看子进程的进程id