Concurrent Programming on Python's process (basic concepts of parallel concurrent, cpu scheduling, blocking)

One: the concept of the process : (Process)

The process is running programs , it is the operating system , the smallest unit of resource allocation .

Resource allocation : allocation is the cpu and memory and other physical resources

Process ID is the unique identifier of the process

After executing the same program twice two processes

Between process and processes : data isolated from each other , through socket communication

II: Parallel and Concurrent

Concurrency : a cpu same time kept execute multiple programs

Parallel : a plurality of cpu same time executing a plurality of programs continuously

Three: the CPU process scheduling method

# First-come, first-served fcfs (first come first server): first come, first execution

# Short operating priority algorithm : allocation of cpu and more , first complete a short count

# Round-robin algorithm : each task execution time of a time slice . Then on the implementation of the other .

# Multilevel feedback queue algorithm

The more a long time , cpu shorter resource allocation , priority on the list

The more short time , cpu more resource allocation

### process three-state map

(1) Ready (Ready) state

Only the CPU needs to be performed outside , all other resources have been allocated called the ready state.

(2) execution (Running) state

cpu began to be called the execution state of the implementation of the process.

(3) blocked (Blocked) state

When an event occurs because the wait can not be executed , it is blocked , cpu execute other processes . For example, waiting for I / O completion of the INPUT , the application buffer can not meet, and so on.

In these three learning content is doped below together with

(1) acquisition process ID

Use os.getpid () Gets the child process ID

Use os.getppid () to get the parent process ID

# For example:

Import OS, Time 

# acquisition sub-process [the current process] the id 
RES1 = os.getpid ()
 Print (RES1)
 # Gets the id of the parent process 
RES2 = os.getppid ()
 Print (RES2)
 # Linux Process Utilization bottom a fork to create a process, but does not support the fork in the windows inside.

# If you are running on the editor found that the parent process ID is not made, the child process number is changed because the parent process ID is the process editor to run the numbers, two sons process ID is the process of this program running number, the program re-run start a child process restarted.

( 2 basic usage) process

Example:

Import os
 from multiprocessing Import Process
 DEF FUNC ():
     Print ( " 1. child process id >>>% s,% S >>> the parent process the above mentioned id " % (os.getpid (), os.getppid ())) 

IF  __name__ == " __main__ " :
     Print ( " 2. child process id >>>% s,% S >>> the parent process the above mentioned id " % (os.getpid (), os.getppid ()))
     # create a child process, returns a process object .target specified tasks to be completed, then the back is a function of 
    the p-process = (target = FUNC)
     # call the child process 
    p.start ()

The result: the parent process ID of the external function is unchanged, the child and parent process ID number within the function is the same, but each run is in the process of changing the number, the number of child processes within each function is changed .

( 3 ) function with parameters
of concurrent processes to rely on cpu cpu first executive who executive who later , to rely on the cpu scheduling policy # no-argument function

# For example:

from multiprocessing Import Process
 Import os 

DEF FUNC ():
     for i in the Range (1, 6 ):
         Print ( " 2. child process id >>>% s,% S >>> the parent process the above mentioned id " % (os.getpid ( ), os.getppid ()))
 IF  __name__ == " __main__ " :
     Print ( " 1. child process id >>>% s,% S >>> the parent process the above mentioned id " % (os.getpid (), os. getppid ()))
     # create a child process 
    the p-process = (target = FUNC)
     # call the child process 
    p.start ()

    n = 5
    for i in range(1,n+1):
        print("*" * i)

# After the first print stars print circulation within the function func, because the child process created the need to open up the stack space required time frame

Screenshot results:

There are function parameters #

from multiprocessing import Process
import os,time
def func(n):
    for i in range(1,n+1):
        time.sleep(0.5)
        print("2.子进程id>>>%s,父进程id>>>%s" % (os.getpid(), os.getppid()))
if __name__ == "__main__":
    print("1.子进程id>>>%s,父进程id>>>%s" % (os.getpid(), os.getppid()))
    n = 5
    # 创建子进程 返回进程对象 如果有参数用args 关键字参数执行
    # 对应的值是元组,参数塞到元组中,按照次序排列
    p = Process(target=func,args=(n,))
    p.start()

    for i in range(1,n+1):
        time.sleep(0.3)
        print("*" * i)

# 这时候会造成,星星和func里面打印子进程和父进程的语句相互交错
# 因为子进程开辟栈帧空间的时间极短,所有再哪个不在睡眠时间内先运行哪个进程

效果截图：

（4）进程之间的数据彼此是隔离的

#例：

from multiprocessing import Process
count = 99
def func():
    global count
    count +=1
    print("我是子进程,count=",count)

if __name__ == "__main__":
    p = Process(target=func)
    p.start()
    print("我是主进程,count=",count)

结果输出为：
我是主进程,count= 99
我是子进程,count= 100

（5）多个进程的并发

#例：

# 在程序并发时, 因为cpu的调度策略问题,不一定谁先执行,谁后执行
from multiprocessing import Process
import os
def func(args):
    print("args=%s,子进程id号>>>%s, 父进程id号>>>%s" % (args, os.getpid(), os.getppid()))

if __name__ == "__main__":
    for i in range(10):
        Process(target=func, args=(i,)).start()

运行得到结果如下图：

可以看出父进程id号是不变的，子进程id号是变化的，而且子进程开启不是按顺序的是并发的。

(6)子进程和父进程之间的关系
通常情况下,父进程会比子进程速度稍快,但是不绝对
在父进程执行所有代码完毕之后,会默认等待所有子进程执行完毕
然后在彻底的终止程序,为了方便进程的管理
如果不等待,子进程会变成僵尸进程,在后台不停地占用内存和cpu资源
但是本身由于进程太多,并不容易发现
#例：

from multiprocessing import Process
import os,time
def func(args):
    print("args=%s,子进程id号>>>%s,父进程id号>>>%s" % (args, os.getpid(), os.getppid()))
    time.sleep(1)
    print("args= %s, end" % (args))

if __name__ == "__main__":
    for i in range(10):
        Process(target=func,args=(i,)).start()
        """
        Process(target=func,args=(i,)).start()
        Process(target=func,args=(i,)).start()
        Process(target=func,args=(i,)).start()
        Process(target=func,args=(i,)).start()
        Process(target=func,args=(i,)).start()
        Process(target=func,args=(i,)).start()
        ....
        """
    print("*******父进程*******")

运行后的结果为：

可以看出父进程号是不变的，子进程是并发的，父进程的执行语句是最快结束的，因为开启子进程需要时间，而主进程的输出没有阻塞所以最快，从print("args= %s, end" % (args))可以知道，子进程是并发的，因为sleep(1)后，如果是顺序执行的话，输出结果不是这样。

四：同步异步 / 阻塞非阻塞

场景在多任务当中

同步:必须等我这件事干完了,你在干,只有一条主线,就是同步

异步:没等我这件事情干完,你就在干了,有两条主线,就是异步

阻塞:比如代码有了input,就是阻塞,必须要输入一个字符串,否则代码不往下执行

非阻塞:没有任何等待,正常代码往下执行.

# 同步阻塞 :效率低,cpu利用不充分

# 异步阻塞 :比如socketserver,可以同时连接多个,但是彼此都有recv

# 同步非阻塞:没有类似input的代码,从上到下执行.默认的正常情况代码

# 异步非阻塞:效率是最高的,cpu过度充分,过度发热

（1）join阻塞基本用法

#例：

from multiprocessing import Process
def func():
    print("我发送第一封邮件....")

if __name__ == "__main__":
    p = Process(target=func)
    p.start()
    # 等待p对象的这个子进程执行完毕之后,在向下执行代码
    # join实际上是加了阻塞
    p.join()  #如果没有join,往往主进程先运行,因为子进程需要开辟栈帧空间相当于加了阻塞
    print("发送第十封邮件")

输出结果为：

我发送第一封邮件....

发送第十封邮件

#程序在发送第十封邮件前加了阻塞，是的会先运行子进程结束在运行join之后的语句

(2) 多个子进程通过join 加阻塞,进行同步的控制

例：

from multiprocessing import Process
import time
def func(index):
    time.sleep(0.3)
    print("第%s封邮件已经发送..." % (index))

if __name__ == "__main__":
    lst = []
    for i in range(1,10):
        p = Process(target=func,args=(i,))
        p.start()
        lst.append(p)

    # 把列表里面的每一个进程对象去执行join()
    # 必须等我子进程执行完毕之后了,再向下执行,控制父子进程的同步性
    for i in lst:
        i.join()
    # 等前9个邮件发送之后了,再发第十个
    print("发送第十封邮件")

所以输出结果为：

第3封邮件已经发送...

第2封邮件已经发送...

第1封邮件已经发送...

第5封邮件已经发送...

第9封邮件已经发送...

第6封邮件已经发送...

第4封邮件已经发送...

第7封邮件已经发送...

第8封邮件已经发送...

发送第十封邮件

十个子进程是并发的，所以不一定谁先执行结束，所以不是顺序的。

（3）使用类的方法创建子进程

# (1) 基本语法
可以使用自定义的方式创建子进程,
但是必须继承父类Processs
而且所有的逻辑都必须写在run方法里面
#例：

from multiprocessing import Process
import os
class MyProcess(Process):
    # 必须使用叫做run的方法,而且()里面只能是self
    def run(self):
        # 写自定义的逻辑
        print("子进程id>>>%s, 父进程的id>>>%s" % (os.getpid(), os.getppid()))

if __name__ == "__main__":
    p = MyProcess()
    p.start()
    print("主进程:{}".format(os.getpid()))

输出结果为：

主进程:8236

子进程id>>>6412, 父进程的id>>>8236

# (2) 带参数的子进程函数
#例：

from multiprocessing import Process
import os
class MyProcess(Process):

    def __init__(self,arg):
        # 必须调用一下父类的初始化构造方法
        super().__init__()
        self.arg = arg

    # 必须使用叫做run的方法
    def run(self):
        # 在这里就得获取参数
        print("子进程id>>>%s,父进程的id>>>%s" % (os.getpid(),os.getppid()))
        print(self.arg)

if __name__ == "__main__":
    lst = []
    for i in range(1,10):
        p = MyProcess("参数:%s" % (i))
        p.start()
        lst.append(p)

    for i in lst:
        i.join()

    print("最后执行主进程的这句话...",os.getpid())

运行结果为：

子进程id>>>5964,父进程的id>>>5336

参数:2

子进程id>>>10544,父进程的id>>>5336

参数:3

子进程id>>>11512,父进程的id>>>5336

参数:4

子进程id>>>11160,父进程的id>>>5336

参数:5

子进程id>>>12068,父进程的id>>>5336

参数:1

子进程id>>>5288,父进程的id>>>5336

参数:6

子进程id>>>11380,父进程的id>>>5336

参数:8

子进程id>>>8092,父进程的id>>>5336

参数:7

子进程id>>>7120,父进程的id>>>5336

参数:9

最后执行主进程的这句话... 5336

结果分析：父进程是程序运行的主进程号不变，子进程是并发的所以那个参数先打印是不固定的，然后对每个子进程添加了join阻塞，所以没有全部执行完子进程之前，主进程的运行语句是不打印的，也就是在子进程join后主进程运行语句必须等待子进程全部结束才打印。