Day10
io操作不占用cpu
计算占用cpu，1+1
python多线程 不适合cpu密集操作型的任务，适合io操作密集型任务
多进程
1 2 3 #Author： Zachary 4 5 import multiprocessing,threading 6 7 import time 8 9 def thread_run(): 10 11 print(threading.get_ident()) 12 13 def run(name): 14 15 time.sleep(2) 16 17 print("hello",name) 18 19 t = threading.Thread(target=thread_run,) 20 21 t.start() 22 23 if __name__ == '__main__': 24 25 for i in range(10): 26 27 p = multiprocessing.Process(target=run,args=('bob %s'%i,)) 28 29 p.start() 30 31 # p.join()
.png)
1 #Author： Zachary 2 3 from multiprocessing import Process 4 5 import os 6 7 8 9 def info(title): 10 11 print(title) 12 13 print('module name:', __name__) 14 15 print('parent process:', os.getppid()) 16 17 print('process id:', os.getpid()) 18 19 print("\n\n") 20 21 22 23 def f(name): 24 25 info('\033[31;1mcalled from child process function f\033[0m') 26 27 print('hello', name) 28 29 30 31 if __name__ == '__main__': 32 33 info('\033[32;1mmain process line\033[0m') 34 35 p = Process(target=f, args=('bob',)) 36 37 p.start() 38 39 p.join()
.png)
进程间通讯
不同进程间内存是不共享的，要想实现两个进程间的数据交换，可以用以下方法：
1、Queues
使用方法跟threading里的queue差不多
1 #Author： Zachary 2 from multiprocessing import Process, Queue 3 4 def f(qq): 5 qq.put([42, None, 'hello']) 6 7 if __name__ == '__main__': 8 q = Queue() #父进程 9 p = Process(target=f, args=(q,)) #子进程 10 p.start() #可以共享数据-> q ,想当于克隆了一份数据 11 print(q.get()) # prints "[42, None, 'hello']" 12 p.join()
.png)
2、Pipes（数据的传递）
The Pipe() function returns a pair of connection objects connected by a pipe which by default is duplex (two-way). For example:
1 #Author： Zachary 2 from multiprocessing import Process, Pipe 3 4 def f(conn): 5 conn.send([42, None, 'hello from child']) #发 6 print("reveive from parent:>>",conn.recv()) #收 7 conn.close() 8 9 if __name__ == '__main__': 10 parent_conn, child_conn = Pipe() 11 p = Process(target=f, args=(child_conn,)) 12 p.start() 13 print("parent receive:>>",parent_conn.recv()) # prints "[42, None, 'hello']" 14 parent_conn.send("近来可好") 15 p.join()
.png)
3、Managers
A manager object returned by Manager() controls a server process which holds Python objects and allows other processes to manipulate them using proxies.
A manager returned by Manager() will support types list, dict, Namespace, Lock, RLock, Semaphore, BoundedSemaphore, Condition, Event, Barrier, Queue, Value and Array. For example,
1 #Author： Zachary 2 from multiprocessing import Process, Manager 3 import os 4 def f(d, l): 5 d[1] = '1' 6 d['2'] = 2 7 d[0.25] = None 8 l.append(os.getpid()) 9 print(l) 10 11 if __name__ == '__main__': 12 with Manager() as manager: 13 d = manager.dict() #生成一个字典，可在多个进程间共享和传递 14 l = manager.list(range(5)) #生成一个列表，可在多个进程间共享和传递 15 p_list = [] 16 for i in range(10): 17 p = Process(target=f, args=(d, l)) 18 p.start() 19 p_list.append(p) 20 for res in p_list: 21 res.join() 22 print(d) 23 print(l)
.png)
进程同步（进程锁）
1 #Author： Zachary 2 from multiprocessing import Process, Lock 3 def f(l, i): 4 l.acquire() #获取锁 5 print('hello world', i) 6 l.release() #释放锁 7 8 if __name__ == '__main__': 9 lock = Lock() 10 for num in range(10): 11 Process(target=f, args=(lock, num)).start()
.png)
进程池
进程池内部维护一个进程序列，当使用时，则去进程池中获取一个进程，如果进程池序列中没有可供使用的进进程，那么程序就会等待，直到进程池中有可用进程为止。
进程池中有两个方法：
* apply
* apply_async
1 #Author： Zachary 2 from multiprocessing import Process, Pool 3 import time,os 4 sta = time.time() 5 def Foo(i): 6 time.sleep(2) 7 print("in process,",os.getpid()) 8 return i + 100 9 10 def Bar(arg): 11 print('-->exec done:', arg) 12 if __name__ == '__main__': 13 pool = Pool(processes=3) #允许进程池同时放入5个进程 14 for i in range(10): #callback回调，当func结束了就执行callback 15 pool.apply_async(func=Foo, args=(i,), callback=Bar) #并行（异步） 16 # pool.apply(func=Foo, args=(i,)) #同步（串行） 17 print('end') 18 pool.close() #这块只能死记硬背 19 pool.join() # 进程池中进程执行完毕后再关闭，如果注释，那么程序直接关闭。 20 print("spend>>>>",time.time() - sta)
.png)
.png)
