1. Multi-process
1. The child process returns forever 0, while the parent process returns the ID of the child process. getppid()The reason for this is that a parent process can fork many child processes, so the parent process needs to record the ID of each child process, and the child process can get the ID of the parent process only by calling .
2.multiprocessing
multiprocessingThe module provides a Processclass to represent a process object, the following example demonstrates starting a child process and waiting for it to end
1 import os,time,random 2 from multiprocessing import Process 3 #运行多个子进程 4 def run_child_process(name): 5 print("run child process %s(%s)"%(name,os.getpid())) 6 7 if __name__=='__main__': 8 print("parent process %s"%os.getpid()) 9 p1=Process(target=run_child_process,args=("p1",)) 10 p1.start() 11 p1.join() 12 print("child process end")
3. Process pool pool
If you want to start a large number of subprocesses, you can create subprocesses in batches in a process pool:
1 from multiprocessing import Pool 2 import os,time,random 3 def long_time_task(name): 4 print("run task %s(%s)"%(name,os.getpid())) 5 start=time.time() 6 time.sleep(random.random()*3) 7 end=time.time() 8 print("task %s takes %0.2f seconds"%(name,(end-start))) 9 10 if __name__=='__main__': 11 print("parent process %s"%os.getpid()) 12 p=Pool(4)
#pool的默认值是cpu数 13 for i in range(5): 14 p.apply_async(long_time_task,args=(i,)) 15 print("waiting for all subprocess done") 16 p.close() 17 p.join() 18 print("all subprocess done")
Results of the:
Parent process 4984. Waiting for all subprocesses done... Run task 3 (9496)... Task 3 runs 1.65 seconds. Run task 4 (9496)... Task 4 runs 0.16 seconds. Run task 0 (11036)... Task 0 runs 2.25 seconds. Run task 2 (8680)... Task 2 runs 2.67 seconds. Run task 1 (11100)... Task 1 runs 2.97 seconds. All subprocesses done. [Finished in 3.7s]
2. Multithreading
Multitasking can be done by multiple processes or by multiple threads within a process.
We mentioned earlier that a process is composed of several threads, and a process has at least one thread
1. Start the thread
1 import threading,time 2 def loop(): 3 print("thread %s is running..."%threading.current_thread().name) 4 n=0 5 while n<5: 6 n=n+1 7 print("thread %s>>>%s"%(threading.current_thread().name,n)) 8 time.sleep(1) 9 print("thread %s is ended"%threading.current_thread().name) 10 print("thread %s is running"%threading.current_thread().name) 11 t=threading.Thread(target=loop,name="loopthread") 12 t.start() 13 t.join() 14 print("thread %s is ended"%threading.current_thread().name)
Results of the:
1 thread MainThread is running 2 thread loopthread is running... 3 thread loopthread>>>1 4 thread loopthread>>>2 5 thread loopthread>>>3 6 thread loopthread>>>4 7 thread loopthread>>>5 8 thread loopthread is ended 9 thread MainThread is ended 10 [Finished in 5.3s]
2.lock
The biggest difference between multithreading and multiprocessing is that in multiprocessing, a copy of the same variable exists in each process without affecting each other, while in multithreading, all variables are shared by all threads, so any one Variables can be modified by any thread. Therefore, the biggest danger of sharing data between threads is that multiple threads change a variable at the same time, which will mess up the content.
1 import time,threading 2 balance=0 3 lock=threading.Lock() 4 def change_it(n): 5 global balance 6 balance=balance+n 7 balance=balance-n 8 9 def run_thread(n): 10 for i in range(100000): 11 lock.acquire() 12 try: 13 change_it(n) 14 finally: 15 lock.release() 16 t1=threading.Thread(target=run_thread,args=(5,)) 17 t2=threading.Thread(target=run_thread,args=(8,)) 18 t1.start() 19 t2.start() 20 t1.join() 21 t1.join() 22 print(balance)
3. Queue
1. ProcessThere is definitely a need for communication between them. The operating system provides many mechanisms to achieve inter-process communication. Python's multiprocessingmodules wrap the underlying mechanism and provide Queuevarious Pipesways to exchange data.
Let's take Queueas an example, create two child processes in the parent process, one Queueto write data and one Queueto read data from it:
1 from multiprocessing import Process, Queue 2 import os, time, random 3 #The code executed by the write data process: 4 def write(q): 5 print ( ' Process to write: %s ' % os.getpid()) 6 for value in [ ' A ' , ' B ' , ' C ' ]: 7 print ( ' Put %s to queue... ' % value) 8 q.put(value) 9 time.sleep(random.random()) 10 11 #The code executed by the read data process: 12 def read(q): 13 print ( ' Process to read: %s ' % os.getpid( )) 14 while True: 15 value = q.get(True) 16 print ( ' Get %s from queue. ' % value) 17 18 if __name__ == ' __main__ ' : 19 #The parent process creates a Queue and passes it to each Subprocesses: 20 q = Queue() 21 pw = Process(target=write, args= (q,)) 22 pr = Process(target=read, args= (q,)) 23 #Start child process pw, write: 24 pw. start() 25 #Start the child process pr, read: 26 pr.start() 27 #Wait for the end of pw: 28 pw.join() 29 # The pr process is an infinite loop, you can't wait for it to end, you can only force it to terminate: 30 pr.terminate()
Original address: https://www.liaoxuefeng.com/wiki/0014316089557264a6b348958f449949df42a6d3a2e542c000/001431927781401bb47ccf187b24c3b955157bb12c5882d000
https://yuedu.baidu.com/ebook/0f6a093b7dd184254b35eefdc8d376eeaeaa17e3?pn=1&rf=https%3A%2F%2Fyuedu.baidu.com%2Febook%2F0f6a093b7dd184254b35eefdc8d376eeaeaa17e3