Concurrent Programming thread pool, pool process

I. Why introduce the concept of the pool, the pool is what

Pool

　　Development is limited to the development of hardware, software, hardware can not keep up

　　In the case of computer hardware to ensure security, to maximize the use of the computer

　　In fact, the pool is to reduce the efficiency of the program, but to ensure the security of computer hardware

When we re-use processes and threads, unlimited impossible to open the process or thread. So we need to use the process pool, thread pool to address this problem.

Two .Python module ----- concurrent.futures module (concurrent Future)

1.concurrent module is used to create parallel tasks, provides a higher level interface

2. The module import process and thread pool pool

from concurrent.futures import ProcessPoolExecutor  进程池

from concurrent.futures import ThreadPoolExecutor 线程池

3.p = ProcessPoolExecutor (max_works) for the process pool if you do not write max_works default is the number of CPU

   p = ThreadPoolExecutor (max_works) for the thread pool if you do not write max_works default is the number of CPU * 5

4.p.submit (task, i) is a function of an asynchronous task i is submitted to the desired task parameters

　obj = p.submit (task, i) returns an object obj

　obj.result () function returns the result of

　p.shutdown () corresponds to the closed pond close join

III. Thread pool, pool process 

Based on concurrent thread pool and pool process module

 1 from concurrent.futures import ThreadPoolExecutor,ProcessPoolExecutor
 2 import time
 3 
 4 def task(n):
 5     print(n)
 6     time.sleep(2)
 7     return n+1
 8 
 9 def call_back(n):
10     print('结果是:',n.result())
11 
12 if __name__ == '__main__':
13     start = time.time()
14     pool = ThreadPoolExecutor(5)
15     #pool = ProcessPoolExecutor(5)
16     for i in range(7):
17         res = pool.submit(task,i).result()
18         print(res)
19 
20     pool.shutdown()
21 
22     print(time.time()-start)
23 #14.00873589515686

Synchronous execution

from concurrent.futures import ThreadPoolExecutor,ProcessPoolExecutor
import time
 
 
def task(n):
     print(n)
     time.sleep(2)
     return n+1
 
def call_back(n):
     print('结果是:',n.result())
 
if __name__ == '__main__':
     start = time.time()
     pool = ThreadPoolExecutor(5)
    #pool = ProcessPoolExecutor(5)
     for i in range(7):
         res = pool.submit(task,i)
         p_list.append(res)
 
     pool.shutdown()
 
     for p in p_list:
          print('>>>:',p.result())
   
 
     print(time.time()-start)
  # 4.002536296844482

Asynchronous execution

 Using asynchronous callbacks

from concurrent.futures import ThreadPoolExecutor,ProcessPoolExecutor
import time


def task(n):
    print(n)
    time.sleep(2)
    return n+1

def call_back(n):
    print('结果是:',n.result())

if __name__ == '__main__':
    start = time.time()
    pool = ThreadPoolExecutor(5)
    #pool = ProcessPoolExecutor(5)
    for i in range(7):
        res = pool.submit(task,i).add_done_callback(call_back)  #异步
    pool.shutdown()

    print(time.time()-start)
    # 4.002536296844482

Asynchronous callbacks