python 并发之concurrent.futures----用于支持线程池和进程池

转载自： https://blog.csdn.net/dutsoft/article/details/54728706

一： 使用模块

from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import ProcessPoolExecutor

二：使用示例

from concurrent.futures import ThreadPoolExecutor
import time

def return_future(msg):
    time.sleep(3)
    return msg

# 创建一个线程池
pool = ThreadPoolExecutor(max_workers=2)

# 往线程池加入2个task
f1 = pool.submit(return_future, 'hello')
f2 = pool.submit(return_future, 'world')       # 或使用map, 直接pool.map(return_future, ['hello','world'])
                                               # map根据return_future的返回按照['hello','world']的返回值有序返回
print(f1.done())
time.sleep(3)
print(f2.done())

print(f1.result())
print(f2.result())

【future实例是由executor.submit()创建的，future提供了done,result等等方法】

三：一些额外方法
    

from concurrent.futures import ThreadPoolExecutor as Pool
from concurrent.futures import as_completed
import requests

URLS = ['http://qq.com', 'http://sina.com', 'http://www.baidu.com', ]


def task(url, timeout=10):
    return requests.get(url, timeout=timeout)


with Pool(max_workers=3) as executor:
    future_tasks = [executor.submit(task, url) for url in URLS]

    for f in future_tasks:
        if f.running():   # 判断任务f是否完成
            print('%s is running' % str(f))

    for f in as_completed(future_tasks): 
        try:
            ret = f.done()
            if ret:
                f_ret = f.result()
                print('%s, done, result: %s, %s' % (str(f), f_ret.url, len(f_ret.content)))
        except Exception as e:
            f.cancel()
            print(str(e))

另外还有wait(future_task)，直接返回一个元祖（已完成的，未完成的）