py 多线程，多进程

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/qq_26437925/article/details/80689324

• python GIL
  https://wiki.python.org/moin/GlobalInterpreterLock

---

环境
* CPU 2个核
* python 2.7.10

CPU密集型

实践代码

# -*- coding:utf-8 -*-
import time
from threading import Thread
from multiprocessing import Process

def count():
    '''
    # CPU密集操作
    '''
    c = 0
    x = 1
    y = 2
    while c < 100000:
        c += 1
        x += x
        y += y

def countn(n):
    for x in range(n):
        count()


def multi_thread(n):
    counts = []
    t = time.time()
    tcount = 8
    tmp = tcount / n
    for i in range(n):
        thread = Thread(target=countn, args=(tmp,))
        counts.append(thread)
        thread.start()

    for i in range(n):
        counts[i].join()

    print("thread count=%d" % n, time.time() - t)

def multi_process(n):
    counts = []
    t = time.time()
    tcount = 8
    tmp = tcount / n
    for i in range(n):
        process = Process(target=countn, args=(tmp,))
        counts.append(process)
        process.start()

    for i in range(n):
        counts[i].join()

    print("process count=%d" % n, time.time() - t)

if __name__ == '__main__':
    li = [1, 2, 4, 8]
    for x in li:
        multi_thread(x)
    for x in li:
        multi_process(x)

运行结果

('thread count=1', 2.6109211444854736)
('thread count=2', 2.642864942550659)
('thread count=4', 2.6333298683166504)
('thread count=8', 2.6398651599884033)
('process count=1', 2.594637155532837)
('process count=2', 1.3601601123809814)
('process count=4', 1.0247399806976318)
('process count=8', 0.9624059200286865)

结论

python中GIL的存在，进程是有单独的GIL的，线程没有。对于CPU密集型的程序，多线程运行起来跟单线程基本没有区别，因为同一时刻，GIL的存在只能使得一个线程能够执行，所以效率基本不会提高，反而可能下降（线程的创建，切换等都要耗时）。

IO密集型

实践代码

# -*- coding:utf-8 -*-
import time
from threading import Thread
from multiprocessing import Process

def io():
    '''
    # IO密集操作
    '''
    x = 1
    y = 2
    time.sleep(1)
    x += x
    time.sleep(1)
    y += y

def countn(n):
    for x in range(n):
        io()

def multi_thread(n):
    counts = []
    t = time.time()
    tcount = 8
    tmp = tcount / n
    for i in range(n):
        thread = Thread(target=countn, args=(tmp,))
        counts.append(thread)
        thread.start()

    for i in range(n):
        counts[i].join()

    print("thread count=%d" % n, time.time() - t)

def multi_process(n):
    counts = []
    t = time.time()
    tcount = 8
    tmp = tcount / n
    for i in range(n):
        process = Process(target=countn, args=(tmp,))
        counts.append(process)
        process.start()

    for i in range(n):
        counts[i].join()

    print("process count=%d" % n, time.time() - t)

if __name__ == '__main__':
    li = [1, 2, 4, 8]
    for x in li:
        multi_thread(x)
    for x in li:
        multi_process(x)

运行结果

('thread count=1', 16.05018401145935)
('thread count=2', 8.026233196258545)
('thread count=4', 4.011896848678589)
('thread count=8', 2.004067897796631)
('process count=1', 16.032613039016724)
('process count=2', 8.01167893409729)
('process count=4', 4.009658098220825)
('process count=8', 2.012108087539673)

结论

IO密集型操作，如文件处理、网络爬虫等涉及文件读写的操作，其特点是CPU消耗很少，任务的大部分时间都在等待IO操作完成（因为IO的速度远远低于CPU和内存的速度）。

多线程，多进程都能在IO密集型操作下体现优势。因为可以利用IO阻塞等待时的空闲时间去执行其它的线程（或进程）。在某个特定的时刻，可以有多个线程（或）进程运行（例如某些线程阻塞，某个线程执行CPU计算），这显然能提高效率。

参考

1. CPU-bound(计算密集型) 和I/O bound(I/O密集型).剑西楼[EB/OL]https://blog.csdn.net/q_l_s/article/details/51538039

2. Python中单线程、多线程和多进程的效率对比实验.饒木陽 [EB/OL]http://python.jobbole.com/86822/