Python multithreading

1. Perform multiple tasks at the same time

multi-process mode
multithreaded mode
Multiprocessing + Multithreading

Note: A process is the smallest unit of resources allocated by the operating system. A thread is the smallest unit of scheduling by the operating system.

2. Multithreaded programming

Multitasking can be done by multiple processes or by multiple threads within a process (a process has at least one process)

3. Modules in Python

_thread and threading. _thread is a low-level module, and threading is a high-level module that encapsulates _thread. In most cases we use the threading advanced module

3. To start a thread is to pass in a function and create a Thread instance, and then call start() to start execution

import threading
import time


def loop(x):
    print("%s start" % threading.current_thread().name)
    for i in range(x):
        time.sleep(1)
        print("%s : %d" % (threading.current_thread().name, i))
        # print("%s stop" % threading.current_thread().name)
print("%s stop" % threading.current_thread().name)    


print("%s start" % threading.current_thread().name)
t1 = threading.Thread(target=loop, args=(6,))
t1.start()
print("%s stop" % threading.current_thread().name)

result:

MainThread start

Thread-1 start
MainThread stop
Thread-1 : 0
Thread-1 : 1
Thread-1 : 2
Thread-1 : 3
Thread-1 : 4
Thread-1 : 5

Thread-1 stop

Result analysis: The main thread runs first, and then runs the child thread

4. If you want the main thread to wait for the child thread to finish executing, use the join() method

import threading
import time


def loop(x):
    print("%s start" % threading.current_thread().name)
    for i in range(x):
        time.sleep(1)
        print("%s : %d" % (threading.current_thread().name, i))
        # print("%s stop" % threading.current_thread().name)
print("%s stop" % threading.current_thread().name)    


print("%s start" % threading.current_thread().name)
t1 = threading.Thread(target=loop, args=(6,))
t1.start()
t1.join()
print("%s stop" % threading.current_thread().name)

result:

MainThread start
Thread-1 start
Thread-1 : 0
Thread-1 : 1
Thread-1 : 2
Thread-1 : 3
Thread-1 : 4
Thread-1 : 5
Thread-1 stop

MainThread stop

Result analysis: MainThread stops at the join position until the child thread finishes running

6. When the main thread exits, no matter where the child thread runs, force the child thread to exit, use setDaemon(True), this method is called before the thread starts

import threading
import time


def loop(x):
    print("%s start" % threading.current_thread().name)
    for i in range(x):
        time.sleep(1)
        print("%s : %d" % (threading.current_thread().name, i))
        # print("%s stop" % threading.current_thread().name)
    print("%s stop" % threading.current_thread().name)


print("%s start" % threading.current_thread().name)
t1 = threading.Thread(target=loop, args=(6,))
t1.setDaemon(True)
t1.start()
# t1.join()
print("%s stop" % threading.current_thread().name)

结果：

MainThread start
Thread-1 start

MainThread stop

7.线程锁(lock)

多线程和多进程最大的不同在于，多进程同一变量各自有一份拷贝存在每个进程中，互不影响，而多线程所有变量由线程共享，所以任何一个变量都可以由任何一个线程修改，因此线程间共享数据最大的危险在与多个线程同时改变一个变量，把内容给改乱了。解决方法：使用threading.Lock()方法，使用线程锁锁定线程需要使用的对象，确保一次只有一个线程能够使用该对象。

实例：多线程操作同一对象出错

import threading

deposit = 0  # 存款


def change(n):
    global deposit
    deposit = deposit + n
    deposit = deposit - n


def loop(n):
    for i in range(10000000):
        change(n)


t1 = threading.Thread(target=loop, args=(5,))
t2 = threading.Thread(target=loop, args=(8,))
t1.start()
t2.start()
t1.join()
t2.join()
print(deposit)

结果：

结果分析：t1,t2交替运行，当两个线程同时访问一个对象时候，容易出错

使用线程锁结果该问题：

import threading

deposit = 0  # 存款


def change(n):
    global deposit
    deposit = deposit + n
    deposit = deposit - n


def loop(n):
    for i in range(10000000):
        lock.acquire()  # 取得一个锁
        try:
            change(n)
        finally:
            lock.release()  # 释放一个锁


lock = threading.Lock()  # 创建一个锁对象
t1 = threading.Thread(target=loop, args=(5,))
t2 = threading.Thread(target=loop, args=(8,))
t1.start()
t2.start()
t1.join()
t2.join()
print(deposit)

结果：

8.线程锁讨论

线程锁确保某段关键代码只能由一个线程从头到尾完整执行，但是线程锁阻止了多线程并发执行，包含锁的某段代码实际上只能以单线程模式执行，效率大大降低。其次由于可以存在多个锁，不同线程支持不同的锁，试图获得对方的锁时，可能会造成死锁。

9.全局解释器锁(GIL)

任何python线程执行前都必须获得GIL锁，执行代码直到线程睡眠或者python将它挂起，线程释放GIL。一个处理核心只能在同一时间运行一个线程。Python中可以使用多线程，但不能有效的利用多个处理核心。可以使用多进程来利用多核。

cpu密集型(各种循环，计数等)：遇到io操作或者ticks超过100(每执行100条字节码)，释放GIL。触发GIL竞争，线程间的来回切换需要消耗资源，故而多线程对CPU密集型代码并不友好。

io密集型(文件处理/网络爬虫)：动线程能提升效率(单线程io会进行io等待，造成不必要的时间浪费，开启多线程可以让进程A等待时自动切换到线程B，可以不浪费cpu资源，提升效率)

10.为什么多进程不会有上述情况。

每一个进程都有各自独立的GIL，互不影响。

Python multithreading

Guess you like