Introduction to multi-threaded programming in python in ten minutes

1.threading function

When a Python program starts, the Python interpreter will start a threading._MainThread thread object inherited from threading.Thread as the main thread, so when it comes to the methods and functions of threading.Thread, this main thread is usually included.

threading.active_count(): Returns the number of currently alive threading.Thread thread objects, which is equivalent to len(threading.enumerate()).
threading.current_thread(): Returns the threading.Thread thread object controlled by the caller of this function. If the thread currently controlled by the caller was not created through threading.Thread, a virtual thread object with limited functions is returned.
threading.get_ident(): Returns the thread identifier of the current thread. Note that when a thread exits, its thread identifier may be reused by newly created threads.
threading.enumerate(): Returns the list of currently living threading.Thread thread objects.
threading.main_thread(): Returns the main thread object, which is usually the threading._MainThread thread object created by the Python interpreter when the program starts.

2. Thread object: threading.Thread

threading.Thread currently does not have priority and thread group functions, and the created thread cannot be destroyed, stopped, suspended, resumed or interrupted.

Daemon thread: Only when all daemon threads end, the entire Python program will exit, but it does not mean that the Python program will wait for the daemon thread to finish running. On the contrary, when the program exits, if there are still daemon threads running, the program will be forcibly terminated. All daemon threads, the program will not actually exit until all daemon threads terminate. You can designate a thread as a daemon thread by modifying the daemon attribute or specifying the daemon parameter when initializing the thread.

Non-daemon threads: Generally created threads are non-daemon threads by default, including the main thread. That is, when the Python program exits, if there are still non-daemon threads running, the program will wait until all non-daemon threads have finished before exiting.

threading.Thread(group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None)

If the initialization method of this class is overridden, be sure to call the __init__ method of the threading.Thread class before doing anything in the overridden initialization method.

group: should be set to None, that is, no setting is required, just use the default value, because this parameter is reserved for future implementation of the ThreadGroup class.
target: The callable object called in the run method, that is, the callable object that needs to start the thread, such as a function or method.
name: Thread name, defaults to a name in the form of "Thread-N", N is a small decimal number.
args: The parameter tuple of the callable object passed in the parameter target, the default is an empty tuple ().
kwargs: Keyword parameter dictionary of the callable object passed in the parameter target, the default is an empty dictionary {}.
daemon: The default is None, which means it inherits the daemon mode attribute of the current caller thread (that is, the thread that starts the thread, usually the main thread). If it is not None, the thread will be set to "daemon mode" regardless of whether it is in daemon mode or not. ".

Thread function

start(): Start thread activity. It will cause the run() method to be called in an independent control thread. It should be noted that the start() method of the same thread object can only be called once. If it is called multiple times, a RuntimeError will be reported.
run(): This method represents thread activity.
join(timeout=None): Let the current caller thread (that is, the thread that starts the thread, usually the main thread) wait until the thread ends (no matter what reason it ends), the timeout parameter is a floating point number in seconds, Used to set the operation timeout time, and the return value is None. If you want to determine whether a thread has timed out, you can only judge it through the thread's is_alive method. The join method can be called multiple times. If you use the join method on the current thread (that is, the thread calls its own join method internally), or if you use the join method before the thread has started, a RuntimeError will be reported.
is_alive(): Whether the thread is alive, returns True or False. This method returns True after the thread's run() runs until run() ends.
daemon: Indicates whether the thread is a daemon thread, True or False. Setting the daemon of a thread must be before the thread's start() method, otherwise a RuntimeError will be reported. This value is inherited from the thread that created it by default. The main thread is non-daemon thread by default, so the threads created in the main thread are non-daemon thread by default, that is, daemon=False.

A simple example of creating a thread using the threading.Thread class:

#通过实例化threading.Thread类创建线程

import time
import threading

#定义线程
def test_thread(param='hi', sleep=3):
    time.sleep(sleep)
    print(param)


def main():
    # 创建线程
    thread_hi = threading.Thread(target=test_thread)
    # 启动线程
    thread_hi.start()
    print('Main thread has ended!')


if __name__ == '__main__':
    main()

#输出：Main thread has ended!  hi

A simple example of creating a thread using a subclass of the threading.Thread class:


#通过继承threading.Thread的子类创建线程

import time
import threading

#TestThread继承threading.Thread
class TestThread(threading.Thread):
    def __init__(self, param='hello', sleep=3):
        # 重写threading.Thread的__init__方法时，确保在所有操作之前先调用threading.Thread.__init__方法
        super().__init__()
        self.para = param
        self.sleep = sleep

    def run(self):
        #线程内容
        time.sleep(self.sleep)
        print(self.para)


def main():
    # 创建线程
    thread_hi = TestThread()
    # 启动线程
    thread_hi.start()

    print('Main thread has ended!')


if __name__ == '__main__':
    main()
#输出：Main thread has ended! hello

Simple example of join method:


#使用join方法阻塞主线程

import time
import threading

#定义线程函数
def test_thread(para='hello', sleep=5):
    #线程运行函数
    time.sleep(sleep)
    print(para)


def main():
    # 创建线程
    thread_hello = threading.Thread(target=test_thread)

    # 启动线程
    thread_hello.start()
    time.sleep(2)
    print('马上执行join方法了')
    # 执行join方法会阻塞调用线程（主线程），直到调用join方法的线程（thread_hello）结束
    thread_hello.join()
    print('线程thread_hello已结束')
    # 这里不会阻塞主线程，因为运行到这里的时候，线程thread_hello已经运行结束了

    print('Main thread has ended!')



if __name__ == '__main__':
    main()

3. Lock object: threading.Lock

threading.Lock is implemented directly through the _thread module extension.

When a lock is locked, it does not belong to a specific thread.

Locks have only two states: "locked" and "unlocked". When the lock is created, it is in the "unlocked" state. When the lock is already locked, calling the acquire() method again will block execution until the lock is called the release() method to release the lock and change its status to "unlocked".

After the same thread acquires the lock, if it acquires the lock again before releasing the lock, it will cause the current thread to block unless another thread releases the lock. If there is only one thread and this happens, it will cause this thread to block forever. That is, a deadlock is formed. Therefore, when acquiring the lock, you need to ensure that the lock has been released, or use a recursive lock to solve this situation.

Lock

acquire(blocking=True, timeout=-1): Acquire the lock and change the lock status to "locked", returning True if successful and False if failed. When a thread acquires a lock, it blocks other threads trying to acquire the lock until the lock is released. The default value of timeout is -1, which means that it will block and wait indefinitely until the lock is obtained. If it is set to another value (a floating point number in seconds), it will block and wait for the number of seconds specified by timeout at most. When blocking is False, the timeout parameter is ignored

release lock

release(): Releases a lock and changes its status to "unlocked". It should be noted that any thread can release the lock, not just the thread that obtained the lock (because the lock does not belong to a specific thread). The release() method can only be called when the lock is in the "locked" state. If it is called in the "unlocked" state, a RuntimeError will be reported.

Simple example of thread synchronization using locks:


#使用锁实现线程同步

import time
import threading

# 创建锁
lock = threading.Lock()

# 全局变量
global_resource = [None] * 5


# 这段代码如果不加锁，第一个线程运行结束后global_resource中是乱的，输出为：修改全局变量为：  ['小红', '小明', '小明', '小红', '小红']
# 第二个线程运行结束后，global_resource中还是乱的，输出为：修改全局变量为： ['小红', '小明', '小明', '小明', '小明']
def change_resource(param, sleep):
    # 请求锁
    lock.acquire()


    global global_resource
    for i in range(len(global_resource)):
        global_resource[i] = param
        time.sleep(sleep)
    print("修改全局变量为：", global_resource)

    # 释放锁
    lock.release()


def main():
    thread_hi = threading.Thread(target=change_resource, args=('小明', 2))
    thread_hello = threading.Thread(target=change_resource, args=('小红', 1))
    thread_hi.start()
    thread_hello.start()


if __name__ == '__main__':
    main()

4. Recursive lock object: threading.RLock

The difference between recursive locks and ordinary locks is that the concepts of "ownership thread" and "recursion level" are added. To release a lock, a thread that acquires the lock must be released. At the same time, the same thread will not be blocked if it acquires the lock again before releasing the lock. The current thread just adds 1 to the recursion level of the lock (the initial recursion level when acquiring the lock is 1).

When using ordinary locks, you can consider using recursive locks to solve some situations that may cause deadlock.

Lock

acquire(blocking=True, timeout=-1): The difference from ordinary locks is that when using the default value, if this thread already owns the lock, the recursion level of the lock is increased by 1. When a thread acquires a lock, the lock's recursion level is initialized to 1. When multiple threads are blocked, only one thread can acquire the lock when the lock is unlocked. In this case, acquire() has no return value.

release lock

release(): There is no return value. If called once, the recursion level will be reduced by 1. When the recursion level is zero, it means that the lock of this thread has been released and other threads can acquire the lock. It is possible that acquire() has been called multiple times in a thread, causing the recursion level of the lock to be greater than 1. Then you need to call release() the corresponding number of times to completely release the lock and reduce its recursion level to zero. Other threads To acquire the lock, otherwise it will always be blocked.

Simple example of recursive lock usage:


#在普通锁中可能造成死锁的情况，可以考虑使用递归锁解决

import time
import threading


# 使用成一个递归锁就可以解决当前这种死锁情况
rlock_1 = rlock_2 = threading.RLock()


def thread_1():
    # 初始时锁内部的递归等级为1
    rlock_1.acquire()
    print('线程thread_1获得了锁rlock_1')
    time.sleep(2)
    # 如果再次获取同样一把锁，则不会阻塞，只是内部的递归等级加1
    rlock_2.acquire()
    print('线程thread_1获得了锁rlock_2')
    # 释放一次锁，内部递归等级减1
    rlock_2.release()
    # 这里再次减，当递归等级为0时，其他线程才可获取到此锁
    rlock_1.release()


def thread_2():
    rlock_2.acquire()
    print('线程thread_2获得了锁rlock_2')
    time.sleep(2)
    rlock_1.acquire()
    print('线程thread_2获得了锁rlock_1')
    rlock_1.release()
    rlock_2.release()


def main():
    thread_t1 = threading.Thread(target=thread_1)
    thread_t2 = threading.Thread(target=thread_2)
    thread_t1.start()
    thread_t2.start()


if __name__ == '__main__':
    main()

5. Semaphore object: threading.Semaphore

A semaphore manages an internal counter. The acquire() method decrements the counter, and the release() method increases the counter. The value of the counter will never be less than zero. When acquire() is called, if the counter is found to be zero, the thread will be blocked. , until the release() method is called to increase the counter.

threading.Semaphore(value=1): The default value of the value parameter is 1. If the specified value is less than 0, a ValueError will be reported. A semaphore object manages an atomic counter that represents the number of calls to the release() method minus the number of calls to the acquire() method, plus an initial value.

acquire(blocking=True, timeout=None): By default, on entry, if the counter is greater than 0, decrement it by 1 and return True, if it is equal to 0, block until awakened using the release() method, then decrement it by 1 and return it True. The order of threads being awakened is undefined. If blocking is set to False, calling this method will not block. timeout is used to set the timeout time. If the semaphore is not obtained within timeout seconds, it returns False, otherwise it returns True.
release(): Releases a semaphore and increases the internal counter by 1. When the counter value is 0 and other threads are waiting for it to be greater than 0, wake up this thread.

Simple example of semaphore object:


#通过信号量对象管理一次性运行的线程数量

import time
import threading

# 创建信号量对象，初始化计数器值为4
semaphore4 = threading.Semaphore(4)


def thread_semaphore(index):
    # 信号量计数器减1
    semaphore4.acquire()
    time.sleep(2)
    print('thread_%s is running...' % index)
    # 信号量计数器加1
    semaphore4.release()


def main():
    # 虽然会有9个线程运行，但是通过信号量控制同时只能有4个线程运行
    # 第5个线程启动时，调用acquire发现计数器为0了，所以就会阻塞等待计数器大于0的时候
    for index in range(9):
        threading.Thread(target=thread_semaphore, args=(index, )).start()


if __name__ == '__main__':
    main()

6. Condition variable object: threading.Condition

threading.Condition(lock=None): A condition variable object allows one or more threads to wait until notified by another thread. The lock parameter must be a Lock object or RLock object and will be used as the underlying lock. RLock is used by default.

Its wait() method releases the lock and blocks the program until other threads call the notify() or notify_all() method to wake up, and then the wait() method reacquires the lock. This method can also specify the timeout timeout.
Its notify() method wakes up a waiting thread, and notify_all() wakes up all waiting threads. notify() or notify_all() will not release the lock, so the awakened threads will not immediately return from their wait() method and execute. Only the thread that called the notify() or notify_all() method will give up. The corresponding thread will be returned and executed after taking ownership of the lock, that is, notifying first and then releasing the lock.

wait(timeout=None): Release the lock and wait until being notified (reacquiring the lock) or a timeout event occurs. If the thread itself does not have a lock when calling this method (that is, the thread must have a lock first), a RuntimeError will be reported. This method releases the underlying lock, and then blocks the thread until the same condition variable in another thread is awakened using notify() or notify_all(), or a timeout event occurs. Once awakened or times out, it will reacquire the lock and return ( Returns True on success, False otherwise). The timeout parameter is the number of seconds in floating point type. Using the release method once in RLock may not release the lock, because the lock may be acquired() multiple times. However, in the condition variable object, it calls the internal method of the RLock class, which can completely release the lock at one time and reacquire it. The lock's recursion level is also reset when a lock is taken.
notify(n=1): Wake up a thread waiting for this condition. If the thread calling this method calls this method without acquiring the lock, a RuntimeError will be reported. By default, one thread is woken up. You can set the parameter n to wake up n threads waiting for this condition variable. If no thread is waiting, nothing will happen when calling this method. If there are exactly n threads among the waiting threads, then this method can accurately wake up these n threads. However, if the number of waiting threads exceeds the specified n, sometimes more than n threads may be awakened, so relying on the parameter n is not appropriate. safe behavior.
notify_all(): Wake up all threads waiting for this condition. The difference between this method and notify() is that it wakes up all threads, rather than specific n.

Simple example of condition variable:


#让一个线程等待，直到另一个线程通知
import time
import threading

# 创建条件变量对象
condition_lock = threading.Condition()

PRE = 0
# predicate可调用函数
def pre():
    print(PRE)
    return PRE

def test_thread_1():
    # 在使用wait/wait_for之前必须先获得锁
    condition_lock.acquire()

    print('等待线程test_thread_2的通知')
    # 先执行一次pre，返回False后释放掉锁，等另一个线程释放掉锁后再次执行pre，返回True后再次获取锁
    # wait_for的返回值不是True和False，而是predicate参数的返回值
    condition_lock.wait_for(pre)
    # condition_lock.wait()
    print('继续执行')

    # 不要忘记使用wait/wait_for之后要释放锁
    condition_lock.release()

def test_thread_2():
    time.sleep(1)
    condition_lock.acquire()

    global PRE
    PRE = 1
    print('修改PRE值为1')

    print('通知线程test_thread_1可以准备获取锁了')
    condition_lock.notify()

    # 先notify/notify_all之后在释放锁
    condition_lock.release()
    print('你获取锁')

def main():
    thread_1 = threading.Thread(target=test_thread_1)
    thread_2 = threading.Thread(target=test_thread_2)
    thread_1.start()
    thread_2.start()

if __name__ == '__main__':
    main()

7. Event object: threading.Event

An event object manages an internal flag. The initial state defaults to False. The set() method can set it to True. The clear() method can set it to False. The wait() method blocks the thread until the value of the internal flag is True. .

If one or more threads need to know a certain state of another thread before proceeding to the next step, they can use the thread's event object to handle it.

method:

is_set(): Returns True when the internal flag is True.
set(): Set the internal flag to True. At this time, all waiting threads will be awakened, and the thread calling the wait() method will not be blocked.
clear(): Set internal flag to False. All threads calling the wait() method will be blocked until the set() method is called to set the internal flag to True.
wait(timeout=None): Blocks the thread until the internal flag is True, or a timeout event occurs. If the internal flag is True when called, it will not be blocked, otherwise it will be blocked. timeout is the number of seconds in floating point type.

Simple example of event object:


#事件对象使用实例
import time
import threading

# 创建事件对象，内部标志默认为False
event = threading.Event()


def student_exam(student_id):
    print('学生%s等监考老师发卷。。。' % student_id)
    event.wait()
    print('学生%s开始考试了！' % student_id)


def invigilate_teacher():
    time.sleep(5)
    print('考试时间到，学生们可以开始考试了！')
    # 设置内部标志为True，并唤醒所有等待的线程
    event.set()


def main():
    for student_id in range(4):
        threading.Thread(target=student_exam, args=(student_id, )).start()

    threading.Thread(target=invigilate_teacher).start()


if __name__ == '__main__':
    main()

8. Producer and consumer examples

from threading import Thread,current_thread
import time
import random
from queue import Queue

#定义队列
queue = Queue(5)

#定义生产者
class ProducerThread(Thread):
    def run(self):
        name = current_thread().getName()
        nums = range(100)
        global queue
        while True:
            num = random.choice(nums)
            queue.put(num)
            print('生产者 %s 生产了数据 %s' %(name, num))
            t = random.randint(1,3)
            time.sleep(t)
            print('生产者 %s 睡眠了 %s 秒' %(name, t))

#定义消费者
class ConsumerTheard(Thread):
    def run(self):
        name = current_thread().getName()
        global queue
        while True:
            num = queue.get()
            queue.task_done()
            print('消费者 %s 消耗了数据 %s' %(name, num))
            t = random.randint(1,5)
            time.sleep(t)
            print('消费者 %s 睡眠了 %s 秒' % (name, t))


p1 = ProducerThread(name = 'p1')
p1.start()
p2 = ProducerThread(name = 'p2')
p2.start()
p3 = ProducerThread(name = 'p3')
p3.start()
c1 = ConsumerTheard(name = 'c1')
c1.start()
c2 = ConsumerTheard(name = 'c2')
c2.start()

Source code download

If this document is not detailed enough, you can refer to learn python in ten minutes_bilibili_bilibili