Overview
In the previous section, we introduced how to use MySQL in Python, including: introduction to mysql.connector, functions of mysql.connector, using mysql.connector, etc. In this section, we will introduce how to use multithreading in Python. Multi-threading refers to a program running multiple threads at the same time, with each thread performing different tasks independently. In today's computer science field, multi-threading technology has become an important means to solve problems such as high concurrency and performance optimization. Python provides powerful multi-threading support through the built-in threading module. In practical applications, the reasonable use of multi-threads can help us improve program execution efficiency, achieve parallel computing, and optimize resource utilization and user experience.
Using Python's threading module, we can create and manage threads. Threads are the basic execution unit of a process, and they execute in parallel within the process. In Python, multithreading may not improve execution speed in CPU-intensive tasks due to the presence of the Global Interpreter Lock (GIL). However, in IO-intensive tasks, such as network requests, file reading and writing, etc., using multi-threading can significantly improve the execution efficiency of the program.
threading module
Python's threading module is used to provide thread support. The following are some commonly used functions and classes in the threading module.
threading.Thread(target, name, args, kwargs) : The main method to create a thread, target is the function to be executed, name is the name of the thread, args and kwargs are the parameters passed to the function.
threading.current_thread() : Returns the current thread object.
threading.enumerate() : Returns a list of all currently active Thread objects.
threading.active_count() : Returns the number of currently active Thread objects.
threading.Lock() : Thread lock, used to prevent multiple threads from accessing certain resources at the same time causing data confusion.
threading.RLock() : Reentrant thread lock, allowing a thread to acquire the same lock again while already holding the lock.
threading.Event() : used to create event objects for communication between threads.
threading.Condition() : Condition variable, used to make a thread wait until a specific condition is met.
threading.Semaphore() : Semaphore, used to limit the number of threads accessing a specific resource at the same time.
threading.BoundedSemaphore() : A bounded semaphore. Unlike Semaphore, it limits the upper limit of the semaphore.
threading.Timer(interval, function, args, kwargs) : Execute an operation after the specified time interval.
threading.local() : Create a thread-local data object, each thread has its own copy of the data.
Use threads
In Python's threading module, the Thread class is an object used to create threads. A basic Thread object can be created by referring to the sample code below.
import time
import threading
def print_numbers():
for i in range(5):
time.sleep(1)
print('number is:', i)
# 创建线程
t = threading.Thread(target = print_numbers)
# 启动线程
t.start()
# 等待线程结束
t.join()In the above example code, we define a function print_numbers(), and then create a new Thread object. The target function is print_numbers(). Calling t.start() will start this thread, and then your function will automatically run in the new thread. Calling t.join() will wait for the thread to finish executing. In this example, it will wait for the for loop in the print_numbers() function to finish executing.
Of course, you can also create multiple threads at the same time, see the sample code below.
import time
import threading
def print_numbers():
name = threading.current_thread().name
for i in range(5):
time.sleep(1)
print('%s, number is: %d'%(name, i))
# 创建线程
t1 = threading.Thread(target = print_numbers, name = 'thread 1')
t2 = threading.Thread(target = print_numbers, name = 'thread 2')
# 启动线程
t1.start()
t2.start()
# 等待线程结束
t1.join()
t2.join()As you can see, after the above example code is run, thread 1 and thread 2 will alternately output information. The output is as follows:
thread 2, number is: 0
thread 1, number is: 0
thread 1, number is: 1
thread 2, number is: 1
thread 1, number is: 2
thread 2, number is: 2
thread 1, number is: 3
thread 2, number is: 3
thread 1, number is: 4
thread 2, number is: 4In addition to threading.Thread, we can also use threading.Timer to run specified tasks in threads. threading.Timer is mainly used to create a timer to perform an operation after a specified time interval. It should be noted that threading.Timer runs in a new thread. If the function involves modifying shared data and resources, appropriate synchronization mechanisms may need to be used to avoid concurrency problems.
import threading
def print_msg():
print("Hello CSDN")
# 创建定时器，3秒后执行print_msg函数
timer = threading.Timer(3, print_msg)
# 开始计时器
timer.start()In the above example code, the program will wait for 3 seconds and then print the string of "Hello CSDN".
Create custom thread
In Python, you can create a custom thread class by inheriting the threading.Thread class. In a custom thread class, some functions are usually overridden to change the default behavior, such as the run() function.
import threading
# 自定义线程类
class MyThread(threading.Thread):
def __init__(self, data):
threading.Thread.__init__(self)
self.data = data
def run(self):
# 输出：Hello CSDN
print(f"Hello {self.data}")
# 创建自定义线程类的对象
my_thread = MyThread('CSDN')
# 启动线程
my_thread.start()
# 等待线程结束
my_thread.join()In the sample code above, we created a new class called MyThread and inherited threading.Thread. In the init function of the MyThread class, the init function of the parent class threading.Thread is first called to initialize, and then an attribute named data is set. We rewrote the run() function so that when my_thread.start() is called, our custom run() function will be executed instead of the parent class's.
Below is a more complex example of creating and using a custom thread class.
import time
import threading
class MyThread2(threading.Thread):
def __init__(self, thread_id, name):
threading.Thread.__init__(self)
self.thread_id = thread_id
self.name = name
def run(self):
print(f"start thread: {self.name}")
for i in range(5):
time.sleep(1)
print(f"thread {self.name} is running")
print(f"exit sub thread: {self.name}")
# 创建线程对象
threads = []
for i in range(3):
thread = MyThread2(i, f"Thread-{i}")
thread.start()
threads.append(thread)
# 等待所有线程完成
for t in threads:
t.join()
print('exit main thread')In the above example code, the MyThread2 class has a constructor that accepts a thread ID and a name as parameters and internally calls the constructor of the parent class threading.Thread. The run() function is rewritten to perform the task we want the thread to do: print a message, then sleep for a second, print another message, repeat 5 times. Finally, we create 3 threads and start them, then wait for all threads to complete their tasks.
Thread synchronization
Multi-threaded programming may cause some potential problems, such as data inconsistency, race conditions, etc. In order to solve these problems, we need to use thread synchronization technology. Thread synchronization is a mechanism for coordinating the execution of multiple threads to ensure that they share resources or cooperate correctly and efficiently. Python provides several thread synchronization mechanisms, including: Lock, Event, Condition, and Semaphore. Below, they will be introduced separately.
1. Lock is the most basic thread synchronization mechanism. In Python, we can use the threading.Lock class to achieve this. Locks have two states: locked and unlocked. When one thread acquires a lock, other threads trying to acquire the lock will be blocked until the lock is released.
import threading
lock = threading.Lock()
def thread_func():
with lock:
# 线程安全的代码块
pass
def thread_func2():
lock.acquire()
# 线程安全的代码块
lock.release()In the above example code, the thread_func() function uses with lock to lock shared resources, and the thread_func() function uses lock.acquire() and lock.release() to lock shared resources.
Let's take a look at sample code for using locks in multiple threads.
import time
import threading
class Counter:
def __init__(self):
self.count = 0
self.lock = threading.Lock()
def increment(self):
with self.lock:
self.count += 1
print(f"Count: {self.count}")
def worker(counter):
for _ in range(10):
counter.increment()
counter = Counter()
threads = []
for _ in range(3):
t = threading.Thread(target = worker, args = (counter,))
threads.append(t)
t.start()
for t in threads:
t.join()
print(f"Final Count: {counter.count}")In the above sample code, we defined a Counter class, which contains a counter and a lock. The increment function acquires the lock before incrementing the counter and releases the lock when completed, ensuring that only one thread can modify the counter at any time. We then created 3 threads, each of which would try to increment the counter 10 times. Because of the lock, all increments will be serialized correctly and the final count will always be 30.
2. Events are used for communication between threads. The threading.Event class provides a signal flag that a thread can set, and other threads can wait for this flag to be set.
import threading
event = threading.Event()
def thread_func():
# 阻塞线程，直到事件被设置
event.wait()
# 当事件被设置后执行的代码
print('event waited')
event.set()
thread_func()In the above sample code, the event.wait() function will be blocked when called. Only when event.set() is called elsewhere to set the event signal, the blocking will be unblocked and the code will continue to execute.
3. Condition (Condition) is used for more complex thread synchronization issues. The threading.Condition class provides a way to wait for a certain condition to be met. It is usually used with a lock.
import time
import threading
class SharedData:
def __init__(self):
self.lock = threading.Lock()
self.condition = threading.Condition(self.lock)
self.value = 0
def increment(self):
while True:
self.condition.acquire()
while self.value >= 10:
self.condition.wait()
self.value += 1
print(f"Value increased to {self.value}")
self.condition.notify_all()
self.condition.release()
time.sleep(1)
def decrement(self):
while True:
self.condition.acquire()
while self.value <= 0:
self.condition.wait()
self.value -= 1
print(f"Value decreased to {self.value}")
sleep_time = 2 if self.value <= 5 else 0.5
self.condition.notify_all()
self.condition.release()
time.sleep(sleep_time)
sd = SharedData()
thread1 = threading.Thread(target = sd.increment)
thread2 = threading.Thread(target = sd.decrement)
thread1.start()
thread2.start()
thread1.join()
thread2.join()In the sample code above, the SharedData class has a value attribute, and a Condition object. Both the increment and decrement methods use Condition to ensure that the value is always between 0 and 10. If the value exceeds this range, the current thread will call the wait method, put itself into the waiting queue, and release the lock to give other threads a chance to execute. When the value returns to the valid range, the thread will call the notify_all method to wake up all threads in the waiting queue.
4. Semaphore is used to restrict access to resources. The threading.Semaphore class provides a counter to control the number of threads that can access a resource simultaneously.
import time
import threading
# 创建一个Semaphore，最大允许3个线程同时访问共享资源
semaphore = threading.Semaphore(3)
def MyWorker():
# 获取Semaphore
semaphore.acquire()
# 访问共享资源的代码
for i in range(6):
print("MyWorker {} is working: {}".format(threading.current_thread().name, i))
time.sleep(1)
# 释放Semaphore
semaphore.release()
# 创建5个线程
threads = []
for i in range(5):
t = threading.Thread(target = MyWorker, name = str(i))
t.start()
threads.append(t)
# 等待所有线程完成
for t in threads:
t.join()In the above sample code, we created a Semaphore, allowing up to 3 threads to access shared resources at the same time. Each thread acquires the Semaphore before accessing the shared resource and releases the Semaphore when completed. In this way, we can ensure that only a maximum of 3 threads are accessing the shared resource at any time.