Concurrency in python: multiprocessing and multithreading

multi-Progress

official document

Multithreading

Multithreading in python is mainly implemented through the thread module

This module provides low-level primitives for manipulating multiple threads (also known as lightweight processes or tasks) - multiple threads of control sharing global data space. To handle synchronization issues, simple locking mechanisms (also known as mutexes or binary semaphores) are also provided. The threading module provides an easier-to-use high-level multithreading API based on this module.

Multi-threaded example

Create and run multiple threads in Python:

import threading

# 定义要执行的线程任务函数
def task():
    print("Thread is running")

# 创建线程对象
thread = threading.Thread(target=task)

# 启动线程
thread.start()

# 等待线程结束
thread.join()

print("Thread execution completed")

In this example, we first define a thread task function called task that prints a message when executed. Then, we created a thread object using the threading.Thread class and passed the task function as the target to the thread object. Next, start the thread by calling the start method of the thread object. Finally, use the join method to wait for the thread to finish executing.
In practice, you can create multiple thread objects and execute their tasks concurrently. At the same time, pay attention to the data sharing and synchronization mechanism between threads to ensure thread safety.
It should be noted that multithreading in Python is based on thread switching and is not suitable for particularly time-consuming CPU-intensive tasks , because the global interpreter lock (GIL) in Python limits the parallel execution of multiple threads. If you need to process CPU-intensive tasks or improve concurrent performance, you can consider using the multiprocessing module to achieve multi-process parallelism.

lock mechanism

In Python, to achieve multi-threaded mutual exclusion and synchronization, you can use the threading.Lock object to create a mutex. Mutex locks ensure that only one thread can access a critical resource at any given time, avoiding race conditions and data inconsistencies. The following is an example showing how to use a mutex to implement mutual exclusion and synchronization of multiple threads:

import threading
# 创建互斥锁对象

lock = threading.Lock()
counter = 0

def increment():
    global counter
    for _ in range(100000):
        # 获取互斥锁
        lock.acquire()
        counter += 1
        # 释放互斥锁
        lock.release()


# 创建多个线程并执行任务
threads = []
for _ in range(5):
    t = threading.Thread(target=increment)
    threads.append(t)
    t.start()

# 等待所有线程执行完毕
for t in threads:
    t.join()

# 打印计数器的值
print("Counter:", counter)

In this example, we create a mutex object lock and define a global counter counter. Then, we wrote an increment function as the thread's task, which increments the counter each time in the loop. Before each increment, threads acquire the mutex to ensure that only one thread can access and modify the counter's value, and then release the mutex.

After creating multiple thread objects, we start these threads and use the join method to wait for them to finish executing. Finally, we print the value of the counter to check that the mutex and synchronization are working properly.

By using mutexes, we ensure that threads have exclusive access to shared resources, preventing data races and inconsistencies. This ensures safe execution of multiple threads and provides a synchronization mechanism.