Python uses multi-threaded examples to explain
1 Processes and threads
1.1 The concept of process and thread
Process (process) and thread (thread) are the basic concepts of the operating system.
A process is the smallest unit of resource allocation, and a thread is the smallest unit of CPU scheduling.
A thread is a single sequential control flow in a program, a relatively independent and schedulable execution unit in a process, and is the basic unit for the system to independently schedule and allocate CPUs.
1. What is a process?
Computer programs are nothing more than executable binary (or other types) data on disk. They only start their life cycle when they are read into memory and called by the operating system.
A process is an execution of a program. Each process has its own address space, memory, data stack, and other auxiliary data that records its running track. The operating system manages all processes running on it, and allocates time fairly for these processes. Processes can also complete other tasks through fork and spawn operations.
However, a process has its own memory space, data stack, etc., so it can only use inter-process communication (Inter Process communication, IPC), and cannot directly share information.
2. What is a thread?
A thread is somewhat similar to a process, but the difference is that all threads run in the same process and share the same operating environment.
A thread has three parts: start, sequential execution and end. It has its own instruction pointer, which records where it runs. The running of a thread may be preempted (interrupted) or temporarily suspended (sleeping) to allow other threads to run, which is called yielding.
All threads in a process share the same piece of data space, so threads can share data and communicate with each other more conveniently than processes. Threads are generally executed concurrently, and it is precisely because of this concurrency and data sharing mechanism that the cooperation of multiple tasks becomes possible.
In fact, in a single-CPU system, true parallelism is impossible. Each thread will be scheduled to run for a short time each time, and then the CPU will be released to let other threads run. During the entire running process of the process, each thread only does its own thing, and shares the running results with other threads when needed.
Of course, such sharing is not entirely without danger. If multiple threads jointly access the same piece of data, the order of data access may be different, which may lead to inconsistency of data results, that is, race condition. Likewise, most thread libraries come with a set of synchronization primitives to control thread execution and data access.
Another thing to note is that because some functions will block before completion, without special modifications for multi-threading, this "greedy" function will skew the CPU time allocation, resulting in the allocation of each thread Arrived runtimes may vary and not be fair.
1.2 The difference between process and thread
(1) Address space and other resources (such as open files): The processes are independent of each other and shared between threads of the same process. Threads in one process are not visible to other processes.
(2) Communication: Inter-process communication IPC, threads can directly read and write process data segments (such as global variables) to communicate - need the assistance of process synchronization and mutual exclusion means to ensure data consistency.
(3) Scheduling and switching: Thread context switching is much faster than process context switching.
(4) In a multithreaded OS, a process is not an executable entity.
To sum up, processes and threads can be compared to trains and carriages.
(1) Threads travel under processes (pure carriages cannot run).
(2) A process can contain multiple threads (a train can have multiple carriages).
(3) It is difficult to share data between different processes (passengers on one train are difficult to transfer to another train, such as station transfer).
(4) It is easy to share data between different threads under the same process (it is easy to change from car A to car B).
(5) Processes consume more computer resources than threads (using multiple trains consumes more resources than multiple carriages).
(6) The processes will not affect each other, and if one thread hangs up, the whole process will hang up (one train will not affect another train, but if a carriage in the middle of a train catches fire, it will affect the train (7 )
The process can be extended to multiple machines, and the process is most suitable for multi-core (different trains can run on multiple tracks, and the same train’s cars cannot be on different tracks) (8) The memory address used by the
process It can be locked, that is, when a thread uses some shared memory, other threads must wait for it to end before using this piece of memory (such as the toilet on the train [mutex]).
(9) The memory address used by the process can limit the amount of usage (such as the restaurant on the train, only how many people are allowed to enter at most, if it is full, you need to wait at the door, and wait for someone to come out before entering [semaphore semaphore]).
1.3 The concept and difference between multi-process and multi-thread
(1) A process is equivalent to a program to be executed. It will start a main thread, and multi-threading will start multiple sub-threads; (
2) Python did not have a multi-core CPU at the beginning of its design, so its multi-threading is a concurrent operation ( Pseudo-parallel), it is equivalent to dividing the time slice of the CPU into a small segment, and then assigning it to each thread to perform alternately. Since each segment is very short, it looks like a parallel operation.
For example: Now there is a 16-core CPU, and a data reading task A to be executed, we divide A into multiple processes to operate in parallel, and each process is placed on a core. But if this task A is completed with one process (opening multiple threads), although one core processes one thread at the same time, it stands to reason that 16 cores can process 16 threads at the same time (without considering hyper-threading technology), but due to the defects of python , the multithreading inside is still concurrent (pseudo-parallel), so the efficiency is low.
1.4 Python's global interpreter lock GIL
The execution of Python code is controlled by the Python virtual machine (also called the interpreter main loop). At the beginning of the setting, Python considers that only one thread is executing at the same time in the main loop of the interpreter, just like running multiple processes in a single-CPU system, multiple programs can be stored in the memory, but at any time, there is only one thread. Programs run on the CPU.
Although the Python interpreter can run multiple threads, only one thread is running in the interpreter at any time. Access to the Python virtual machine is controlled by the global interpreter lock (GIL), which ensures that only one thread is running at a time.
In a multithreaded environment, the Python virtual machine executes as follows:
(1) 设置GIL
(2) 切换到一个线程去运行
(3) 运行：
a. 指定数量的字节码的指令，或者
b. 线程主动让出控制（可以调用time.sleep(0)）
(4) 把线程设置为睡眠状态
(5) 解锁GIL
(6) 再次重复以上所有步骤
When calling external code (such as C/C++ extension functions), the GIL will be locked until the end of this function (since no Python bytecode is run during this period, no thread switching will be performed). Programmers writing extensions can actively unlock the GIL. Python developers, however, don't have to worry about your Python code getting locked up in these cases.
2 python multithreading instance
2.1 Ordinary single thread
# -*- coding: utf-8 -*-
from time import ctime,sleep
def music(name):
for i in range(2):
print("I was listening to music.----{}:{}".format(name, ctime()))
sleep(1)
def coding(code):
for i in range(2):
print("I was coding codes!----{}:{}".format(code, ctime()))
sleep(5)
if __name__ == '__main__':
music("my love music")
coding("python code")
print("all over.----{}".format(ctime()))
We first listened to a piece of music, and controlled the playback of the music twice through a for loop. Each piece of music takes 1 second to play, and sleep() controls the duration of the music. Then I typed the code again, each piece of code takes 5 seconds, and typed twice through the for loop.
2.2 Multithreading
Python provides two modules to implement multi-threaded thread and threading. Thread has some shortcomings, which are made up for in threading. We can just learn threading directly. Continue to modify the above example and introduce threadring to play music and write code at the same time:
2.2.1 Set daemon thread (do not wait)
In Python, a daemon thread refers to a thread that runs in the background while the program is running. When the main thread ends, the daemon thread also ends. Daemon threads are usually used to perform some tasks that do not need to block the main thread or run for a long time.
# -*- coding: utf-8 -*-
from time import ctime,sleep
import threading
def music(name):
for i in range(2):
print("I was listening to music.----{}:{}".format(name, ctime()))
sleep(1)
def coding(code):
for i in range(2):
print("I was coding codes!----{}:{}".format(code, ctime()))
sleep(5)
if __name__ == '__main__':
threads = []
t1 = threading.Thread(target=music, args=('my love music',))
threads.append(t1)
t2 = threading.Thread(target=coding, args=('python code',))
threads.append(t2)
for t in threads:
# setDaemon(True)将线程声明为守护线程,必须在start()方法调用之前设置
t.setDaemon(True)
t.start()
# 子线程启动后，主线程也继续执行下去
print("all over.----{}".format(ctime()))
Because it is a daemon thread, when the main thread executes the last statement print, it exits without waiting for the child thread, and the child thread also ends together.
Judging from the execution results, the sub-threads (muisc, coding) and the main thread (print all over) are started at the same time, but because the main thread finishes executing, the sub-threads also terminate.
If the main thread waits for 8 more seconds, it will output normally.
if __name__ == '__main__':
threads = []
t1 = threading.Thread(target=music, args=('my love music',))
threads.append(t1)
t2 = threading.Thread(target=coding, args=('python code',))
threads.append(t2)
for t in threads:
# setDaemon(True)将线程声明为守护线程,必须在start()方法调用之前设置
t.setDaemon(True)
t.start()
# 子线程启动后，主线程也继续执行下去
sleep(8)
print("all over.----{}".format(ctime()))
2.2.2 Do not set daemon thread (waiting)
if __name__ == '__main__':
threads = []
t1 = threading.Thread(target=music, args=('my love music',))
threads.append(t1)
t2 = threading.Thread(target=coding, args=('python code',))
threads.append(t2)
for t in threads:
t.start()
# 子线程启动后，主线程也继续执行下去
print("all over.----{}".format(ctime()))
After the execution of the main thread ends, the process will wait for the child thread to finish running before the process exits.
2.2.3 Thread blocking join method
A join() method is added to the above program to wait for the thread to terminate. The function of join() is that the parent thread of this child thread will be blocked until the child thread finishes running.
if __name__ == '__main__':
threads = []
t1 = threading.Thread(target=music, args=('my love music',))
t2 = threading.Thread(target=coding, args=('python code',))
threads.append(t2)
threads.append(t1)
for t in threads:
t.setDaemon(True)
t.start()
t.join()
# 子线程t1运行结束后,继续执行主线程
print("all over.----{}".format(ctime()))
Use the child thread t1 to block, and the operation ends after 2 seconds, and the thread t2 has not yet completed the operation.
