Python processes, multi-processes, threads, and synchronization and deadlocks

A shortcoming of traditional programming

Disadvantages of traditional programming:

# 必须按照顺序执行，多个任务无法同时在还行
import time


def sing():
    for i in range(5):
        print("sing: hero")
        time.sleep(1)       # 每唱一次，等1秒再唱


def dance():
    for i in range(5):
        print("dance: swan")
        time.sleep(1)       # 每唱一次，等1秒再跳


def main():
    sing()
    dance()


if __name__ == "__main__":
    main()

The time spent on the two tasks is 10 seconds. If you want to dance and sing at the same time, in fact, the two tasks can be completed when the longest task is completed.

There are many ways to implement multi-task programming, such as: multi-process, multi-thread, coroutine, etc.

2. Use multi-process method to achieve multi-tasking

# 必须按照顺序执行，多个任务无法同时在还行
import time
import multiprocessing


def sing():
    for i in range(5):
        print("sing: hero")
        time.sleep(1)       # 每唱一次，等1秒再唱


def dance():
    for i in range(5):
        print("dance: swan")
        time.sleep(1)       # 每唱一次，等1秒再跳


def main():
    p1 = multiprocessing.Process(target=sing)
    p2 = multiprocessing.Process(target=dance)
    p1.start()
    p2.start()


if __name__ == "__main__":
    main()

Some operations of the three processes

htopA detailed process list can be viewed through the command.

Note: kill -9 pidAfter killing the main process, the child process will not be killed. At this time, the command line will not be able to exit normally because the signal of this command is sent to the main process to perform the killing task. The child process does not have a parent process. , became an orphan process, and was later adopted by the init process. That is to say, after killing the main process, the parent process of the child process is called the init process.

Through , os.getpid()you can get the pid of the current process and os.getppid()the id of the parent process.

Four inter-process communication

Processes cannot communicate directly with each other because they are independent applications.

To achieve communication between processes, common methods include: socket, etc. In Python, you can use the queue method to achieve:

queue = multiprocessing.Queue(3)
queue.put("111")
queue.put(222)

# 取数据
res = queue.get()
print(res)
# 判断： q.full()  q.empty()

Five process pool

p = multiprocessing.Pool(3)

Six multi-threading methods to achieve multi-tasking

import time
import threading


def sing():
    for i in range(5):
        print("sing: hero")
        time.sleep(1)       # 每唱一次，等1秒再唱


def dance():
    for i in range(5):
        print("dance: swan")
        time.sleep(1)       # 每唱一次，等1秒再跳


def main():
    t1 = threading.Thread(target=sing)      # 创建一个线程对象
    t2 = threading.Thread(target=dance)     # 创建一个线程对象
    t1.start()                              # 开启线程
    t2.start()                              # 开启线程


if __name__ == "__main__":
    main()

Seven thread related API

threading.Thread() Threads can be created and threading.enumerate()all current threads can be viewed:

import time
import threading


def sing():
    for i in range(5):
        print("sing: hero")
        time.sleep(1)       # 每唱一次，等1秒再唱


def dance():
    for i in range(5):
        print("dance: swan")
        time.sleep(1)       # 每唱一次，等1秒再跳


def main():

    t1 = threading.Thread(target=sing)
    t2 = threading.Thread(target=dance)

    t1.start()
    t2.start()

    while True:
        length = len(threading.enumerate())
        print("当前线程数为：%d" % length)
        if length <= 1:
            break
        time.sleep(0.5)


if __name__ == "__main__":
    main()

8. Synchronization and Deadlock

Threads share global variables, which will cause data confusion:

import time
import threading

# 定义共享的全局变量
num = 0


def add100():
    global num
    for i in range(100000):
        num = num + 0.00001


def add1000():
    global num
    for i in range(100000):
        num = num + 1000


def main():

    t1 = threading.Thread(target=add100)
    t2 = threading.Thread(target=add1000)

    t1.start()
    t2.start()

    time.sleep(5)
    print(num)              # 每次输出的结果是不相同的


if __name__ == "__main__":
    main()

Solution to use mutex locks to achieve synchronization:

import time
import threading

# 定义共享的全局变量
num = 0


def add100():
    global num
    mutex.acquire()     # 加锁：若已经加锁，则会直到锁被揭开
    for i in range(100000):
        num = num + 0.00001
    mutex.release()     # 解锁


def add1000():
    global num
    mutex.acquire()     # 加锁：若已经加锁，则会直到锁被揭开
    for i in range(100000):
        num = num + 1000
    mutex.release()     # 解锁


# 创建互斥锁，默认不会上锁
mutex = threading.Lock()


def main():

    t1 = threading.Thread(target=add100)
    t2 = threading.Thread(target=add1000)

    t1.start()
    t2.start()

    time.sleep(5)
    print(num)          # 100000001.0 永远不会变


if __name__ == "__main__":
    main()