Python multithreaded programming: how to gracefully close threads

In concurrent programming, we may create a new thread, run tasks in it, and may decide to stop the thread for some reason. For example:

• The result of the thread task is no longer needed.
• Application is closing.
• An exception may have occurred in thread execution

For python multi-threaded programming knowledge, please refer to Master Python multi-threaded programming from shallow to deep

The Thread class of the Threading module does not provide a method to close a thread. If the child thread is not properly closed, the following problems may be encountered:

• After the main thread is terminated, the child thread is still running and becomes a zombie process
• The file opened by the child thread failed to close properly, resulting in data loss
• The database opened by the child thread failed to submit the update, resulting in data loss

So how to close the thread correctly?

1. How Python closes threads by default

After the thread object is created, call the start (method to run, and after the execution is completed, it will be automatically closed. As in the following sample code:

#!/usr/bin/python
# -*- coding: UTF-8 -*-

import threading    #导入threading 模块
import time

# 定义任务函数 print_time
def print_time ( threadName ,delay ):
    count = 0
    while count < 5:
        time.sleep(delay)
        count += 1
        print("%s: %s \n" % (threadName,time.ctime(time.time())))

# 定义任务函数 print_cube
def print_cube(num):
    #pring cube
    print("Cube:{} \n".format(num*num*num))
 
# 创建两个线程
if __name__ == "__main__":
        # 创建两个子线程
        t1 = threading.Thread( target=print_cube,args=(10,))
        t2 = threading.Thread( target=print_time,args=("Thread-2",4,))
        #start threads
        t1.start()   # start 后，子线程开始运行
        t2.start()
        t1.join()     #join 命令：让主线程暂停运行，等待子线程运行结束。
        t2.join()
        print("Done") # The statement is executed after sub threads done

2. How to close the thread gracefully?

In the example in the previous section, the execution time of the thread is short and can end soon, so the main thread can wait for it to end. However, if the child thread executes a time-consuming task, such as providing a service or executing a Monitor task, there may be a permanent loop in the child thread. At this time, after the child thread object runs start(), it will continue to process the running state .

In WIndows system, if the application directly exits, the sub-thread will naturally be forcibly terminated, but the tasks being executed by the sub-thread may be affected, such as the file being accessed may be closed correctly, resulting in data loss, etc.
In the Linux system, if the application exits directly, such as using the kill command to kill the process, the sub-threads that are not properly closed may still be running and become zombie processes.

So how to gracefully stop the child thread? There are two ideas:
1) Close the thread by setting the global state variable
2) Close the thread through the threading.Event object

The following example shows the implementation process of the two methods

2.1. Using global variables to close threads

Implementation steps:

• Add state variables inside the thread
• In the thread loop body, check the state variable, if it is False, exit the loop.
• When the main thread needs to close the thread, set the state variable of the child thread object to False.

2.1.1 Close the thread implemented by the thread class

class CountdownTask:
      
    def __init__(self):
          self._running = True   # 定义线程状态变量
      
	def terminate(self):
	    self._running = False 
	      
	def run(self, n):
	    # run方法的主循环条件加入对状态变量的判断
	    while self._running and n > 0:
	        print('T-minus', n)
	        n -= 1
	        time.sleep(5)
	    print("thread is ended") 
  
c = CountdownTask()
th = Thread(target = c.run, args =(10, ))
th.start()
# 对于耗时线程，没必要再用join()方法了，注意主线程通常也需要有个监控循环
# … any code … 
# Signal termination
q = input("please press any key to quit ")
c.terminate() 

2.1.2 Closing functional threads

Close functional threads, you can use global variables as state variables

import threading
import time
 
def run():
    while True:
        print('thread running')
        global stop_threads
        if stop_threads:
            break
 
stop_threads = False
t1 = threading.Thread(target = run)
t1.start()
time.sleep(1)
stop_threads = True
t1.join()
print('thread killed')

2.2. Use the threading.Event object to close the child thread

2.2.1 Working principle of Event mechanism

Events are a way of communicating between threads. Its function is equivalent to a global flag, and the main thread coordinates the pace of the sub-threads by controlling the state of the event object.

How to use

1. The main thread creates an event object and passes it as a parameter to the child thread
2. The main thread can use set()the method to eventset the object to true, and clear()the method to set it to false.
3. In the loop body of the child thread, check the value of the event object, and if it is True, exit the loop.
4. Child thread, you can use event.wait()to block the current child process until the event object is set to true.

Common methods of the event class

• set() sets True
• clear() sets False,
• wait() makes the process wait until the flag is changed to true.
• is_set() Query the event object, if it is set to true, it returns True, otherwise it returns False.

if event.is_set():
     # do something before end worker 
     break

The advantage of this method is that the Event object is thread-safe and faster. It is recommended to use this method to close time-consuming threads.

2.2.2 Complete code:

from time import sleep
from threading import Thread
from threading import Event
 
# define task function
def task(event):
    # execute a task in a loop
    for i in range(100):
        # block for a moment
        sleep(1)
        # check for stop
        if event.is_set():
            # 在此添加退出前要做的工作，如保存文件等
            break
        # report a message
        print('Worker thread running...')
    print('Worker is ended')
 
# create the event
event = Event()
# create a thread 
thread = Thread(target=task, args=(event,))
# start the new thread
thread.start()
# block for a while
sleep(3)
# stop the worker thread
print('Main stopping thread')
event.set()
# 这里是为了演示，实际开发时，主进程有事件循环，耗时函数不需要调用join()方法
thread.join()

The child thread executes its task loop, and it checks the event object every time it loops. If the object remains false, it will not trigger the thread to stop.

After the main thread calls the set() method of the event object, in the loop body of the sub-thread, call the is_set() method of the event object. After finding that the event object is True, it immediately exits the task loop and ends the operation.