1. The reason and solution of time.sleep() suspended animation (hang)

Recently, I used chatGPT to help write the program and completed the program to obtain weather data through the API. After running, something happened: inexplicable suspended animation (the program was suspended and not executed), and it was easier to hang in the Linux environment.

For details on the procedure, see the author's blog post " Using chatGPT to develop and obtain gridded weather data ".

At first, my purpose was to develop a program for temporary access to meteorological data for recent use, and the business application was implemented by professionals in its application, without considering the content of thread development. Under the reminder of chatGPT, I wrote a thread service program in a targeted manner.

Note: Both are chatGPT solutions, this time, thread content is added.

main reason:

Try to avoid using the sleep() function in the main thread, because this will cause the program to block and cannot be woken up. If you must use the sleep() function in the main thread, consider submitting the task to a separate thread.
insert image description here
Solution:

Adopt thread programming method.

Through communicating with chatGPT, finally, under my strong request, the "start_thread():" function was completed, the main code is as follows:

import requests
import csv
import time
import datetime
import threading

# API接口
url = "https://devapi.qweather.com/v7/grid-weather/now"

# 请求参数，location为经纬度坐标，key为API密钥
params = {
    
    
    "location": "103.59984,36.18057",
    "key": "****************"
}

fieldnames = [ 'getTime', 'obsTime', 'temp', 'icon', 'text', 'wind360', 'windDir', 'windScale',
              'windSpeed', 'humidity', 'precip', 'pressure', 'cloud', 'dew' ,'status']
# csv文件名
filename = 'qweather_data.csv'
logname = 'log2.txt'
time_status = ["00", "30"]

# 获取天气数据
def get_weather_data():
	while True:
		try:        
			# 获取当前时间
			current_time = time.strftime('%Y-%m-%dT%H:%M:%S', time.localtime())

			# 判断是否为整点或半点
			if current_time.split(":")[1] in time_status:     
				# 发送API请求并获取返回结果
				response = requests.get(url, params=params)
				result = response.json()

				# 提取关键数据并写入csv文件
				now_data = result['now']
				......				
			
			# 等待到整点或半点
			now = time.localtime()
			if now.tm_min < 30:
				wait_time = (30 - now.tm_min) * 60 - now.tm_sec
			else:
				wait_time = (60 - now.tm_min) * 60 - now.tm_sec
			
			print(f"{
      
      current_time}: Server is Runing, Wait_time: {
      
      wait_time}s\n")
			time.sleep(wait_time)
	   
		except requests.exceptions.ConnectionError as e:
			print(f"{
      
      current_time}: {
      
      e}. Retrying in 5 seconds...")

			....
			now = time.localtime()
			if now.tm_min < 5:
				wait_time = (5 - now.tm_min) * 60 - now.tm_sec
			else:
				wait_time = (35 - now.tm_min) * 60 - now.tm_sec

			time.sleep(wait_time)
			# time.sleep(30)
		except Exception as e:
			print(f"{
      
      current_time}: {
      
      e}.")
			break    

def start_thread():
    t = threading.Thread(target=get_weather_data)
    print('start server ... ')
    t.start()
    t.join()
    
start_thread()

In the start_thread() function, we create a new thread t and start it. The objective function of t is get_weather_data(), which is the function of obtaining weather data. Then, we call t.join() to wait for thread t to finish. Since our thread t never ends, there won't actually be any effect here.

This program will fetch weather data every hour and half hour, and print a successful message. When an exception occurs while acquiring data, the exception information will be printed. Note that in order to avoid occupying CPU resources for unnecessary time, we use the time.sleep() function to wait for a certain time between the hour and the half hour.

2. Summary

Use time.sleep(), preferably using the thread method;
Reasonable guidance of chatGPT communication can help us solve problems quickly.

3. chatGPT exchange excerpt

chatGPT analysis source code

I understand, then this problem may be related to some particularities of the Linux environment, such as the restrictions on processes in Linux. You can try the following methods to solve this problem:

Check your code for problems such as infinite loops or deadlocks, which may cause the program to fail to exit normally.
Use nohupthe command to run the program in the background and output the log to a file, so that the program can keep running even if you close the terminal. For example:nohup python3 your_program.py > log.txt &
Check the restrictions on processes in the Linux environment, such as whether there are restrictions on the number of processes and open files, and if so, modify these restrictions appropriately. You can use ulimit -athe command to view the current system limitations.
Use some system-level process management tools to monitor your program, such as systemdor supervisor, these tools can help you monitor the status of the program and automatically restart the program when the program exits abnormally.

Explain the principle of sleep:

In Python, the sleep function is a method used to block a program for a period of time. When the program calls the sleep function, it will suspend the current thread, let the CPU execute other threads or processes, wait for the specified time, and then wake up the current thread to continue execution.

The blocking of the sleep function is a relatively violent method, which will occupy the CPU time slice of the current thread, so that other threads cannot obtain the CPU time slice to perform tasks. Therefore, if the sleep function is used in the main thread, it may cause the performance of the entire program to degrade.

In order to avoid the sleep function from blocking the main thread, we can use threads to solve it. In the case of threads, time-consuming operations can be performed in sub-threads without affecting the operation of the main thread. In the child thread, you can use the sleep function to wait for a certain period of time without affecting the operation of other threads.

The following is an example of using threads to solve sleep blocking:

import threading
import time

def worker():
    while True:
        print("Worker is running...")
        time.sleep(1)

if __name__ == '__main__':
    t = threading.Thread(target=worker)
    t.start()
    while True:
        print("Main thread is running...")
        time.sleep(1)

In this example, we create a worker thread that will run forever and output every second. At the same time, the main thread will always run and output once every second. In the main thread, we use the sleep function to block for one second instead of using the sleep function in the worker thread.

The advantage of this is that the worker thread can let the CPU execute the main thread while blocking, avoiding the problem that the sleep function blocks the main thread. Moreover, the use of threads can also make the program more scalable, and can easily add or delete sub-threads.

It should be noted that if there are shared resources in the thread, locks or other synchronization mechanisms need to be used to ensure thread safety. In addition, if there are long-term blocking operations in the thread, such as network I/O or disk I/O, it is recommended to use asynchronous programming to avoid blocking.

In the Linux environment, using the sleep function in the main thread can easily cause the program to hang and cannot be woken up. Is there a way to wake it up? Please explain in detail.

In the Linux environment, using the sleep function will block the current process, let the CPU sleep for a specified time, and continue to execute after waiting for the time to expire. If some kind of signal (such as SIGINT or SIGTERM) occurs during the blocking period, it may not wake up the program in the expected time. This is because while blocked, the process cannot handle signals.

In order to solve this problem, we can use signal processing function and sigwait function to realize. The sigwait function allows us to wait for one or more specific signals in our program and block the execution of the program until the signal is received.

Here is a sample code showing how to use the sigwait function instead of the sleep function:

import signal
import threading

def signal_handler(signum, frame):
    """信号处理函数"""
    print('Received signal %d' % signum)

def wait_for_signal(signum):
    """等待信号的线程函数"""
    signal.sigwait([signum])

# 注册信号处理函数
signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)

# 创建等待信号的线程
signal_thread = threading.Thread(target=wait_for_signal, args=(signal.SIGINT,))
signal_thread.start()

# 主线程继续执行其他操作
while True:
    print('Doing something...')

In this example, we create a thread waiting for a signal, use the sigwait function to block the execution of the thread, and wait for the SIGINT signal. When a SIGINT signal is received, the signal handler is called and a message is printed. In the main thread, we use a simple loop to print a message to simulate other operations of the program. In this way, we can continue to perform other operations in the main thread without using a blocking sleep function.

It should be noted that if the sigwait function is used to wait for multiple signals, the blocking will be lifted only when any one of the signals is triggered. Therefore, if you need to wait for multiple signals, you need to register a signal handler function to each signal separately, and set a flag in the handler function to record the received signal.

reference:

louis. Process pool blocking process and non-blocking process. Zhihu. 2022.05
Xiao Yongwei. Using chatGPT development to obtain grid weather data . CSDN blog. 2023.04

chatGPT gives a solution to Python time.sleep() suspended animation (hang)

1. The reason and solution of time.sleep() suspended animation (hang)

2. Summary

3. chatGPT exchange excerpt

Guess you like