Table of contents
1. Demand
The python main process evokes a python sub-process, which will print information in real time when the sub-process is running, and the main process needs to obtain the real-time information of the sub-process.
Here, two situations need to be distinguished: when there are few sub-process messages, such as a sub-process written by myself, which uses print to print part of the information, there is no need to consider the caching problem at this time; when there are many sub-process messages and the output speed is fast, For example, in the training of the yolov5 model, there is a progress bar display in the output, and the data volume is large and the speed is fast. At this time, the caching problem needs to be considered.
Mainly refer to the blog "How to start a new process in python and get the output of the program in real time." and the blog "The use of python subprocess.Popen"
2. Implementation without caching
This situation is applicable when there are few sub-process messages, such as a sub-process written by myself, which uses print to print part of the information.
2.1. Subprocess test code
This code is the subprocess test code testSub.py, and print is used to output the code.
import time
i = 0
while (i<40):
print("当前处于第{}秒".format(i))
time.sleep(1)
i += 1
2.2. Main process code
The code is the main process code testMain.py, use subprocess.popen to run the subprocess testSub.py, and obtain the real-time printing result of the subprocess.
import subprocess
import threading
class CMDProcess(threading.Thread):
'''
执行CMD命令行的 进程
'''
def __init__(self, args,callback):
threading.Thread.__init__(self)
self.args = args
self.callback=callback
def run(self):
self.proc = subprocess.Popen(
self.args,
bufsize=0,
shell = False,
stdout=subprocess.PIPE
)
while self.proc.poll() is None:
line = self.proc.stdout.readline()
line = line.decode("utf8")
if(self.callback):
self.callback(line)
def getSubInfo(text):
print("子进程测试代码实时输出内容=>" + text)
def main():
cmd = [
'python',
'-u', # 注意，这里必须带上-u
'testSub.py'
]
print("子进程测试代码的运行命令：", ' '.join(cmd))
testProcess = CMDProcess(cmd,getSubInfo )
testProcess.start()
main()
3. Consider the implementation of caching
This situation is applicable to the sub-process with many messages and fast output speed, such as the training of yolov5 model, there is a progress bar display in the output, the data volume is large and the speed is fast.
Here, you only need to provide the main process code, please pay attention to the relevant comments. The code is the main process code testMain.py, use subprocess.popen to run the subprocess testSub.py, and obtain the real-time printing result of the subprocess.
import subprocess
import threading
class CMDProcess(threading.Thread):
'''
执行CMD命令行的 进程
'''
def __init__(self, args,callback):
threading.Thread.__init__(self)
self.args = args
self.callback=callback
self.cwd = './'
def run(self):
self.proc = subprocess.Popen(
self.args,
bufsize=1, # bufsize=0时，为不缓存；bufsize=1时，按行缓存；bufsize为其他正整数时，为按照近似该正整数的字节数缓存
shell = False,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, # 这里可以显示yolov5训练过程中出现的进度条等信息
text=True, # 缓存内容为文本，避免后续编码显示问题
cwd=self.cwd # 这个参数意思是，当前子进程结束后，其结果保存地址，比如yolov5训练进程结束后会输出模型、检测图片等，可在cwd中找到
)
while self.proc.poll() is None:
line = self.proc.stdout.readline()
self.proc.stdout.flush() # 刷新缓存，防止缓存过多造成卡死
#line = line.decode("utf8")
if(self.callback):
self.callback(line)
def getSubInfo(text):
print("子进程测试代码实时输出内容=>" + text)
def main():
cmd = [
'python',
'-u', # 注意，这里必须带上-u
'testSub.py'
]
# 这里可以改成yolov5的训练代码，如下
"""
cmd = [
'python',
'-u', # 注意，这里必须带上-u
'train.py',
'--img',
'640',
'--batch',
'4',
'--epochs',
'1' #等等，请自行添加参数
]
"""
print("子进程测试代码的运行命令：", ' '.join(cmd))
testProcess = CMDProcess(cmd,getSubInfo )
testProcess.start()
main()
With qt display, the effect is as follows
