yolov5将训练好的模型(yolov5s.pt)转换成onnx格式,在使用转换后的onnx格式的权重进行推理时作者使用如下语句:
# Inference
if pt:
visualize = increment_path(save_dir / Path(path).stem, mkdir=True) if visualize else False
pred = model(img, augment=augment, visualize=visualize)[0]
elif onnx:
if dnn:
net.setInput(img)
pred = torch.tensor(net.forward())
else:
pred = torch.tensor(session.run([session.get_outputs()[0].name], {session.get_inputs()[0].name: img}))
else: # tensorflow model (tflite, pb, saved_model)
使用onnx权重模型时进到
pred = torch.tensor(session.run([session.get_outputs()[0].name], {session.get_inputs()[0].name: img}))
运行时,检测一张图片需要花费160-180ms
此时将上述语句替换成如下:
# Inference
if pt:
visualize = increment_path(save_dir / Path(path).stem, mkdir=True) if visualize else False
pred = model(img, augment=augment, visualize=visualize)[0]
elif onnx:
if dnn:
net.setInput(img)
pred = torch.tensor(net.forward())
else:
# pred = torch.tensor(session.run([session.get_outputs()[0].name], {session.get_inputs()[0].name: img}))
pred = np.array(session.run([session.get_outputs()[0].name], {session.get_inputs()[0].name: img}))
pred = torch.from_numpy(pred)
else: # tensorflow model (tflite, pb, saved_model)
此时运行后,检测同一张图片的耗时减少为80-90ms,时间减少了一半。