TensorRT is NVIDIA's official deep learning network acceleration inference library. It has c++ and python APIs. The python API only supports linux systems and can accelerate ONNX, Caffe, and TensorFlow networks. Now there is not much official support for the target detection framework, among which is the onnx version of yolov3, which is also the mainstream target detection network.
ONNX is an open file format designed for machine learning to store trained models. It enables different artificial intelligence frameworks (such as Pytorch, MXNet) to store and interact with model data in the same format. The specifications and codes of ONNX are mainly jointly developed by companies such as Microsoft, Amazon, Facebook and IBM, and are hosted on Github in an open source manner. Currently, the deep learning frameworks that officially support loading ONNX models and performing inference include: Caffe2, PyTorch, MXNet, ML.NET, TensorRT and Microsoft CNTK, and TensorFlow also unofficially supports ONNX. ---Wikipedia
Environmental information:
- Ubuntu 18.04
- Queue10.0
- Cudnn7.5.0
- Python3.6 (Python2.7 is used to convert weights to onnx)
- TensorFlow1.13.1
- Keras2.2.4
- TensorRT5.1.5.0
- pycuda2019.1.2
- onnx1.4.1 (the sample of tensorrt5 only supports onnx1.4.1) see: https://elinux.org/TensorRT/YoloV3
Download TensorRT:
Official website download (need to log in to Nvidia in advance): https://developer.nvidia.com/nvidia-tensorrt-download
Select the recommended stable version GA TensorRT 5.1 GA
Download the tar package, in which the wheel package of python Api needs to be installed, select the corresponding python version to install
pip install tensorrt-5.1.5.0-cp36-none-linux_x86_64.whl
The directory where the project is located /TensorRT-5.1.5.0/samples/python/yolov3_onnx, this is the official demo of yolov3's darknet weight transfer to onnx and then transfer to trt
Install onnx1.4.1 version
pip install onnx==1.4.1Install dependencies:
pip install -r requirements.txt
Download the keras version of yolov3 from the official website:
https://github.com/qqwweee/keras-yolo3
Here we first use the weights of coco's standard dataset as a demonstration:
Download link:
https://pjreddie.com/media/files/yolov3.weightsModify the input size of input_layer in line 88 of convert.py to 608*608, which is the input size supported in the official TensorRT sample
input_layer = Input(shape=(608, 608, 3))Run the convert.py file under the project to convert yolov3.cfg and yolov3.weights to yolov3.h5:
python convert.py yolov3.cfg yolov3.weights model_data/yolo.h5The h5 file saved here contains the model and weights. When saving the model in your own training data, it must also be saved to contain the model and weights. Use the save method instead of
save_weights method
h5 model to darknet weights file:
Use the conversion method in the tutorial of model conversion [yolov3 model conversion between keras and darknet] , the pro-test is available
Need to install tensorflow and keras
pip install tensorflow-gpu==1.13.1 keras==2.2.4
weights file to onnx file:
Use yolov3_to_onnx.py under the official demo to convert to onnx file
Note that only python2 is supported here, and a python2 environment needs to be created separately and onnx1.4.1 version and other related dependent packages should be installed
Modify the input and output paths to your own weights file path and generated name
weights_file_path = 'yolov3c_d2k_k2d.weights'
output_file_path = 'yolov3.onnx'Specify the relevant mask, anchor, threshold, and input size parameters
postprocessor_args = {"yolo_masks": [(6, 7, 8), (3, 4, 5), (0, 1, 2)], # A list of 3 three-dimensional tuples for the YOLO masks
"yolo_anchors": [(10, 13), (16, 30), (33, 23), (30, 61), (62, 45), # A list of 9 two-dimensional tuples for the YOLO anchors
(59, 119), (116, 90), (156, 198), (373, 326)],
"obj_threshold": 0.6, # Threshold for object coverage, float value between 0 and 1
"nms_threshold": 0.5, # Threshold for non-max suppression algorithm, float value between 0 and 1
"yolo_input_resolution": input_resolution_yolov3_HW}Onnx file to trt file and inference:
Use onnx_to_tensorrt.py under the official demo to convert to trt file
Here we will use the original python3 environment to run
Modify the path under the main function to your own file path
onnx_file_path = 'yolov3.onnx'
engine_file_path = "yolov3.trt"After the operation is completed, a marked image will be output, indicating that the trt file was successfully saved and inferred successfully
Content comprehension:
The 608*608 size image input used by yolov3 in the official demo here, and then there are 3 scales of output 19*19, 38*38, 76*76. This is the processing done when the weights file is converted to an onnx file, and onnx was used before The official keras to onnx can only have one output, and the intermediate layer is not detected during parsing, so you can only use tensorrt's demo to transfer onnx through weights, and then transfer onnx to trt, so that it can be used normally. And specify the input size as 608*608.
If you use 416*416 size as input, you need to change the output of the three scales to 13*13, 26*26, 52*52. Since the yolo of keras maintains the weight by default, and it is true that the weight h5 is what we need, because during training, we generally freeze the front layers and other operations, which will affect the model structure when saving, so we need to h5 First convert the weight file into a weights file, then use yolov3’s official convert.py to convert it into h5 and then into a weights file, and then the operation is the same as above. It is also necessary to modify the class in coco_labels.txt to be its own category, and at the same time modify the CATEGORY_NUM in data_processing.py to be the number of its own class.
Reference article:
Use TensorRT to accelerate yolo3 https://www.cnblogs.com/justcoder/p/10428100.html
TensorRT&Sample&Python[yolov3_onnx] https://www.cnblogs.com/shouhuxianjian/p/10550262.html
Model conversion [yolov3 model conversion between keras and darknet] https://www.cnblogs.com/shouhuxianjian/p/10567201.html