Linux安装mmdeploy1.2

企业开发 2023-07-15 20:37:00 阅读次数: 0

参考链接

mmdeploy/docs/en/01-how-to-build/linux-x86_64.md at main · open-mmlab/mmdeploy

Toolchains安装

  • 操作系统：Ubuntu 18.04
  • cuda：11.4

安装cuda之前需要确认自己机器的驱动是否支持该版本的cuda，cuda版本与驱动之间的关系可以打开链接查看，驱动下载的载链接为：https://www.nvidia.com/download/index.aspx。
确认上述信息或者更新自己驱动之后，就可以打开链接，根据自己机器选择合适的配置，得到下载命令，下载文件并安装。image.png
image.png
将cuda环境变量添加到系统中

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
  • cudnn：8.2

cudnn的下载链接为：https://developer.nvidia.com/rdp/cudnn-archive。本文中下载的cudnn版本为8.2.1，适配cuda 11.x的版本。具体安装过程请自行搜索。
image.png

  • cmake：cmake版本需要大于等于3.14.0，安装方式如下：
wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0-linux-x86_64.tar.gz
tar -xzvf cmake-3.20.0-linux-x86_64.tar.gz
sudo ln -sf $(pwd)/cmake-3.20.0-linux-x86_64/bin/* /usr/bin/

依赖环境

MMdeploy可以分为两部分：Model Converter和SDK。其中Model Converter是将模型转换为目标推理引擎要求的格式文件，如TensorRT，则生成TensorRT Engine文件。SDK提供多种编程语言在工业生产来部署目标推理引擎。

为Model Conveter安装依赖

miniconda

  1. 安装miniconda
  • 下载miniconda文件，打开链接，找到适合自己目标机器的文件
  • 安装minconda

通过如下命令安装即可，过程中一直enter就行，默认情况下，conda的环境变量会写入到bashrc文件中。

sudo bash Miniconda3-latest-Linux-x86_64.sh
  • 激活conda环境

通过下面的命令可以激活conda环境，进入base虚拟环境。

source ~/.bashrc
  • 替换conda国内源
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
conda config --set show_channel_urls yes

如果只想在安装时使用国内源，则可以通过如下命令实现：

conda install -y -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/linux-64 opencv tensorboardX
  1. 为mmdeploy创建虚拟环境
conda create -n mmdeploy python=3.8 -y
conda activate mmdeploy

pytorch：1.10.0

通过如下的命令安装pytorch 1.10.0，对应的cudatoolkit为11.3，而安装的cuda为11.4，但是对于后续的测试没有什么影响。

conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cudatoolkit=11.3 -c pytorch -c conda-forge

mmcv

export cu_version=cu114 # cuda 11.4
export torch_version=torch1.10
pip install -U openmim
mim install mmengine
mim install "mmcv>=2.0.0rc2"

为SDK安装依赖环境

  • OpenCV
sudo apt-get install libopencv-dev
  • pplcv
git clone https://github.com/openppl-public/ppl.cv.git
cd ppl.cv
export PPLCV_DIR=$(pwd)
git checkout tags/v0.7.0 -b v0.7.0
./build.sh cuda

安装推理引擎

image.png
mmdeploy支持以不同的backend推理模型，在这篇文章中主要测试Tensor RT backend。本文以TensorRT推理引擎的安装为例，其他推理平台可以参考文档。

TensorRT

  1. 下载TensorRT tar文件
tar -xvf TensorRT-8.2.4.2.Linux.x86_64-gnu.cuda-11.4.cudnn8.2.tar.gz
  1. 添加环境变量
# <<< TensorRT 8 <<<

export PATH=/home/xxxx/local/TensorRT-8.2.4.2/bin:$PATH
export LD_LIBRARY_PATH=/home/xxxx/local/TensorRT-8.2.4.2/lib:$LD_LIBRARY_PATH
export C_INCLUDE_PATH=/home/xxxx/local/TensorRT-8.2.4.2/include:$C_INCLUDE_PATH
export CPLUS_INCLUDE_PATH=/home/xxxx/local/TensorRT-8.2.4.2/include:$CPLUS_INCLUDE_PATH
  1. 安装TensorRT python 包
cd python
# 根据python的版本安装
pip install tensorrt-8.2.4.2-cp38-none-linux_x86_64.whl
  1. 安装pycuda
conda install -c conda-forge pycuda

编译mmdeploy

编译mmdeploy分为两部分，Model Converter和SDK。其中converter的作用是将pytorch模型转换为对应backend的推理引擎文件，并测试起在不同数据集上的性能；SDK提供不同编程语言来将backend的推理引擎部署到工业实际生成中。

Model Converter

编译model converter包含两个步骤，一个是编译backend的自定义算子，一个是安装mmdeploy-python库。

编译tensorrt算子

cd ${
    
    MMDEPLOY_DIR}
mkdir -p build && cd build
cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=trt -DTENSORRT_DIR=${
    
    TENSORRT_DIR} -DCUDNN_DIR=${
    
    CUDNN_DIR} ..
make -j$(nproc) && make install

编译成功之后会生成libmmdeploy_tensorrt_ops.so动态库。mmdeploy-python会在进行模型转换时，修改模型的function、module和symbolic，需要链接相应的tensorrt自定义算子。

安装mmdeploy-python

cd ${
    
    MMDEPLOY_DIR}
mim install -e .

安装完成之后，就可以对mmdeploy支持的模型进行相应的转换。其实后续的SDK可以不安装。

Build SDK and Demo

这个应该就是基于mmdeploy的部署sdk，通过mmdeploy- python得到tensorrt engine之后，通过这个sdk可以直接部署模型。

cmake -DCMAKE_CXX_COMPILER=g++-7 \
-DMMDEPLOY_TARGET_BACKENDS=trt \
-DTENSORRT_DIR=/home/xxxx/local/TensorRT-8.2.4.2 \
-DCUDNN_DIR=/home/xxxx/local/cuda/ \
-DMMDEPLOY_BUILD_SDK=ON -DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
-DMMDEPLOY_BUILD_EXAMPLES=ON -DMMDEPLOY_TARGET_DEVICES="cuda;cpu" ..

转换模型

本文测试了常见模型Faster R-CNN、Cascade R-CNN、DeepLabv3和Mask R- CNN转换之后性能之间的对比。转换模型和测试性能可以分别参考如下两个链接：

benchmark

mmdeploy支持不同类型模型的转换，具体运行时间和精度可以参考如下链接。

mmdet

对应mmdetection，测试了Yolov3、Faster R-CNN、Cascade R-CNN和Mask R-CNN。
mmdetection/docs/en/model_zoo.md at main · open-mmlab/mmdetection
在安装MM Detection遇到numpy的版本问题，解决方案源码编译安装cocoapi。

git clone https://github.com/pdollar/coco.git

截屏2023-07-04 17.36.36.png

Yolov3

转换engine脚本如下（static shape转换正常，dynamic shape存在问题）：

#!/bin/bash

export MMDET_ROOT=/home/xxxx/workspace/mmlab/mmdetection
python ./tools/deploy.py \
    configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py \
    $MMDET_ROOT/configs/yolo/yolov3_d53_8xb8-ms-608-273e_coco.py \
    $MMDET_ROOT/exp/yolov3_d53_mstrain-608_273e_coco_20210518_115020-a2c3acb8.pth \
    $MMDET_ROOT/demo/demo.jpg \
    --work-dir work_dir \
    --device cuda:0

Yolo v3就进行了Tensor RT Engine转换未对性能进行相应的对比。

Faster R-CNN

转换engine文件的脚本与Yolo v3转换类似，只需要替换对应的deploy_config和model_config文件路径。

#!/bin/bash

export MMDET_ROOT=/home/xxxx/workspace/mmlab/mmdetection
python ./tools/deploy.py \
    configs/mmdet/detection/faster_rcnn_tensorrt_static-800x1344.py \
    $MMDET_ROOT/configs/faster_rcnn/faster-rcnn_r50_fpn_2x_coco.py \
    $MMDET_ROOT/exp/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth \
    $MMDET_ROOT/demo/demo.jpg \
    --work-dir work_dir \
    --device cuda:0

测试其在COCO2017-val数据集上的性能。

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.384
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.590
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.420
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.215
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.421
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.503
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.520
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.520
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.520
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.326
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.557
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.661
07/05 14:00:14 - mmengine - INFO - bbox_mAP_copypaste: 0.384 0.590 0.420 0.215 0.421 0.503
07/05 14:00:16 - mmengine - INFO - Results has been saved to exp/results.pkl.
07/05 14:00:16 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.3840  
coco/bbox_mAP_50: 0.5900  
coco/bbox_mAP_75: 0.4200  
coco/bbox_mAP_s: 0.2150  
coco/bbox_mAP_m: 0.4210  
coco/bbox_mAP_l: 0.5030  
data_time: 0.0028  time: 0.0402

通过onnx转换得到对应的engine文件之后，可以通过tools/test.py文件测试精度和速度。

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.384
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.590
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.419
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.215
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.421
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.502
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.519
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.519
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.519
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.325
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.556
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.661
07/05 14:38:00 - mmengine - INFO - bbox_mAP_copypaste: 0.384 0.590 0.419 0.215 0.421 0.502
07/05 14:38:01 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.3840  coco/bbox_mAP_50: 0.5900  
coco/bbox_mAP_75: 0.4190  coco/bbox_mAP_s: 0.2150  
coco/bbox_mAP_m: 0.4210  coco/bbox_mAP_l: 0.5020  
data_time: 0.0030  time: 0.0297
  • tensorrt-fp16
Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.384
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.590
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.418
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.215
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.420
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.501
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.519
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.519
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.519
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.325
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.556
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.659
07/05 14:58:28 - mmengine - INFO - bbox_mAP_copypaste: 0.384 0.590 0.418 0.215 0.420 0.501
07/05 14:58:29 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.3840  coco/bbox_mAP_50: 0.5900  
coco/bbox_mAP_75: 0.4180  coco/bbox_mAP_s: 0.2150  
coco/bbox_mAP_m: 0.4200  coco/bbox_mAP_l: 0.5010  
data_time: 0.0028  time: 0.0172

运行时间测试

07/05 15:20:09 - mmengine - INFO - [tensorrt]-4750 times per count: 10.58 ms, 94.51 FPS
07/05 15:20:09 - mmengine - INFO - Epoch(test) [4750/5000]    eta: 0:00:04  time: 0.0146  data_time: 0.0017  memory: 38  
07/05 15:20:10 - mmengine - INFO - Epoch(test) [4800/5000]    eta: 0:00:03  time: 0.0208  data_time: 0.0048  memory: 39  
07/05 15:20:11 - mmengine - INFO - [tensorrt]-4850 times per count: 10.58 ms, 94.54 FPS
07/05 15:20:11 - mmengine - INFO - Epoch(test) [4850/5000]    eta: 0:00:02  time: 0.0173  data_time: 0.0031  memory: 39  
07/05 15:20:12 - mmengine - INFO - Epoch(test) [4900/5000]    eta: 0:00:01  time: 0.0142  data_time: 0.0016  memory: 38  
07/05 15:20:13 - mmengine - INFO - [tensorrt]-4950 times per count: 10.58 ms, 94.55 FPS
07/05 15:20:13 - mmengine - INFO - Epoch(test) [4950/5000]    eta: 0:00:00  time: 0.0197  data_time: 0.0024  memory: 39  
07/05 15:20:13 - mmengine - INFO - Epoch(test) [5000/5000]    eta: 0:00:00  time: 0.0148  data_time: 0.0016  memory: 39

截屏2023-07-06 18.41.16.png
同样，当得到TensorRT Engine之后，我们可以采用trtexec工具来加载engine测试时间，trtexec推理命令：

trtexec --loadEngine=work_dir/faster_rcnn_dynamic_fp16/end2end.engine --iterations=5000 --plugins=mmdeploy/lib/libmmdeploy_tensorrt_ops.so --minShapes=input:1x3x320x320 --optShapes=input:1x3x800x1344 --maxShapes=input:1x3x1344x1344 --shapes=input:1x3x800x1344 --fp16 --workspace=8000
  • tensorrt-fp32
07/05 15:35:09 - mmengine - INFO - [tensorrt]-4750 times per count: 21.97 ms, 45.51 FPS
07/05 15:35:09 - mmengine - INFO - Epoch(test) [4750/5000]    eta: 0:00:07  time: 0.0252  data_time: 0.0016  memory: 38  
07/05 15:35:10 - mmengine - INFO - Epoch(test) [4800/5000]    eta: 0:00:06  time: 0.0262  data_time: 0.0016  memory: 39  
07/05 15:35:11 - mmengine - INFO - [tensorrt]-4850 times per count: 21.96 ms, 45.53 FPS
07/05 15:35:11 - mmengine - INFO - Epoch(test) [4850/5000]    eta: 0:00:04  time: 0.0283  data_time: 0.0024  memory: 39  
07/05 15:35:13 - mmengine - INFO - Epoch(test) [4900/5000]    eta: 0:00:03  time: 0.0262  data_time: 0.0018  memory: 38  
07/05 15:35:14 - mmengine - INFO - [tensorrt]-4950 times per count: 21.97 ms, 45.52 FPS
07/05 15:35:14 - mmengine - INFO - Epoch(test) [4950/5000]    eta: 0:00:01  time: 0.0304  data_time: 0.0029  memory: 39  
07/05 15:35:16 - mmengine - INFO - Epoch(test) [5000/5000]    eta: 0:00:00  time: 0.0363  data_time: 0.0033  memory: 39

截屏2023-07-06 18.28.18.png

  • pytorch
07/06 14:49:56 - mmengine - INFO - (GB) mem_used: 81.76 | uss: 3.89 | pss: 3.96 | total_proc: 1
07/06 14:50:45 - mmengine - INFO - ==================================
07/06 14:50:45 - mmengine - INFO - Done image [1000/5000], fps: 27.6 img/s, times per image: 36.2 ms/img, cuda memory: 520 MB
07/06 14:50:45 - mmengine - INFO - (GB) mem_used: 78.25 | uss: 4.74 | pss: 4.81 | total_proc: 1
07/06 14:51:32 - mmengine - INFO - ==================================
07/06 14:51:32 - mmengine - INFO - Done image [2000/5000], fps: 27.8 img/s, times per image: 36.0 ms/img, cuda memory: 520 MB
07/06 14:51:33 - mmengine - INFO - (GB) mem_used: 84.05 | uss: 4.74 | pss: 4.81 | total_proc: 1
07/06 14:52:21 - mmengine - INFO - ==================================
07/06 14:52:21 - mmengine - INFO - Done image [3000/5000], fps: 27.8 img/s, times per image: 36.0 ms/img, cuda memory: 534 MB
07/06 14:52:22 - mmengine - INFO - (GB) mem_used: 83.13 | uss: 4.74 | pss: 4.81 | total_proc: 1
07/06 14:53:09 - mmengine - INFO - ==================================
07/06 14:53:09 - mmengine - INFO - Done image [4000/5000], fps: 27.8 img/s, times per image: 36.0 ms/img, cuda memory: 520 MB
07/06 14:53:09 - mmengine - INFO - (GB) mem_used: 82.96 | uss: 4.74 | pss: 4.81 | total_proc: 1
07/06 14:53:56 - mmengine - INFO - ==================================
07/06 14:53:56 - mmengine - INFO - Done image [5000/5000], fps: 27.7 img/s, times per image: 36.0 ms/img, cuda memory: 534 MB
07/06 14:53:56 - mmengine - INFO - (GB) mem_used: 83.42 | uss: 4.74 | pss: 4.81 | total_proc: 1
07/06 14:53:56 - mmengine - INFO - ============== Done ==================
07/06 14:53:56 - mmengine - INFO - Overall fps: 27.7 img/s, times per image: 36.1 ms/img
07/06 14:53:56 - mmengine - INFO - cuda memory: 163 MB
07/06 14:53:57 - mmengine - INFO - (GB) mem_used: 83.29 | uss: 4.74 | pss: 4.81 | total_proc: 1

截屏2023-07-06 18.36.43.png

统计

FasterR-CNN-R50 mAP latency(ms) memory(G)
pytorch 0.384 36.1 3.903
tensorrt-fp32 0.384 21.69 4.243
tensorrt-fp16 0.384 10.58 3.634

Cascade R- CNN

性能对比

  • pytorch

截屏2023-07-07 09.40.03.png

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.403
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.586
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.440
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.225
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.438
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.529
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.543
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.543
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.543
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.333
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.582
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.689
07/07 09:41:21 - mmengine - INFO - bbox_mAP_copypaste: 0.403 0.586 0.440 0.225 0.438 0.529
07/07 09:41:23 - mmengine - INFO - Results has been saved to exp/results.pkl.
07/07 09:41:23 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.4030  coco/bbox_mAP_50: 0.5860  
coco/bbox_mAP_75: 0.4400  coco/bbox_mAP_s: 0.2250  
coco/bbox_mAP_m: 0.4380  coco/bbox_mAP_l: 0.5290  
data_time: 0.0028  time: 0.0474
  • tensorrt-fp32

截屏2023-07-07 10.08.51.png

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.403
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.586
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.439
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.225
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.437
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.529
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.543
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.543
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.543
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.333
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.581
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.689
07/07 10:06:05 - mmengine - INFO - bbox_mAP_copypaste: 0.403 0.586 0.439 0.225 0.437 0.529
07/07 10:06:05 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.4030  coco/bbox_mAP_50: 0.5860  
coco/bbox_mAP_75: 0.4390  coco/bbox_mAP_s: 0.2250  
coco/bbox_mAP_m: 0.4370  coco/bbox_mAP_l: 0.5290  
data_time: 0.0022  time: 0.0310

07/07 10:09:36 - mmengine - INFO - [tensorrt]-3050 times per count: 25.29 ms, 39.54 FPS
07/07 10:10:08 - mmengine - INFO - [tensorrt]-4050 times per count: 25.33 ms, 39.48 FPS
  • tensorrt-fp16

截屏2023-07-07 10.22.51.png

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.403
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.586
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.439
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.225
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.438
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.531
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.333
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.581
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.691
07/07 10:23:56 - mmengine - INFO - bbox_mAP_copypaste: 0.403 0.586 0.439 0.225 0.438 0.531
07/07 10:23:56 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.4030  coco/bbox_mAP_50: 0.5860  
coco/bbox_mAP_75: 0.4390  coco/bbox_mAP_s: 0.2250  
coco/bbox_mAP_m: 0.4380  coco/bbox_mAP_l: 0.5310  
data_time: 0.0026  time: 0.0192

07/07 10:22:42 - mmengine - INFO - [tensorrt]-3050 times per count: 13.16 ms, 75.96 FPS
07/07 10:23:01 - mmengine - INFO - [tensorrt]-4050 times per count: 13.15 ms, 76.03 FPS

统计

cascade rcnn-R50 mAP latency(ms) memory(G)
pytorch 0.4030 40.3 4.124
tensorrt-fp32 0.4030 25.33 4.347
tensorrt-fp16 0.4030 13.16 3.757

Mask R-CNN

在进行实例分割模型转换时，用的是MMdeploy/configs/mmdet/instance-seg中的配置文件，这一点很重要‼️。

性能对比

  • pytorch

截屏2023-07-07 16.14.18.png

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.354
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.564
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.380
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.166
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.382
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.525
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.481
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.481
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.481
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.283
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.515
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.646
07/07 16:21:43 - mmengine - INFO - segm_mAP_copypaste: 0.354 0.564 0.380 0.166 0.382 0.525
07/07 16:21:45 - mmengine - INFO - Results has been saved to exp/results.pkl.
07/07 16:21:45 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.3920  coco/bbox_mAP_50: 0.5960  
coco/bbox_mAP_75: 0.4280  coco/bbox_mAP_s: 0.2290  
coco/bbox_mAP_m: 0.4260  coco/bbox_mAP_l: 0.5120  
coco/segm_mAP: 0.3540  coco/segm_mAP_50: 0.5640  
coco/segm_mAP_75: 0.3800  coco/segm_mAP_s: 0.1660  
coco/segm_mAP_m: 0.3820  coco/segm_mAP_l: 0.5250  
data_time: 0.0225  time: 0.1063

Overall fps: 26.4 img/s, times per image: 37.9 ms/img
  • tensorrt-fp32
    image.png
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.354
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.564
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.380
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.166
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.381
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.525
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.481
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.481
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.481
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.283
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.514
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.646
07/08 16:01:46 - mmengine - INFO - segm_mAP_copypaste: 0.354 0.564 0.380 0.166 0.381 0.525
07/08 16:01:47 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.3910  coco/bbox_mAP_50: 0.5960  
coco/bbox_mAP_75: 0.4280  coco/bbox_mAP_s: 0.2290  
coco/bbox_mAP_m: 0.4250  coco/bbox_mAP_l: 0.5110  
coco/segm_mAP: 0.3540  coco/segm_mAP_50: 0.5640  
coco/segm_mAP_75: 0.3800  coco/segm_mAP_s: 0.1660  
coco/segm_mAP_m: 0.3810  coco/segm_mAP_l: 0.5250  data_time: 0.0206  time: 0.0796

07/08 15:55:20 - mmengine - INFO - [tensorrt]-1050 times per count: 25.77 ms, 38.80 FPS
07/08 15:56:37 - mmengine - INFO - [tensorrt]-2050 times per count: 25.80 ms, 38.76 FPS
07/08 15:59:15 - mmengine - INFO - [tensorrt]-4050 times per count: 25.85 ms, 38.69 FPS
  • tensorrt-fp16

image.png

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.354
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.564
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.380
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.166
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.381
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.524
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.481
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.481
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.481
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.283
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.514
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.646
07/08 16:29:03 - mmengine - INFO - segm_mAP_copypaste: 0.354 0.564 0.380 0.166 0.381 0.524
07/08 16:29:04 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.3920  coco/bbox_mAP_50: 0.5950  
coco/bbox_mAP_75: 0.4290  coco/bbox_mAP_s: 0.2280  
coco/bbox_mAP_m: 0.4270  coco/bbox_mAP_l: 0.5100  
coco/segm_mAP: 0.3540  coco/segm_mAP_50: 0.5640  
coco/segm_mAP_75: 0.3800  coco/segm_mAP_s: 0.1660  
coco/segm_mAP_m: 0.3810  coco/segm_mAP_l: 0.5240  data_time: 0.0200  time: 0.0663

07/08 16:25:37 - mmengine - INFO - [tensorrt]-3050 times per count: 12.55 ms, 79.70 FPS
07/08 16:26:43 - mmengine - INFO - [tensorrt]-4050 times per count: 12.55 ms, 79.69 FPS

统计

Mask R-CNN R50 mAP segm_mAP latency(ms) memory(G)
pytorch 0.3920 0.3540 37.9 4.355
tensorrt-fp32 0.3910 0.3540 25.85 4.636
tensorrt-fp16 0.3920 0.3540 12.55 4.325

mmseg

cityscapes数据集准备

参考如下链接准备cityscapes数据集。
mmsegmentation/docs/en/user_guides/2_dataset_prepare.md at main · open-mmlab/mmsegmentation
此外对于cityscapes数据集，还需要进行如下操作：
截屏2023-07-10 10.33.50.png

deeplabv3

测试cityscapes数据集上的结果
Tutorial 4: Train and test with existing models — MMSegmentation 1.1.0 documentation
官方给出的deeplabv3在不同配置下的性能如下：
mmsegmentation/configs/deeplabv3 at main · open-mmlab/mmsegmentation

  • pytorch

截屏2023-07-07 11.57.48.png

+---------------+-------+-------+
|     Class     |  IoU  |  Acc  |
+---------------+-------+-------+
|      road     | 98.18 | 98.88 |
|    sidewalk   |  85.3 | 93.17 |
|    building   | 92.71 | 96.52 |
|      wall     | 53.12 | 58.34 |
|     fence     | 61.57 | 71.94 |
|      pole     | 65.34 | 77.46 |
| traffic light | 71.21 | 82.56 |
|  traffic sign | 79.58 | 87.34 |
|   vegetation  | 92.47 | 96.84 |
|    terrain    | 64.42 | 72.11 |
|      sky      | 94.46 | 98.12 |
|     person    | 82.53 | 92.13 |
|     rider     | 62.77 | 75.76 |
|      car      | 95.37 | 97.82 |
|     truck     | 82.07 |  92.3 |
|      bus      | 89.74 | 93.15 |
|     train     | 84.59 | 89.49 |
|   motorcycle  | 69.78 |  81.5 |
|    bicycle    | 78.04 | 89.03 |
+---------------+-------+-------+
07/07 11:53:43 - mmengine - INFO - Iter(test) [500/500]    
aAcc: 96.1700  mIoU: 79.1200  mAcc: 86.5500  data_time: 0.0068  time: 0.1914

时间：fps: 6.01 img / s

  • tensorrt-fp32

截屏2023-07-07 14.31.43.png

+---------------+-------+-------+
|     Class     |  IoU  |  Acc  |
+---------------+-------+-------+
|      road     | 98.18 | 98.88 |
|    sidewalk   |  85.3 | 93.17 |
|    building   | 92.71 | 96.52 |
|      wall     | 53.12 | 58.34 |
|     fence     | 61.57 | 71.94 |
|      pole     | 65.34 | 77.46 |
| traffic light | 71.21 | 82.56 |
|  traffic sign | 79.58 | 87.34 |
|   vegetation  | 92.47 | 96.84 |
|    terrain    | 64.42 |  72.1 |
|      sky      | 94.46 | 98.12 |
|     person    | 82.53 | 92.13 |
|     rider     | 62.76 | 75.76 |
|      car      | 95.37 | 97.82 |
|     truck     | 82.07 |  92.3 |
|      bus      | 89.74 | 93.15 |
|     train     | 84.59 | 89.49 |
|   motorcycle  | 69.78 |  81.5 |
|    bicycle    | 78.04 | 89.03 |
+---------------+-------+-------+
07/07 14:28:39 - mmengine - INFO - Epoch(test) [500/500]    
aAcc: 96.1700  mIoU: 79.1200  mAcc: 86.5500  data_time: 0.0069  time: 0.1515

07/07 14:27:57 - mmengine - INFO - [tensorrt]-205 times per count: 131.08 ms, 7.63 FPS
07/07 14:28:11 - mmengine - INFO - [tensorrt]-305 times per count: 131.18 ms, 7.62 FPS
07/07 14:28:25 - mmengine - INFO - [tensorrt]-405 times per count: 131.46 ms, 7.61 FPS
  • tensorrt-fp16

截屏2023-07-07 14.44.53.png

+---------------+-------+-------+
|     Class     |  IoU  |  Acc  |
+---------------+-------+-------+
|      road     | 98.18 | 98.88 |
|    sidewalk   |  85.3 | 93.17 |
|    building   | 92.71 | 96.53 |
|      wall     | 53.15 | 58.38 |
|     fence     | 61.58 | 71.95 |
|      pole     | 65.34 | 77.46 |
| traffic light | 71.21 | 82.55 |
|  traffic sign | 79.58 | 87.33 |
|   vegetation  | 92.47 | 96.84 |
|    terrain    | 64.41 | 72.07 |
|      sky      | 94.47 | 98.12 |
|     person    | 82.53 | 92.12 |
|     rider     | 62.76 | 75.77 |
|      car      | 95.37 | 97.82 |
|     truck     |  82.1 | 92.31 |
|      bus      | 89.74 | 93.14 |
|     train     | 84.58 | 89.49 |
|   motorcycle  | 69.76 | 81.49 |
|    bicycle    | 78.04 | 89.01 |
+---------------+-------+-------+
07/07 14:45:04 - mmengine - INFO - Epoch(test) [500/500]    
aAcc: 96.1700  mIoU: 79.1200  mAcc: 86.5500  
data_time: 0.0361  time: 0.0830

07/07 14:44:40 - mmengine - INFO - [tensorrt]-205 times per count: 36.42 ms, 27.46 FPS
07/07 14:44:48 - mmengine - INFO - [tensorrt]-305 times per count: 36.49 ms, 27.40 FPS
07/07 14:44:56 - mmengine - INFO - [tensorrt]-405 times per count: 36.41 ms, 27.47 FPS
deeplabv3 mIoU latency(ms) memory(G)
pytorch 79.12 166.39 12,587
tensorrt-fp32 79.12 131.46 5.308
tensorrt-fp16 79.12 36.49 4.032

总结

本文介绍了如何安装MMDeploy，详细叙述了安装的流程，并给出了相应的版本号。之后对不同类型的算法在转TensorRT Engine之后的性能进行了对比，可以看到的是评价指标基本上没有掉点，运行速度在FP16下提升了3-4倍。个人感觉使用MMdeploy来部署MMDetection和MMsegmetation的模型效率较高，且兼容性很好，推荐使用哈。

猜你喜欢

转载自blog.csdn.net/hello_dear_you/article/details/131680687
今日推荐
基于大语言模型的开源知识库问答系统 MaxKB GitHub Star 数量突破 5,000 个！
美国拟限制 AI 大模型出口中国和俄罗斯
苹果将与 OpenAI 达成协议，将 ChatGPT 应用于 iPhone
openKylin 社区生态委员会第六次会议圆满召开
阿里云正式发布通义千问 2.5
Python 3.13 发布首个 Beta：实验性自由线程模式和 JIT、改进交互式解释器
Stack Overflow 拿我的代码去训练 AI 大模型，还封了我的账号
Pop!_OS 的 COSMIC 桌面完成 App Store 上架工作
《2024 年一季度互联网投融资运行情况》研究报告
报告：Django 仍然是 74% 开发者的首选
15 年前上了“FFmpeg 耻辱柱”，今天他还得谢谢咱——腾讯QQPlayer一雪前耻？
TIOBE 5 月榜单：Fortran “复活”进入 Top 10
周排行
记一下去大梅沙的准备（2018-05-26）
Spring 注解 事务
基于HTTP协议的客户端缓存
阿里云rds 备份和还原
[PHP] 几个拖慢 PHP 程序/API 运行速度的点
python 代码风格------------PEP8规则
js控制json生成菜单——自制菜单（一）
将字符串: 'k:1|k1:2|k2:3|k3:4 ' ,处理成 python 字典: {'k':1, 'k1':2, ...}
微信小程序转支付宝小程序
Qt551.窗口滚动条
每日归档
更多
2024-05-13(18)
2024-05-12(0)
2024-05-11(38)
2024-05-10(38)
2024-05-09(35)
2024-05-08(42)
2024-05-07(14)
2024-05-06(40)
2024-05-05(0)
2024-05-04(7)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022