把onnx模型转TensorRT模型的trt模型报错：Your ONNX model has been generated with INT64 weights. while TensorRT

其他 2020-08-07 10:41:34 阅读次数: 0
欢迎大家关注笔者，你的关注是我持续更博的最大动力


原创文章，转载告知，盗版必究


把onnx模型转TensorRT模型的trt模型报错：[TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.

文章目录：

本人环境声明：

  • 系统环境：Ubuntu18.04.1
  • cuda版本：10.2.89
  • cudnn版本：7.6.5
  • torch版本：1.5.0
  • torchvision版本：0.6.0
  • mmcv版本：0.5.5
  • 项目代码mmdetection v2.0.0，官网是在20200506正式发布的v2.0.0版本
  • TensorRT-7.0.0.11
  • uff0.6.5

1 错误原因分析

我是在把mmdetection的模型转换为onnx模型之后，再把onnx模型转化为trt模式的时候，遇到的这个错误。从Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. 提示信息可以看出;

  • 我们转化后的ONNX模型的参数类型是INT64
  • 然而：TensorRT本身不支持INT64
  • 而对于：INT32的精度，TensorRT是支持的，因此可以尝试把ONNX模型的精度改为INT32，然后再进行转换

错误代码内容：

(mmdetection) shl@zfcv:~/TensorRT-7.0.0.11/bin$ ./trtexec --onnx=retinate_hat_hair_beard.onnx --saveEngine=retinate_hat_hair_beard.trt --device=1
&&&& RUNNING TensorRT.trtexec # ./trtexec --onnx=retinate_hat_hair_beard.onnx --saveEngine=retinate_hat_hair_beard.trt --device=1
[07/31/2020-13:56:39] [I] === Model Options ===
[07/31/2020-13:56:39] [I] Format: ONNX
[07/31/2020-13:56:39] [I] Model: retinate_hat_hair_beard.onnx
[07/31/2020-13:56:39] [I] Output:
[07/31/2020-13:56:39] [I] === Build Options ===
[07/31/2020-13:56:39] [I] Max batch: 1
[07/31/2020-13:56:39] [I] Workspace: 16 MB
[07/31/2020-13:56:39] [I] minTiming: 1
[07/31/2020-13:56:39] [I] avgTiming: 8
[07/31/2020-13:56:39] [I] Precision: FP32
[07/31/2020-13:56:39] [I] Calibration:
[07/31/2020-13:56:39] [I] Safe mode: Disabled
[07/31/2020-13:56:39] [I] Save engine: retinate_hat_hair_beard.trt
[07/31/2020-13:56:39] [I] Load engine:
[07/31/2020-13:56:39] [I] Inputs format: fp32:CHW
[07/31/2020-13:56:39] [I] Outputs format: fp32:CHW
[07/31/2020-13:56:39] [I] Input build shapes: model
[07/31/2020-13:56:39] [I] === System Options ===
[07/31/2020-13:56:39] [I] Device: 1
[07/31/2020-13:56:39] [I] DLACore:
[07/31/2020-13:56:39] [I] Plugins:
[07/31/2020-13:56:39] [I] === Inference Options ===
[07/31/2020-13:56:39] [I] Batch: 1
[07/31/2020-13:56:39] [I] Iterations: 10
[07/31/2020-13:56:39] [I] Duration: 3s (+ 200ms warm up)
[07/31/2020-13:56:39] [I] Sleep time: 0ms
[07/31/2020-13:56:39] [I] Streams: 1
[07/31/2020-13:56:39] [I] ExposeDMA: Disabled
[07/31/2020-13:56:39] [I] Spin-wait: Disabled
[07/31/2020-13:56:39] [I] Multithreading: Disabled
[07/31/2020-13:56:39] [I] CUDA Graph: Disabled
[07/31/2020-13:56:39] [I] Skip inference: Disabled
[07/31/2020-13:56:39] [I] Input inference shapes: model
[07/31/2020-13:56:39] [I] Inputs:
[07/31/2020-13:56:39] [I] === Reporting Options ===
[07/31/2020-13:56:39] [I] Verbose: Disabled
[07/31/2020-13:56:39] [I] Averages: 10 inferences
[07/31/2020-13:56:39] [I] Percentile: 99
[07/31/2020-13:56:39] [I] Dump output: Disabled
[07/31/2020-13:56:39] [I] Profile: Disabled
[07/31/2020-13:56:39] [I] Export timing to JSON file:
[07/31/2020-13:56:39] [I] Export output to JSON file:
[07/31/2020-13:56:39] [I] Export profile to JSON file:
[07/31/2020-13:56:39] [I]
----------------------------------------------------------------
Input filename:   retinate_hat_hair_beard.onnx
ONNX IR version:  0.0.6
Opset version:    9
Producer name:    pytorch
Producer version: 1.5
Domain:
Model version:    0
Doc string:
----------------------------------------------------------------
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2020-13:56:40] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
While parsing node number 191 [Upsample]:
ERROR: builtin_op_importers.cpp:3240 In function importUpsample:
[8] Assertion failed: scales_input.is_weights()
[07/31/2020-13:56:40] [E] Failed to parse onnx file
[07/31/2020-13:56:40] [E] Parsing model failed
[07/31/2020-13:56:40] [E] Engine creation failed
[07/31/2020-13:56:40] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # ./trtexec --onnx=retinate_hat_hair_beard.onnx --saveEngine=retinate_hat_hair_beard.trt --device=1
(mmdetection) shl@zfcv:~/TensorRT-7.0.0.11/bin$ ls

2 错误解决方式

2.1 错误解决方式一（亲测可行）

可能是我们生成的.onnx模型的graph太复杂，我们先把它变简单点

1、安装onnx-simplifier

pip install onnx-simplifier

2、把之前转化的onnx模型转化为更简单的onnx模型

python -m onnxsim retinate_hat_hair_beard.onnx retinate_hat_hair_beard_sim.onnx

在这里插入图片描述

扫描二维码关注公众号，回复： 11539037 查看本文章

3、然后在把onnx模型转换为TensorRT的trt模型

./trtexec --onnx=retinate_hat_hair_beard_sim.onnx --saveEngine=retinate_hat_hair_beard_sim.trt --device=1

(mmdetection) shl@zfcv:~/TensorRT-7.0.0.11/bin$ ./trtexec --onnx=retinate_hat_hair_beard_sim.onnx --saveEngine=retinate_hat_hair_beard_sim.trt --device=1
&&&& RUNNING TensorRT.trtexec # ./trtexec --onnx=retinate_hat_hair_beard_sim.onnx --saveEngine=retinate_hat_hair_beard_sim.trt --device=1
[07/31/2020-14:15:14] [I] === Model Options ===
[07/31/2020-14:15:14] [I] Format: ONNX
[07/31/2020-14:15:14] [I] Model: retinate_hat_hair_beard_sim.onnx
[07/31/2020-14:15:14] [I] Output:
[07/31/2020-14:15:14] [I] === Build Options ===
[07/31/2020-14:15:14] [I] Max batch: 1
[07/31/2020-14:15:14] [I] Workspace: 16 MB
[07/31/2020-14:15:14] [I] minTiming: 1
[07/31/2020-14:15:14] [I] avgTiming: 8
[07/31/2020-14:15:14] [I] Precision: FP32
[07/31/2020-14:15:14] [I] Calibration:
[07/31/2020-14:15:14] [I] Safe mode: Disabled
[07/31/2020-14:15:14] [I] Save engine: retinate_hat_hair_beard_sim.trt
[07/31/2020-14:15:14] [I] Load engine:
[07/31/2020-14:15:14] [I] Inputs format: fp32:CHW
[07/31/2020-14:15:14] [I] Outputs format: fp32:CHW
[07/31/2020-14:15:14] [I] Input build shapes: model
[07/31/2020-14:15:14] [I] === System Options ===
[07/31/2020-14:15:14] [I] Device: 1
[07/31/2020-14:15:14] [I] DLACore:
[07/31/2020-14:15:14] [I] Plugins:
[07/31/2020-14:15:14] [I] === Inference Options ===
[07/31/2020-14:15:14] [I] Batch: 1
[07/31/2020-14:15:14] [I] Iterations: 10
[07/31/2020-14:15:14] [I] Duration: 3s (+ 200ms warm up)
[07/31/2020-14:15:14] [I] Sleep time: 0ms
[07/31/2020-14:15:14] [I] Streams: 1
[07/31/2020-14:15:14] [I] ExposeDMA: Disabled
[07/31/2020-14:15:14] [I] Spin-wait: Disabled
[07/31/2020-14:15:14] [I] Multithreading: Disabled
[07/31/2020-14:15:14] [I] CUDA Graph: Disabled
[07/31/2020-14:15:14] [I] Skip inference: Disabled
[07/31/2020-14:15:14] [I] Input inference shapes: model
[07/31/2020-14:15:14] [I] Inputs:
[07/31/2020-14:15:14] [I] === Reporting Options ===
[07/31/2020-14:15:14] [I] Verbose: Disabled
[07/31/2020-14:15:14] [I] Averages: 10 inferences
[07/31/2020-14:15:14] [I] Percentile: 99
[07/31/2020-14:15:14] [I] Dump output: Disabled
[07/31/2020-14:15:14] [I] Profile: Disabled
[07/31/2020-14:15:14] [I] Export timing to JSON file:
[07/31/2020-14:15:14] [I] Export output to JSON file:
[07/31/2020-14:15:14] [I] Export profile to JSON file:
[07/31/2020-14:15:14] [I]
----------------------------------------------------------------
Input filename:   retinate_hat_hair_beard_sim.onnx
ONNX IR version:  0.0.6
Opset version:    9
Producer name:    pytorch
Producer version: 1.5
Domain:
Model version:    0
Doc string:
----------------------------------------------------------------
[07/31/2020-14:15:16] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[07/31/2020-14:15:57] [I] [TRT] Detected 1 inputs and 10 output network tensors.
[07/31/2020-14:16:03] [W] [TRT] Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles
Cuda failure: out of memory
已放弃 (核心已转储)

转后后结果，可以看到有生成.trt文件：
在这里插入图片描述

2.2 解决方法二：从新生成onnx模型的精度为INT32（还没有尝试）

在这里插入图片描述

在这里插入图片描述
♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠ ⊕ ♠

猜你喜欢

转载自blog.csdn.net/weixin_41010198/article/details/107712487
今日推荐
《美国对全球网络空间安全与发展的威胁和破坏》报告发布
火速冲上 GitHub 热榜 —— 开源编程语言、框架哪有这么可爱？
北京人形机器人创新中心发布全球首个纯电驱拟人奔跑的全尺寸人形机器人“天工”
LFOSSA 源来如此公开课 | 掌握云原生未来：CNCF 认证全面攻略与备考秘籍
国产云输入法——仅华为无云端数据上传安全问题
周排行
Python环境安装与基础语法（1）——计算机基础知识
IMU预积分
ADAS中的LDW、FCW、BSD、LCA、ACC、AEB、APA、DMS代表的含义
B站笔试两道题
skyeye arm 硬件虚拟机环境的搭建
Web前端静态页面示例
数组-合并排序数组 II-简单
springcloud之版本问题启动报错
面向对象-------------匿名对象(六)
输入URL到页面呈现中间发生了什么？
每日归档
更多
2024-04-30(1)
2024-04-29(40)
2024-04-28(0)
2024-04-27(56)
2024-04-26(39)
2024-04-25(22)
2024-04-24(36)
2024-04-23(26)
2024-04-22(39)
2024-04-21(0)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022