Paddle2.0: Export and deployment of ONNX model

introduce

• In addition to exporting inference models by converting dynamic graphs to static graphs
• The export function of ONNX model is also officially built in Paddle2.0
• This article will demonstrate how to export the Paddle model to ONNX model through an example
• And complete the inference prediction of the model on ONNXRunTime

ONNX

• Open Neural Network Exchange (ONNX), an open standard for machine learning models, facilitates model exchange between different frameworks
• You can build models on many frameworks that support ONNX model export, and then export ONNX models for predictive deployment
  

Paddle2ONNX

• Currently Paddle export ONNX model needs to install Paddle2ONNX for additional support
• Paddle2ONNX supports converting PaddlePaddle model format to ONNX model format.
  • The model format supports the conversion of Paddle static and dynamic graph models to ONNX, and can convert static graph models exported by save_inference_model. Dynamic graph conversion is currently in an experimental state, and will be accompanied by the release of the official version of Paddle 2.0, and a detailed tutorial will be provided
  • Operator support, currently supports exporting ONNX Opset 9~11 stably, some Paddle operators support lower ONNX Opset conversion, please refer to the operator list for details
  • Model type, please refer to the model library for official test convertible models
• For more details, please refer to the Github homepage of Paddle2ONNX

# PIP 安装
$ pip install paddle2onnx

# 源码安装
$ git clone https://github.com/paddlepaddle/paddle2onnx
$ cd paddle2onnx 
$ python setup.py install

Dynamic graph ONNX model export

• If you use the dynamic graph model built by Paddle2.0, you can quickly export the ONNX model by calling paddle.onnx.export()
• The general principle is that dynamic to static is converting the static graph model into an ONNX model, so the converted code is very similar to dynamic to static
• Next, let's demonstrate the export process through code

import os
import time
import paddle

# 从模型代码中导入模型
from u2net import U2NETP

# 实例化模型
model = U2NETP()

# 加载预训练模型参数
model.set_dict(paddle.load([path to the pretrained model]))

# 将模型设置为评估状态
model.eval()

# 定义输入数据
input_spec = paddle.static.InputSpec(shape=[None, 3, 320, 320], dtype='float32', name='image')

# ONNX模型导出
paddle.onnx.export(model, [path to the save onnx model], input_spec=[input_spec], opset_version=[opset version])

2021-01-09 17:12:24 [INFO]	ONNX model saved in u2netp.onnx

Static graph model conversion

• In addition to dynamic graph models can be exported as ONNX models
• The inference model of the static graph can of course also be converted to the ONNX model
• The conversion can be completed by invoking the following command from the command line
• For more details, please refer to paddle2onnx official Github documentation

$ paddle2onnx \
    --model_dir [model dir] \
    --model_filename [model filename] \
    --params_filename [params filename] \
    --save_file [save file] \
    --opset_version [opset version]

2021-01-09 17:12:28 [INFO]	ONNX model saved in u2netp_static.onnx

model visualization

• The visual view of the model structure can be easily performed through the VisualDL tool
• Select the model file with the suffix .onnx just saved
• The specific visualization image is as shown in the figure below:

model testing

• Here ONNXRunTime is used for ONNX model validation testing

# 安装ONNXRunTime
$ pip install onnxruntime

import time
import numpy as np
from onnxruntime import InferenceSession

# 加载ONNX模型
model = InferenceSession([path to the save onnx model])

# 准备输入
x = np.random.random((1, 3, 320, 320)).astype('float32')

# 模型预测
d0, _, _, _, _, _, _ = model.run(output_names=None, input_feed={
    
    'image': x})

# 打印输出形状
print(d0.shape)

# 速度测试
start = time.time()
d0, _, _, _, _, _, _ = model.run(output_names=None, input_feed={
    
    'image': x})
end = time.time()
print('predict time: %.04f s' % (end - start))

(1, 1, 320, 320)
predict time: 0.7178 s

Deployment instance

• Next, complete model inference deployment by adding data preprocessing and postprocessing

import cv2
import time
import numpy as np
import matplotlib.pyplot as plt

from onnxruntime import InferenceSession
from processor import preprocess, postprocess

# 输入输出设置
img_path = [path to the input image]
output_dir = [output dir]

# 数据预处理
img = preprocess(img_path)

# 加载模型
model = InferenceSession([path to the save onnx model])

# 模型推理
start = time.time()
d0, _, _, _, _, _, _ = model.run(output_names=None, input_feed={
    
    'image': img})
end = time.time()
print('predict time: %.04f s' % (end - start))

# 结果后处理
mask_path, result_path = postprocess(d0, img_path, output_dir)

# 图像显示
img = np.concatenate([
    cv2.imread(img_path),
    cv2.imread(mask_path),
    cv2.imread(result_path)
], 1)
plt.axis('off')
plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
plt.show()

predict time: 0.7305 s

Summarize

• Paddle2.0 currently has a built-in interface for exporting ONNX models, which can easily export ONNX models
• For the previous static graph model, the Paddle2ONNX tool can also be used for conversion
• It is also relatively simple and convenient to use the ONNX model for reasoning deployment
• However, there are still compatibility issues with some Paddle operators, causing some models to fail to export normally.
• The overall experience is good, and I hope that this function will be more perfect in the future and support more operators