[Deep Learning] [Opencv] Python/C++ calls onnx model [Basics]

[Deep Learning] [Opencv] python/C++ calls onnx model [Basics]

Tip: The blogger has selected many blog posts from big guys and personally tested them to be effective. He shares his notes and invites everyone to study and discuss them together.

---

Preface

OpenCV is a cross-platform computer vision and machine learning software library (open source) released under the BSD license and can run on Linux, Windows, Android and Mac OS operating systems. The model trained in pytorch can be exported using ONNX, and then directly loaded using the dnn module in opencv.
Series learning directory:
[CPU] Detailed explanation of the Pytorch model to ONNX model process
[GPU] Detailed explanation of the Pytorch model to ONNX format process
[ONNX model] Rapid deployment
[ONNX model] Multi-threaded rapid deployment
[ONNX model] Opencv calls onnx

---

Python versionOpenCV

Install OpenCV on Windows platform

The blogger installed the anaconda environment in the win10 environment , and then built the openCV environment required to run the onnx model.

# 搭建opencv环境
conda create -n opencv_onnx python=3.10 -y
# 激活环境
activate opencv_onnx
# 安装opencv
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple opencv-python

View opencv version

import cv2
cv2.__version__

opencv calls onnx model

Now the code puts aside any pytorch-related dependencies and onnx-related dependencies, and only uses opencv to complete the entire inference process.

import cv2
import numpy as np

def normalizeImage(image,mean,std):
    normalized = image.astype(np.float32)
    normalized = normalized / 255.0 - mean
    normalized = normalized / std
    return normalized

def main():
    # 读取图片
    image = cv2.imread(r"./animal-1.jpg")
    # 将BGR图像转换为RGB格式
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    # 获取图像的大小
    ori_w, ori_h, = image.shape[0], image.shape[1]

    # 指定调整后的大小
    new_width = 416
    new_height = 416
    # 图片尺寸缩放
    resized_img = cv2.resize(image, (new_width, new_height), interpolation=cv2.INTER_AREA)

    # 定义每个通道的归一化参数
    mean = np.array([0.485, 0.456, 0.406]).astype(np.float32)  # 均值
    std = np.array([0.229, 0.224, 0.225]).astype(np.float32)  # 标准差
    # 图片归一化
    normalized = normalizeImage(resized_img, mean, std)

    # 加载ONNX模型
    net = cv2.dnn.readNetFromONNX("PFNet.onnx")  # 加载训练好的识别模型
    # onnx是多输出,每个输出都会对应一个name,因此需要获取所有输出的name
    output_layer_names = net.getUnconnectedOutLayersNames()
    blob = cv2.dnn.blobFromImage(normalized)  # 由图片加载数据 这里还可以进行缩放、归一化等预处理
    # 将Blob设置为模型的输入
    net.setInput(blob)
    # 运行前向传播,将所有输出name作为参数传递
    out = net.forward(output_layer_names)
    out = np.squeeze(out[3]) * 255.0
    output = cv2.resize(out, (ori_h, ori_w), interpolation=cv2.INTER_AREA)

    # 保存图像
    cv2.imwrite('saved_opencv_python_image.png', output)

if __name__ == '__main__':
    main()

---

C++ version OpenCV

Install OpenCV on Windows platform

Download the installation file address from the official website . The blogger uses the opencv-4.8.0-windows.exe version and double-clicks to run and decompress it to obtain the following files:
Open VS 2019: Create a new project---->Console application---->Configuration Project ----> Project path and check "Place solution and project in the same directory ----> Click Create.

Set OpenCV path: Project ----> Properties.

Add additional include directory: Debug | x64 ---->C/C+±—>General---->Additional include directories.

D:\C++_demo\opencv\build\x64\vc16\bin
D:\C++_demo\opencv\build\bin
D:\C++_demo\opencv\build\include
D:\C++_demo\opencv\build\include\opencv2

Linker: Debug | x64---->Linker---->General---->Additional include directories.

D:\C++_demo\opencv\build\x64\vc16\lib

Linker: Debug | x64---->Linker---->Input---->Additional dependencies.

Find the files for additional dependencies under D:\C++_demo\opencv\build\x64\vc16\lib.

opencv_world480d.lib

To test in Debug x64 mode, copy the opencv_world480d.dll file with d to the Debug directory of your project.

When there is no Debug directory, you need to run the code in Debug | x64 mode.

D:\C++_demo\opencv\build\x64\vc16\bin
===>
D:\C++_demo\opencv_onnx\x64\Debug

The blogger here installed the debug version for the convenience of debugging. Readers can install the release version. They only need to change the Debug |

opencv calls onnx model

Simple to use

Here is a simple verification of whether opencv is installed successfully. It is suitable for many people including bloggers who are not familiar with C++. The code completes the reading and display of simple images.

#include <opencv2/opencv.hpp>
using namespace cv;
using namespace std;
int main(int argc, char** argv) {
    
    
	Mat src = imread("./animal-1.jpg");
	//没有图像输入
	if (src.empty()) {
    
    
		printf("....\n");
		return -1;
	}
	//namedWindow("输入窗口", WINDOW_FREERATIO);
	imshow("输入窗口", src);
	waitKey(0);
	destroyAllWindows();
	return 0;
}

Call onnx model

After converting the python version of opencv into the corresponding c++ version, we found that the output effects are exactly the same. The onnx model can be used as a c++ interface for other applications to call.

#include <iostream>
#include <string>
#include <vector>
#include<opencv2/opencv.hpp>
#include <opencv2/dnn.hpp>
using namespace std;
cv::Mat normalizeImage(const cv::Mat& image, const cv::Scalar& mean, const cv::Scalar& std) {
    
    
    cv::Mat normalized;
    image.convertTo(normalized, CV_32F);
    cv::subtract(normalized / 255.0, mean, normalized);
    cv::divide(normalized, std, normalized);
    return normalized;
}
int main()
{
    
       
    // 读取图片
    cv::Mat bgrImage = cv::imread("./animal-1.jpg", cv::IMREAD_COLOR);
    // 图片格式转化bgr-->rgb
    cv::Mat rgbImage;
    cv::cvtColor(bgrImage, rgbImage, cv::COLOR_BGR2RGB);
    // 获取图像的大小
    cv::Size originalSize(rgbImage.cols, rgbImage.rows);
    cv::Mat resizedImage;

    // 定义目标图像大小
    cv::Size targetSize(416, 416);
    //图片尺寸缩放
    cv::resize(rgbImage, resizedImage, targetSize, 0, 0, cv::INTER_AREA);

    // 定义每个通道的归一化参数
    cv::Scalar mean(0.485, 0.456, 0.406); // 均值
    cv::Scalar std(0.229, 0.224, 0.225);  // 标准差
    // 图片归一化
    cv::Mat normalized = normalizeImage(resizedImage, mean, std);
    
    // 加载ONNX模型
    cv::dnn::Net net = cv::dnn::readNetFromONNX("D:/C++_demo/opencv_onnx/PFNet.onnx");
    cv::Mat blob = cv::dnn::blobFromImage(normalized);
    // 将Blob设置为模型的输入
    net.setInput(blob);
    // 运行前向传播
    std::vector<cv::Mat> output_probs;
    // 获取多输出对应的名称
    std::vector<cv::String> output_layer_names = net.getUnconnectedOutLayersNames();
    net.forward(output_probs, output_layer_names);
    cv::Mat prediction = output_probs[3];
    cv::Mat mask;
    cv::resize(prediction.reshape(1, 416) * 255.0, mask, originalSize, 0, 0, cv::INTER_AREA);
    cv::imwrite("saved_opencv_c++_image.png", mask);
    return 0;
}

---

Summarize

Introduce the process of calling ONNX model by POpencv under Python and C++ as simply and in detail as possible.