c++ deployment yolov5 model

foreword

It is undeniable that yolov5 has killed the Quartet in target detection and SOTAhas left much-anticipated results on the list. However, the official website code only gives pythonthe version of inferthe code. If the requirements are not high, it can barely be deployed. Now, I want to The main reasons winfor implementing c++the deployment process are as follows:

1. For the files compiled by c++, you can directly transfer cpall dlland exeto the target machine, but pythonyou need to install various environments;
2. c++ is more efficient than python;
3. winI feel c++that the convenience of deployment and transplantation is much higher than that python;

When it comes to the deployment of deep learning models, several common deployment tools are introduced, such as OpenVINO, tensorrt,, onnxruntimeetc., and one layer of encapsulation based on these frameworks, such as FastDeploy, mmdeployetc. These tools have their own characteristics, but for the current I Speaking of it, I want to complete the deployment of the ultracity/yolov5generated model as quickly as possible pt, so I chose the relatively simple method of FastDeployadding a link description
. Now this toolboxcan achieve many functions and supports many models. If necessary, you can check it out on the official website. Look, I won't introduce too much here.

1. Prepare the model

FastDeployWhen calling yolov5the model, it is not the file that is called directly , but the file (a general deep learning model) that is *.ptcalled . Therefore, it is first necessary to convert the file format, but not all codes support the function. Here, you need to pay attention to the ones that are greater than or equal to have this file, so if you are an old user, you don't need to read this article.*.onnx*.ptyolov5export *.pt to *.onnxyolov5tagv6.0export.py

python export --weights *.pt --include onnx

Two, Fastdeploy preparation

This still needs to be compiled by yourself. I cmakecompiled it with . The specific steps are:

1. Set ENABLEthe options under the directory: I use the visual module, so I set ENABLE_VISIONto ON, and, because I need to call onnx, also set ORT_BACKENDand PADDLE_BACKENDto ON, personal suggestion, choose what to use, don’t choose what not to use, I have encountered Over the pit, when compiling ENABLE_OPENVINO_BACKEND=ON, and then when the program was running, it got stuck. It turned out that it was because of a missing *.xmlfile. If no one pointed out, it is estimated that I will not know where the problem is in this month. 2. Setting with,
   personal suggestion WITH_GPU, because when running, it GPUwill be called if there GPUis, and if there is no, GPUit will be called CPU, and the compatibility is better.

three calls

If there is no problem in the first and second steps, congratulations, the subsequent calls will be simpler, just look at the code:

bool demo_yolov5onnx(const std::string& model_file, const std::string& img_path, const int& loop, const int& use_gpu) {
	fastdeploy::RuntimeOption option; // Configuration information for model inference 	                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
	if (use_gpu == 1) {
		option.UseGpu();
	} else {
		option.UseCpu();
	}
	auto model = fastdeploy::vision::detection::YOLOv5(model_file, "hello", option);

	if (!model.Initialized()) {
		std::cerr << "Failed to initialize." << std::endl;
		return false;
	}
	std::cout << "init successful" << std::endl;

	cv::Mat im = cv::imread(img_path);
	fastdeploy::vision::DetectionResult result;

	clock_t t1, t2;
	for (int i = 0; i < loop; i++) {
		t1 = clock();
		if (!model.Predict(im, &result)) {
			std::cerr << "Failed to predict." << std::endl;
			return false;
		}
		t2 = clock();
		std::cout << "predict image cost time: " << t2 - t1 << std::endl;
	}

	std::cout << "=================================" << std::endl;
	std::cout << result.Str() << std::endl;
	std::cout << "=================================" << std::endl;

	cv::Mat vis_im = fastdeploy::vision::VisDetection(im, result, 0.5);
	cv::imwrite("vis_result.jpg", vis_im); // The visualization results are saved locally
	return true;
}

The call is relatively simple, just three or five useful sentences.

Summarize

FastDeployThe package is quite simple and easy to use, but after all, it is a tool for secondary development, and its functions are still limited. Therefore, if you want to be more proficient in implementing model deployment, you still need to learn the bottom layer The use of the deployment tool ( OpenVINO，ONNXRuntime, TensorRT) and the understanding of the model framework itself.