Windows uses ncnn vulkan to deploy yolox records (Xiao Baixiang, more detailed)

foreword

Before, I transplanted ncnn to deploy Yolo-Fastest on Raspberry Pi 4B, and also deployed a set in the windows of the notebook for debugging and development. But because of the age, many details are blurred. Today, I ran the yolox model again and found that it is much easier than at that time. The main reason is that I found out that there is a pre-compiled windows version in the official ncnn warehouse. You don’t need to compile it yourself, which is too convenient. too much.

Environment build

I am using the win10 x64 system, and mainly need to use 4 things:
1. Visual Studio Community 2019
2. ncnn corresponding to the compilation library of vs2019
3. The opencv library is also the official pre-compiled version of windows
4. The sdk of vulkan

• For vs2019,
  go to the Internet to find the installation file, and you can find it on the official website, but you have to look carefully, the current version is 2022 (of course, you can also install 2022, but ncnn needs to download the corresponding pre-compiled version), go to the link of the old version to find it, it is 1 An online installation file of more than 1MB. If you don’t want the volume to be too large during installation, just choose C++ desktop to develop 1 module is enough

• ncnn
  does not need to be compiled, just go to the official warehouse to download the pre-compiled version corresponding to Visual Studio, here I am: https://github.com/Tencent/ncnn/releases/download/20220420/ncnn-20220420-windows-vs2019.zip
  If the github download is slow, the code cloud also has

• 
  I chose version 4.5.4 of opencv , download link: https://opencv.org/releases/
  After decompressing opencv, you need to set the system environment path (corresponding to the storage path where you decompressed opencv), let vs 2019 find it path
  

• 
  Just download the latest version of vulkan (currently 1.3.211.0), all the way to next, it will be installed in the VulkanSDK directory of the c drive
  Address: https://vulkan.lunarg.com/sdk/home#windows

Then unzip opencv and ncnn into a folder, where I put it is D:\ncnn There
should be two folders in the directory

vs project configuration

Download the project for my test, which contains the inference code of ncnn yolox, the converted model of yolox and the test picture.
Network disk address:
https://pan.baidu.com/s/1g5XSDd30Kjalj6DtDWcS4A
Extraction code: 1234

After opening the project with vs 2019, you need to pay attention to the following configurations:

1. Set the configuration manager to Release. Because the official precompiled ncnn version is the release version, not the debug version, if it is not configured as Release, a bunch of errors will be reported when running

2. Check whether the configuration of VC++ directory -> include directory and library directory in the project properties is correctly configured with ncnn, opencv and vulkan, and it must be consistent with the directory path you actually store

3. The linker->input->additional dependencies in the properties should be as follows:

I included all the libs in the three libraries into
opencv_world455.lib (be careful not to choose opencv_world455d.lib, d means the debug version)
ncnn .lib
vulkan-1.lib
GenericCodeGen.lib
glslang.lib
MachineIndependent.lib
OGLCompiler.lib
OSDependent.lib
SPIRV.lib

4. Fill in the name of the reasoning picture in the command parameters in the attribute debugging

After saving, the project can be run and debugged. Because vulkan is used, ncnn will call the graphics card for acceleration. I compared it. It takes more than 100 ms to infer with the CPU, and it only takes more than 10 ms to use vulkan. It seems that the new version mentioned by ncnn has greatly optimized the acceleration effect of vulkan.
Code to switch vulkan acceleration:

	yolox.opt.use_vulkan_compute = true;
	// yolox.opt.use_bf16_storage = true;

Summarize

Now it is very easy to use ncnn to develop under windows. It is very convenient to debug C++ code with breakpoints through Visual Studio IDE, which is very helpful for learning how ncnn decodes various models. In addition, although ncnn is a neural network reasoning framework optimized for embedded cpu, with the continuous optimization of vulkan api, graphics card acceleration is also very practical. I think the advantage of vulkan is that it has wide compatibility, as long as the device supports the vulkan driver, the three major platforms of windows, macos, and linux + various gpus and npus are fine. I tried to use the vega core display on amd 5600h notebook and it can run correctly, but cuda can only run on nvidia hardware.