Table of contents
System Configuration
Operating system: Linux (Galaxy Kylin)
Compiler: C++ compiler (such as g++)
cpu: Feiteng
Introduction to ncnn
ncnn (Ncnn Convolutional Neural Network) is a lightweight high-performance deep learning framework developed by Tencent. It is specifically optimized for mobile and embedded devices and designed to provide efficient inference performance and low memory footprint.
Here are some key features and advantages of ncnn:
Lightweight and high performance: ncnn is designed as a lightweight framework with efficient inference performance. It uses an optimized computational graph and memory management strategy to minimize memory footprint and computational overhead.
Cross-platform support: ncnn supports multiple operating systems and hardware platforms, including Android, iOS, Linux, Windows, etc. It can run on different processor architectures such as ARM, x86 and MIPS etc.
Hardware acceleration support: ncnn provides support for a variety of hardware acceleration libraries, including OpenMP, Vulkan, ARM Compute Library, and Metal. These hardware-accelerated libraries can provide higher computing performance while keeping power consumption low.
Flexible model conversion tool: ncnn provides a model conversion tool that can convert models of mainstream deep learning frameworks (such as Caffe, TensorFlow, ONNX, etc.) to ncnn's model format. This allows users to easily deploy existing models on ncnn.
Open source and active community support: ncnn is open source and its source code is hosted on GitHub. It has an active community where users can get support, ask questions, and contribute code.
All in all, ncnn is a high-performance deep learning framework focused on mobile and embedded devices. Its lightweight and efficient performance make it a good application potential in resource-constrained environments, especially for scenarios such as mobile applications, embedded devices, and edge computing.
ncnn installation steps
1. Download ncnn source code
git clone https://github.com/Tencent/ncnn.git
2. Enter the ncnn source code directory
cd ncnn
3. Create a build directory and enter
mkdir build
cd build
4. Configure compilation parameters
cmake ..
Some options can be added as needed, such as:
-DNCNN_VULKAN=ON: enable Vulkan support
-DNCNN_OPENMP=ON: enable OpenMP support
-DNCNN_PIXEL_ROTATE=ON : enable pixel rotation support
-DNCNN_PIXEL_AFFINE=ON: enable pixel affine transformation support
-DNCNN_BUILD_EXAMPLES=ON: compile example code
After adding the build options, but with the following build options turned on, the supplemental pack should also be downloaded:
cmake -DCMAKE_BUILD_TYPE=Release -DNCNN_SYSTEM_GLSLANG=ON -DNCNN_VULKAN=ON -DNCNN_OPENMP=ON -DNCNN_PIXEL_ROTATE=ON -DNCNN_PIXEL_AFFINE=ON -DNCNN_BUILD_EXAMPLES=OFF ..
In the ncnn root directory, use the following command to complete the module:
git submodule update --init
In Git, a submodule is when one Git repository is nested inside another Git repository. Typically, the main project repository will use submodules to reference other repositories as dependencies or subprojects.
There are two things you can do with the git submodule update --init command:
Initialize submodules: If the main project contains references to submodules, but the contents of the submodules have not been downloaded to the local, then this command will download the contents of the submodules to the local and initialize the submodules as an available Git repository.
Update submodule: If the submodule in the main project already exists locally and there is a new commit in the remote repository, then this command will update the submodule to the latest commit.
To summarize, the git submodule update --init command is used to initialize and update submodules in the main project, ensuring that the contents of the submodules are up-to-date and available to the main project.
5. Compile ncnn
make -j8
6. Install ncnn
sudo make install
7. Configure environment variables
Add the library path of ncnn to the LD_LIBRARY_PATH environment variable, for example:
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
ncnn is installed.
onnx model to ncnn model
Model conversion: commonly used online model conversion address
https://convertmodel.com/#outputFormat=ncnn
Modes for command-line conversion:
./onnx2ncnn model.onnx model.param model.bin## ncnn推理示例
If you want to generate the onnx2ncnn tool by the way when compiling, you need to install the protobuf library before compiling ncnn, otherwise it will not be generated:
protobuf library installation
download link:
wget https://github.com/protocolbuffers/protobuf/releases/download/v3.4.0/protobuf-cpp-3.4.0.zip
unzip /home/kylin/ncnnbuild2/protobuf-cpp-3.4.0.zip
cd protobuf-3.4.0/
./configure
make -j16
make install
If you want to make an fp16 model, you can use the model optimization tool as follows:
./ncnnoptimize ncnn.param ncnn.bin new.param new.bin flag
注意这里的flag指的是fp32和fp16，其中0指的的是fp32，1指的是fp16
#include <iostream>
#include "net.h"
int main() {
// 创建ncnn网络对象
ncnn::Net net;
// 加载ncnn模型和参数文件
std::string model_path = "model.param";
std::string weights_path = "model.bin";
net.load_param(model_path.c_str());
net.load_model(weights_path.c_str());
// 创建输入数据
ncnn::Mat input = ncnn::Mat::from_pixels(image_data, ncnn::Mat::PIXEL_BGR, image_width, image_height);
// 创建输出数据
ncnn::Mat output;
// 设置输入数据
ncnn::Extractor ex = net.create_extractor();
//设置推理的轻量模式
ex.set_light_mode(true);
ex.input("input", input);
// 进行推理
ex.extract("output", output);
// 处理输出结果
float* result = output.channel(0);
// ...
// 释放资源
net.clear();
return 0;
}
Note:
ex.set_light_mode(true); is used to set the light mode of ncnn::Extractor.
In ncnn, lightweight mode is an optimization option, which can reduce memory consumption and calculation to a certain extent to improve inference speed. Lightweight mode usually sacrifices some inference accuracy, but for some application scenarios with high real-time requirements, this performance optimization is very valuable.
After setting ex.set_light_mode(true);, the Extractor object will try to use lightweight algorithms and data structures when performing model inference to reduce the use of memory and computing resources. However, it should be noted that the lightweight mode may affect the inference accuracy of the model, so when using the lightweight mode, you should choose carefully and make a trade-off according to actual needs.
If you don't need to use the light mode, you can change ex.set_light_mode(true); to ex.set_light_mode(false); or omit this line of code, and the light mode is turned off by default.