Nanny-level tutorials, this one is enough.
After reading a lot of information on the Internet, I found that many versions of the information were more than matching.
It took a week to configure the environment and verify the environment, and then wrote this long article.
There are processes, codes, and experience. Everyone is welcome to criticize and correct.
1. Environment preparation
[edge device] yolov5 training and rknn model export and deployment on RK3588 ~ 1. Environment preparation (pro-test effective)
2. Environmental verification
[edge device] yolov5 training and rknn model export and deployment in RK3588 ~ 2. Environmental verification (pro-test is effective)
3. yolov5 trains its own model
[edge device] yolov5 training and rknn model export and deploys on RK3588 ~ 3. YOLOv5 trains its own model (effective for personal testing)
4. Export model
[edge device] yolov5 training and rknn model export and deploy on RK3588 ~ 4. Export model (effective for personal testing)
5. Deploy the model
[edge device] yolov5 training and rknn model export and deploy on RK3588 ~ 5. Model deployment (pro-test effective)
2. Environmental verification
(1) Board verification
Baidu network disk-rknn_yolov5_rk3588_v14 extraction code: 8888
Open it on the host computer Remmina, enter the IP address, user name, and password to connect to the development board. After downloading the above files, put them on the development board and decompress them.
It can be seen that running the model on RK3588using takes about . The board environment is already available.NPUyolov533.8 ms
(2) RKNN-Toolkit2-Lite (board end)
1. InstallationMiniconda
Download Miniconda on the board , the version is
bash Miniconda3-py39_23.3.1-0-Linux-aarch64.sh
Install all the way until it succeeds.
2. Virtual environment
After the end of the board is installed. Create and activate a virtual environment
conda create -n rknn python=3.9
conda activate rknn
into rknn_toolkit_lite2the directory,
3. Installation environment dependencies
pip install packages/rknn_toolkit_lite2-1.4.0-cp39-cp39-linux_aarch64.whl
If the installation is slow, modify the domestic pipsource,
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
pipIf other package libraries are missing, install them directly with .
4. Test verification
into the directory examples/inference_with_lite,
Just execute python test.pyit.
So far, the rknn_toolkit_lite2 on the board has been installed and verified. Later, the program can be developed and deployed on the board side python.
NOTE
If the execution
python test.pyprompts thatlibrknnrt.sothe orlibrga.solibrary is missing, please download it from github, or copylibrknnrt.soorlibrga.soto/usr/libthe directory in another directory.
If you want to start the virtual environment of rknn by default when you open the terminal on the board,
vi ~/.bashrc
add inconda activate rknn
(3) Host machine
1. Create and activate a virtual environment
Install rknnvirtual environment
conda create -n rknn python=3.8
conda activate rknn
Install pytorchvirtual environment
conda create -n pytorch
It can be simpler here, create pytorch environment dependencies, or follow pytorchthe official tutorial to download the corresponding package.
pip install ultralytics
What is lacking and what to make up for:pip install xx
2. Install dependencies
Enter the rknn virtual environment, and enter rknn-toolkit2the directory,
source activate rknn
pip install -r doc/requirements_cp38-1.4.0.txt
The contents of the requirements_cp38-1.4.0.txt file are as follows:
# if install failed, please change the pip source to 'https://mirror.baidu.com/pypi/simple'
# base deps
numpy==1.19.5
protobuf==3.12.2
flatbuffers==1.12
# utils
requests==2.27.1
psutil==5.9.0
ruamel.yaml==0.17.4
scipy==1.5.4
tqdm==4.64.0
bfloat16==1.1
opencv-python==4.5.5.64
# base
onnx==1.9.0
onnxoptimizer==0.2.7
onnxruntime==1.10.0
torch==1.10.1
torchvision==0.11.2
tensorflow==2.6.2
3. Installation environment
Enter the rknn virtual environment, and enter rknn-toolkit2the directory
source activate rknn
pip install packages/rknn_toolkit2-1.4.0_22dcfef4-cp38-cp38-linux_x86_64.whl
4. Enter the rknn virtual environment and verify whether the installation is successfulrknn
source activate rknn
pythonEnter the development environment
from rknn.api import RKNN
If no error is reported, it means that RKNNthe installation has been successful.
5. Test
On the host machine , enter rknnthe virtual environment,
cd examples/onnx/yolov5/
tree .
.
├── bus.jpg
├── dataset.txt
├── test.py
└── yolov5s.onnx
python test.py
6. Put rknnthe model on the board for verification
Put the generated in the above steps yolov5s.rknnon the board end for testing.
.
├── lib
│ ├── librga.so
│ └── librknnrt.so
├── model
│ ├── RK356X
│ │ └── yolov5s-640-640.rknn
│ ├── RK3588
│ │ ├── yolov5s-640-640.rknn
│ │ └── yolov5s.rknn
│ ├── RV110X
│ │ └── yolov5s-640-640.rknn
│ ├── bus.jpg
│ └── coco_80_labels_list.txt
├── out.jpg
├── rknn_yolov5_demo
└── testYolov5_rk3588.sh
Modify board file
bash testYolov5_rk3588.shneutral
./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg
for
./rknn_yolov5_demo model/RK3588/yolov5s.rknn model/bus.jpg。
and execute on the board sidebash testYolov5_rk3588.sh
ztl@ztl:~/workspace/demo_rknn_yolov5_rk3588_v14$ bash testYolov5_rk3588.sh
post process config: box_conf_threshold = 0.25, nms_threshold = 0.45
Read model/bus.jpg ...
img width = 640, img height = 640
Loading mode...
E RKNN: [05:25:08.050] failed to check rknpu hardware version: 0x46495245
E RKNN: [05:25:08.050] This rknn model is for RK3566, but current platform is RK3588
E RKNN: [05:25:08.050] rknn_init, load model failed!
rknn_init error ret=-1
If an error message appears, set the platform information on the host platform again;
Will
rknn.config(mean_values=[[0, 0, 0]], std_values=[[255, 255, 255]])
change into
rknn.config(mean_values=[[0, 0, 0]], std_values=[[255, 255, 255]], target_platform='rk3588')
and re-execute
python test.py
Generate yolov5s.rknnthe model and upload it to the corresponding position on the board.
Re-execute on the board side bash testYolov5_rk3588.sh, the operation effect is normal.
So far, the given conversion model can be used on the host computer , and the model can be uploaded to the board , which can run correctly.demo
NOTE: There are four ways to develop and deploy
- The first is to run the model
RKNN-Toolkit2on the simulation with the help of the function and get the inference results (on the PC side)NPURKNN- The second is to connect
RKNN-Toolkit2the board with the function ofPC,RKNNdistribute the model to the specifiedNPUdevice for inference and obtain the inference results (network reasoning is on the board side, and the script is written onPCthe side)- The third is to call
RKNN SDKtheC/C++languageAPIto write the test code, compile it with a cross-compiler, and copy the executable file to the board to run (develop and compile on the sidePC, run on the board side), here you can also connectVS Codethe board end, develop applications on the board end.- The fourth is to install
rknn-toolkit2-litetools on the board side, and usepythonscripts to reason on the board side (code development and operation are all on the board side)
References (the following are cited references for the compilation of this article)
- yolov5 trains the pt model and converts it to the rknn model, and deploys it on the RK3588 development board - the whole process from training to deployment
- RK3588 model reasoning summary
- NPU use
- Step by step to teach you how to use RK3588's NPU to cross-compile yolov5 on the computer to run on the ARM platform-哔哩哔哩
- Convert yolov5s.onnx to rknn model RK3588 motherboard real-time reasoning and computer-side simulation reasoning-哔哩哔哩
- Get rid of the computer, reason directly on the RK3588 platform, and help users deploy RKNN models to accelerate the implementation of AI applications-哔哩哔哩]
- Warehouse download Refer to the rknn-toolkit tutorial , you need to switch to the commit ID
c5360f6e7009eb4d05f14d1cc9dae0963e949213, you can download it directly from the official website github , and then execute it on the terminalgit checkout c5360f6e7.