PaddleDetection2.x trains and deploys its own model

PaddleDetection is a target detection development kit of the Baidu Paddle family. Personally, I feel that the advantage of Paddle is that it has richer models and supports more deployment methods (python, C++, mobile terminals, etc.). The disadvantage is that there are more pitfalls. This article studies the use process of PaddleDetection, and records the entire training, reasoning, and deployment process from my own data set (used in my work).

1. Installation of PaddleDetection

The installation is relatively simple, just refer to the official tutorial. Installation tutorial path: PaddleDetection/docs/tutorials/INSTALL.md

Let me first talk about a big pit that I encountered during installation:

Paddle2.3 (the latest) is not compatible with ubuntu22.04, because the gcc version of ubuntu22.04 is higher, and paddle does not support such a higher gcc version.

The version installed in this blog post is paddlepaddle-gpu==2.3.1 paddledetection==2.4 

2. PaddleDetection trains its own data set (coco)

2.1 Paddle detection file introduction

The picture shows the file structure of paddle detection, where configs is the configuration file, basically the configuration file defines all the structures, data sets, training strategies, etc. of the network. output is used to store the trained model, and tools contains basic tool scripts such as train val. 

2.2 Construct your own dataset

paddledetection supports voc, coco, and widerface formats, among which coco has the most models. The data format of coco depends on labelme's labeling file. After labeling, it can be converted into coco's data format. As for the converted code, there are many labelme2coco scripts on the Internet, and tools/x2coco.py in paddlededeciont's file directory is also available. The purpose is to construct the corresponding coco data format as follows:

 

Among them, the annotations directory contains instances_train2017.json and instaances_val2017.json (for detection and segmentation), the train2017 directory (the original image of the training set) and val2017 (the original image of the verification set). For coco data set description, please refer to PaddleDetection/docs/tutorials/PrepareDataset.md

This time I am using a target detection data set containing 5 categories. It contains a total of 2k pictures. It has been divided into training set and verification set according to the ratio of 9:1. Since the data involves work and business, it will not be shown in detail. explained.

2.3 Modify the configuration file

This article selects the faster_rcnn_r50_vd_fpn_ssld_2x_coco.yml model for training. In the description file configs/faster_rcnn/README.md, this network has reached the optimal map on the coco data: 42.6

Open configs/faster_rcnn/faster_rcnn_r50_vd_fpn_ssld_2x_coco.yml and find that this is a configuration file of nesting doll mode, just open it layer by layer and configure it.

The general configuration will include three main contents: data configuration, training strategy, and network model. As shown below

Among them, the data configuration is in coco_detection.yml

• num_classes: Modify the number of categories of your own data set
• TrainDataset: training set path configuration
• EvalDataset: validation set path configuration
• TestDataset: test set path configuration, just use the verification set as the test set

Configure network faster_rcnn_r50_fpn.yml

This is a network file, which defines backbone, head and other information. If you do not modify the model yourself (big brother level), this file basically does not need to be modified, just use the official network architecture. Note that the pretrained pre-training model here is of SwinTransformer, not of the entire network. 

Configure the training strategy faster_fpn_reader.yml, optimizer_1x.yml, runtime.yml. In fact, the content of the above configuration file can be understood from the name.

• reader.yml indicates the reading configuration of the data tensor, including input_size, batch_size, transformer and other parameter settings;
• optimizer_swin_1x.yml represents the configuration of the optimizer, including learning_rate, optimizer and other parameter settings;
• runtime.yml represents the parameter configuration during operation, including use_gpu, log_iter, save_dir and other parameters

2.4 Training

Train & Val & Infer

# Train
export CUDA_VISIBLE_DEVICES=0 
python tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_vd_fpn_ssld_2x_coco.yml --eval --use_vdl=true

• -c : configuration file.yml
• --eval : switch to evaluate while training
• --use_vdl : Visualize the training process, used in pairs with the --vdl_log_dir parameter

Among them, the default value of --vdl_log_dir is vdl_log_dir/scalar. When --use_vdl=true, a .log file of the training process will be generated to record the changes of loss and map. then use the command

visualdl --logdir vdl_log_dir/scalar/

A web service can be generated in " http://localhost:8040/ ", and the loss changes can be seen. As follows:

 

# Val
export CUDA_VISIBLE_DEVICES=0 
python tools/eval.py -c configs/faster_rcnn/faster_rcnn_r50_vd_fpn_ssld_2x_coco.yml -o weights=output/faster_rcnn_r50_vd_fpn_ssld_2x_coco/ model_final.pdparams --classwise

• -c : configuration file.yml
• -o : weight file.pdparams
• --classwise : Output AP values ​​for each individual class

# Infer
export CUDA_VISIBLE_DEVICES=0
python tools/infer.py -c configs/faster_rcnn/faster_rcnn_r50_vd_fpn_ssld_2x_coco.yml -o weights=output/faster_rcnn_r50_vd_fpn_ssld_2x_coco/model_final.pdparams --infer_img=demo/DJI_1116.png --output_dir=output/ --draw_threshold=0.5 --use_vdl=True

•  -c : set configuration file.yml
• -o : set weight file.pdparams
• --infer_img : image path
• --output_dir : Path to save results
• --draw_threshold : Confidence threshold for bounding box selected when drawing

The above is a complete Train & Val & Infer process. But in actual use, it is often not often used like this, because the Infer here is coupled with a lot of code in the training part, and the call is not flexible enough. In actual use, another step is deploy. Similarly, there is an Infer process in deploy, which does not depend on the training process and is optimized to be better embedded in other projects.

3. Model export and deployment 

The model file saved during the model training process includes forward prediction and backpropagation. In actual industrial deployment, backpropagation is not required, so the model needs to be imported into the model format required for deployment. The tools/export_model.py script is provided in PaddleDetection to export the model

Python deployment method

# 导出模型
python tools/export_model.py -c configs/faster_rcnn/faster_rcnn_r50_vd_fpn_ssld_2x_coco.yml -o weights=output/faster_rcnn_r50_vd_fpn_ssld_2x_coco/model_final.pdparams --output_dir=./inference_model

 After the model is exported, four files are generated in the corresponding folder as follows:

infer_cfg.yml : preprocessing transformer parameters, input_size parameters, etc.

model.pdiparams

model.pdiparams.info

model.pdmodel

# 预测
python deploy/python/infer.py --model_dir=./inference_model/faster_rcnn_r50_vd_fpn_ssld_2x_coco --image_file=demo/DJI_1116.png --device=GPU

There are too many files and codes for paddle detection, and they are not used in actual deployment. More ways to use it should be: embed paddle into the project to use its reasoning process. Because after the model is exported, it has nothing to do with the training process, so we only need to copy the model folder and deploy folder needed in ./inference_model to other projects to use its inference function.

It is still necessary to refer to the source code of deploy/python/infer.py. It is best to write an Infer function by yourself to meet your business needs. Examples are as follows:

import os
import yaml
import glob
import json
from pathlib import Path
import time

import cv2
import numpy as np
import math
import paddle
from paddle.inference import Config
from paddle.inference import create_predictor

from infer import Detector, visualize_box_mask


# paddle.enable_static()
model_dir = "/home/elvis/CodeReconstruction/Det_Tinktek_V2.0/paddledetection/inference_model/faster_rcnn_r50_vd_fpn_ssld_2x_coco"

detector = Detector(model_dir,
                 device='GPU',
                 run_mode='paddle',
                 batch_size=1,
                 trt_min_shape=1,
                 trt_max_shape=1280,
                 trt_opt_shape=640,
                 trt_calib_mode=False,
                 cpu_threads=1,
                 enable_mkldnn=False,
                 enable_mkldnn_bfloat16=False,
                 output_dir='output',
                 threshold=0.5,
                 delete_shuffle_pass=False)

img_path = "/home/elvis/paddle/PaddleDetection/demo/1116.png"
frame = cv2.imread(img_path)
results = detector.predict_image([frame[:, :, ::-1]], visual=False)  # bgr-->rgb
print(results)
print(detector.det_times.info())
im = visualize_box_mask(frame, results, detector.pred_config.labels, detector.threshold)
im = np.array(im)
cv2.imshow('Mask Detection', im)
cv2.waitKey(0)


# def pre_img(detector, frame:cv2):
#     results = detector.predict_image([frame[:, :, ::-1]], visual=False)  # bgr-->rgb

    

 

4. Possible errors

When I use the model configs/faster_rcnn/faster_rcnn_swin_tiny_fpn_3x_coco.yml for training, the reasoning with tools/infer.py is also normal, but after the model is exported, the reasoning error using deploy/python/infer.py is as follows. I don’t know the problem for the time being. where.