[OpenMMLab AI Combat Camp Phase II Notes] MMPose Code Tutorial on the Third Day

The whole process of RTMPose key point detection

Code tutorial address: github.com/TommyZihao/MMPose_Tutorials

Note: All the following codes are running locally, local environment Ubuntu22, GPU RTX3090, CUDA v11.3

0. Configure the conda environment

conda create -n openmmlab-learning python=3.9 # 创建环境
conda activate openmmlab-learning # 激活环境
pip install jupyter #安装jupyternotebook
cd """到你的目录下面"""
git clone https://github.com/TommyZihao/MMPose_Tutorials.git # 拉取教程代码
jupyter-notebook#打开notebook

1. Install MMDetection and MMPose

1.1Install MMPose

After the installation is completed according to the sample code in the tutorial, the file structure is shown in the figure below, and no error is reported.

The figure below shows the downloaded videos and pictures in the data folder.

1.2 Install MMDetection

The installation process is similar to that in 1.1.

After installation, there will be two subfolders in the folder as shown in the figure above, namely mmdetection and mmpose

1.3 MMPose pre-trained model prediction-command line

The main running code: (the top-down demo used requires two weight files, target detection and key point detection)

# HRNet
!python demo/topdown_demo_with_mmdet.py \ #程序路径
        demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \ #配置文件路径
        https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
        configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py \
        https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth \
        --input data/test/multi-person.jpeg \ #输入路径
        --output-root outputs/B1_HRNet_1 \ #输出路径
        --device cuda:0 \ #计算设备
        --bbox-thr 0.5 \ #bbox的置信度阈值
        --kpt-thr 0.2 \ #关键点检测的置信度阈值
        --nms-thr 0.3 \ #框的非极大抑制的IOU（交并比）阈值
        --radius 8 \ #可视化关键点的半价
        --thickness 4 \ #线宽
        --draw-bbox \ #是否要画边界框
        --draw-heatmap \ #是否画热力图
        --show-kpt-idx #是否标明几号关键点

The result shows:

Compared with the above code, the following code just replaces the weight file of the model, and the others are the same

# RTMPose
!python demo/topdown_demo_with_mmdet.py \
        demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
        https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
        projects/rtmpose/rtmpose/body_2d_keypoint/rtmpose-s_8xb256-420e_coco-256x192.py \
        https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmpose-s_simcc-aic-coco_pt-aic-coco_420e-256x192-fcb2599b_20230126.pth \
        --input data/test/multi-person.jpeg \
        --output-root outputs/B1_RTM_1 \
        --device cuda:0 \
        --bbox-thr 0.5 \
        --kpt-thr 0.5 \
        --nms-thr 0.3 \
        --radius 8 \
        --thickness 4 \
        --draw-bbox \
        --draw-heatmap \
        --show-kpt-idx

Result display:

predicting video only needs to change the input file from the image path to the video path

1.4 MMPose pre-trained model prediction-Python API

Through the API provided by mmpose, write the whole process by hand, pay attention to the file path.

2. MMDetection triangle target detection

2.1 Download the triangle key point detection data set

Both the mmdetection and mmpose folders have to be run once.
After running in the mmpose folder, the content is as shown in the figure (the same is true for running in the mmdetection folder):

Supplement:
The COCO format data set is a commonly used image recognition and object detection data set format, jointly developed by Microsoft and Carnegie Mellon University. The dataset in this format contains a large number of images, object categories, object bounding boxes, and segmentation information of object instances.
COCO format data sets usually include the following files:
Image folder: contains all image files, and the file name is usually the unique identifier of the image.
Annotation file: contains all object categories, bounding boxes, segmentation information, etc., and is usually stored in JSON format.
Category file: contains all object category information, usually stored in JSON format.
Image list file: Contains all image file names, used for the division of training and test data sets.
The data in the annotation file contains the following fields:
"image_id": the unique identifier of the image.
"category_id": Category identifier of the object, corresponding to the category ID in the category file.
"bbox": The bounding box information of the object, including the coordinates of the upper left corner, width, height and other information.
"segmentation": Segmentation information of the object, usually a polygon or a mask.
"area": ​​the area size of the object.
"iscrowd": Identifies whether the object is a group (such as a group of people) and whether it needs to be segmented.

2.2 Download the config configuration file

Both the mmdetection and mmpose folders have to be run once, and two configuration files will be obtained respectively.

2.3 Training

Using the configuration file and data set provided in the tutorial, the training results will be stored in the path "mmdetection/work_dirs/faster_r_cnn_triangle/", and the log files will be in the folder named by date under this path

2.4 Visual training log

There is a detailed introduction to the evaluation indicators in the intensive reading of the YOLOV3 paper. By visualizing the training log, you can monitor whether there are errors in the training process. An evaluation index of -1 for small and medium targets indicates that there are no such targets in the data set.

2.5 Simplified conversion of model weight files

Convert large weight files into small weight files. This is because the pth file obtained during the training process contains a lot of information during training. We do not need this information when predicting. As the prompt in the figure below is written in

my After running this code, the model was reduced from 331.0 MB to 165.7MB

2.5 Download the trained model weights

If you don't want to train yourself, you can download the trained model, and be careful to run it in both folders.

2.6 Forecast

Prediction results will be saved as json files and pictures

{"bboxes": [[1816.48095703125, 1074.160400390625, 3174.092041015625, 3364.85791015625], [991.135009765625, 4024.948974609375, 3228.870361328125, 5352.5830078125], [191.60765075683594, 1780.585693359375, 1543.306640625, 4016.48291015625]], "labels": [0, 0, 0], "scores": [0.9999654293060303, 0.9999547004699707, 0.9999374151229858]}

3. MMPose, RTMPose triangle key point detection

The basic process and the target detection process are similar, and the path in the configuration file needs to be changed

4. Error sorting

(1) When running [D3] Triangular Plate Target Detection-Visual Training Log.ipynb , the error 'DataFrame' object has no attribute 'append' is reported.
Solution: Change .append to ._append
(2) When running [D3] Triangular Plate Target Detection - When visualizing the training log .ipynb , the error No such file or directory
is reported. Solution: Pay attention to modifying the path. The download target path must be modified to be under the matplotlib package of your virtual environment. At the same time, the cached file must be obtained through mpl.get_cachedir() Address, mine is ~/.cache/matplotlib/fontlist-v330.json, then delete the json file, and then modify the envs\openmmlab-learning\lib\site-packages\matplotlib\mpl-data\matplotlibrc file as follows

# 找到font.family，取消该行注释
font.family:  sans-serif
# 向font.sans-serif行中添加SimHei
font.sans-serif: SimHei, DejaVu Sans, ...

Then restart Terminal
(3) Run [F2] Triangular Plate Key Point Detection-Training RTMPose, error albumments is not installed
solution: pip install albumments -i https://pypi.tuna.tsinghua.edu.cn/simple