CenterPoint engineering reproduction

NoteMMDet3d v0.17.3 : The version used in this article: the mmdet3d ( ) version under the BEVerse project , and the use of other versions in MMDetection3D 大同小异.
欢迎有疑问的同学在下方留言，如果觉得有用的话，可以点赞+收藏，再次表示感谢！

Reference :
1. Nuscenes official use example
2. CenterPoint paper speed reading
3. BEVerse paper speed reading

1. Download and install the project

Detailed reference : MMdet3d official document

pip install openmim
mim install mmcv-full   # 此时1.6.1版本
mim install mmdet
mim install mmsegmentation
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
pip install -e .

Note :
If you need to install a specific version of mmcv, you can refer to the following command. For details, please refer to the official mmcv document , mmdet3d_v1.1.0

pip install mmcv==2.0.0rc1 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.12.0/index.html

2. Prepare the nuscenes dataset

Refer to the previous article to download the full version of the Nuscenes dataset in batches . You can download the nuscenes v1.0-mini version if you are just playing. However, the data volume of the mini version is too small, and the training results must be unsatisfactory.

3. Data preprocessing and coordinate system transformation

python tools/create_data.py nuscenes --root-path ./data/nuscenes --out-dir ./data/nuscenes --extra-tag nuscenes --version v1.0-mini

Note :
1) If you use the official master branch code of mmdet3d, an error FileNotFoundError: NuScenesDataset: [Errno 2] No such file or directory will be reported during the generation process. Define the path, in order to ensure that the original data and the preprocessed data are in the same path, that is

mmdetection3d
├── mmdet3d
├── tools
├── configs
├── data
│   ├── nuscenes
│   │   ├── maps
│   │   ├── samples
│   │   ├── sweeps
│   │   ├── v1.0-test
|   |   ├── v1.0-trainval
│   │   ├── nuscenes_database
│   │   ├── nuscenes_infos_train.pkl
│   │   ├── nuscenes_infos_trainval.pkl
│   │   ├── nuscenes_infos_val.pkl
│   │   ├── nuscenes_infos_test.pkl
│   │   ├── nuscenes_dbinfos_train.pkl
│   │   ├── nuscenes_infos_train_mono3d.coco.json
│   │   ├── nuscenes_infos_trainval_mono3d.coco.json
│   │   ├── nuscenes_infos_val_mono3d.coco.json
│   │   ├── nuscenes_infos_test_mono3d.coco.json

2) MMDet3d versions after v1.0.x need to reconstruct the coordinate system.
The data needs to be reconstructed , especially the nuscenes dataset. Pay special attention to the mmdet3d version after v1.0, because the official coordinate system has been changed to avoid ambiguity. carried out 统一.
For details, please refer to Coordinate system refactoring , execute the script as follows, and save the new file to /data/nuscenes_convert_infos/.

python tools/update_data_coords.py nuscenes --root-dir ./data/nuscenes_infos/ --out-dir ./data/nuscenes_convert_infos/

4. Model training

4.1 Configuration file name description
Take as centerpoint_02pillar_second_secfpn_4x8_cyclic_20e_nus.pyan example

`{model}`: 模型类型 `centerpoint`.
`{model setting}`:体素尺寸和体素类型 ，如 `01voxel`, `02pillar`.
`{backbone}`: 主干网路，如 `second`.
`{neck}`: neck类型，如 `secfpn`.
`[dcn]`: 是否使用可变卷积.
`[circle]`: 是否使用循环nms
`[batch_per_gpu x gpu]`: 每个gpu上batch数目*gpu数目 , 默认4x8，即4batch×8gpu
`{schedule}`: 训练计划, 可选项 1x, 2x, 20e, etc. 1x = 12epochs ， 2x = 24 epochs ， 20e = 20 epochs. 对于 1x/2x, 初始化学习率在 8/16th and 11/22th epochs 衰减到0.1 . 对于 20e, 初始化学习率在16th and 19th epochs 衰减到0.1.
`{dataset}`: 数据集，如 nus-3d, kitti-3d, lyft-3d, scannet-3d, sunrgbd-3d. 我们也可制定类别或者使用单一类别，如 kitti-3d-3class and kitti-3d-car。

Note : During training, centerpoint_02pillar_second_secfpn_4x8_cyclic_20e_nus_novelo.pythe bug that the divisor will be reported as 0 at the position of the data sampler is not clear for the time being novelo?

4.2 Execute the training script

CUDA_VISIBLE_DEVICES=0,1,2,3 bash tools/dist_train.sh configs/centerpoint/centerpoint_02pillar_second_secfpn_4x8_cyclic_20e_nus.py 8

Note : The method is 4batch × 8gpu adopted . If the training is not configured in this way, the corresponding ones may need to be modified 超参, such as the learning rate.

5. Model Evaluation

5.1 Execute the test script

注意：官方在configs/centerpoint/metafile.yml提供了一些预训练模型的下载，可以根据需要自行下载并测试。

5.1.1 View mAP, NDS and other evaluation indicators

bash tools/dist_test.sh configs/centerpoint/centerpoint_02pillar_second_secfpn_4x8_
cyclic_20e_nus.py work_dirs/centerpoint_02pillar_second_secfpn_4x8_cyclic_20e_nus/epoch_20.pth 4 --eval mAP

Note : If a GPU, remove the --mtl
evaluation indicator screenshot (centerpoint_0075voxel_second_secfpn_circlenms_4x8_cyclic_20e_nus_20200925_230905-358fbe3b.pth):
insert image description here

5.1.2 Visualization

Offline visualization, suitable for servers
First generate the result file on the server, then copy it to the local for viewing, and view the visualization for details .

## 生成pkl检测结果文件
python tools/test.py configs/centerpoint/centerpoint_02pillar_second_secfpn_4x8_cyclic_20e_nus.py work_dirs/centerpoint_02pillar_second_secfpn_4x8_cyclic_20e_nus/epoch_50.pth --eval 'mAP' --eval-options 'show=False' --out './eval_visualize/mini_test.pkl'

## 针对pkl结果文件进行本地可视化
python tools/misc/visualize_results.py configs/centerpoint/centerpoint_02pillar_second_secfpn_4x8_cyclic_20e_nus.py --result ./data/mini_test.pkl --show-dir ./figs/

Visualize pkl detection effect:
insert image description here

The second idea is real-time visualization, which is suitable for this machine and supports GUI display instead of server, but it can only display预测值

python tools/test.py configs/centerpoint/centerpoint_02pillar_second_secfpn_4x8_cyclic_20e_nus.py checkpoints/centerpoint_epoch_50.pth --show --show-dir ./figs

Online visualization:
insert image description here

5.2 Summary of Possible Reasons for Abnormal Evaluation Results

After preprocessing the data, whether to execute the coordinate system reconstruction script, very old 老版本不需要to perform coordinate system reconstruction.
Note that using the version of the data set to which it belongs, if 用mini版本the evaluation results are used, many categories of APs will be 0, resulting in the final mAP being higher than that in the text 低6-8个点.

6. Bug debugging may encounter problems

Q1 : mmdet3d/ops/spconv/src/indice_cuda.cu 124
A1 : ops operator error

Q2 : ImportError: /opt/conda/lib/python3.8/site-packages/mmcv/_ext.cpython-38-x86_64-linux-gnu.so: undefined symbol:
A2 : Usually the mmcv version does not correspond to the torch version, Uninstall mmcv first, then reinstall, pay attention to modify the version number behind torch (gpu) below.

pip uninstall mmcv-full 
pip install mmcv==2.0.0rc1 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.12.0/index.html

Q3 : TypeError: 'DataContainer' object is not iterable occurs during the test
A3 : The mmcv version is wrong, you can try one by one according to the interval requirements, mmcv-full 1.6.0 is recommended, refer to Q2 for installation.

Q4 : fatal error: THC/THC.h: No such file or directory
A4 : The torch version needs to be downgraded.

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.1 -f https://download.pytorch.org/whl/torch_stable.html

Q5 : ImportError: cannot import name 'ball_query_ext' from partially initialized module 'mmdet3d.ops.ball_query'
A5 : After forgetting torch is reinstalled, the operator needs to be recompiled

cd mmdetection3d
python setup.py develop

Q6 : mportError: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory and returned non-zero exit status 1
A6 : Server problem, sometimes torch is not installed successfully, exit the virtual environment, and re-enter the installation

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.1 -f https://download.pytorch.org/whl/torch_stable.html