Article directory
1.OpenPCDet
- OpenPCDet official github
- Zhihu introduction of the framework author
- Point cloud 3D target detection code library
- Data-model separation normalized coordinate system
- 3D bounding box: (cx, cy, cz, dx, dy, dz, heading)
- (cx, cy, cz) is the geometric center position of the object’s 3D frame
- (dx, dy, dz) are the lengths of the object's 3D frame along the three directions of xyz when the heading angle is 0.
- Heading is the orientation angle of the object in the top view (0 degrees along the x-axis, and the angle increases counterclockwise from x to y)
- 3D bounding box: (cx, cy, cz, dx, dy, dz, heading)
- Model modular design, config file specifies modules, PCDet automatic topology combination framework
- Code concise reconstruction based on numpy+PyTorch data enhancement module and data preprocessing module
- Multiple high-performance 3D target detection algorithms open sourced in PCDet
- Train your own data set
- Loading your own data point cloud and 3D label box in self.getitem() are transferred to the aforementioned unified coordinate definition and entered into self.prepare_data() provided by the data base class .
- In self.generate_prediction_dicts(), receive the 3D detection box predicted by the model and expressed in a unified coordinate system, and convert it back to the format you need.
- Combined Improvement Model
- Change the corresponding module to replace the original module
2.OpenPCDet training KITTI data set
- The installation of the environment can be compiled according to the official documentation. python setup.py install
2.1Placement of KITTI data set
- < KITTI data set official website >
- Sharing and brief analysis of KITTI data set
- Place the downloaded data set in the mmdetection project path in the following order (a new ImageSets folder is created to store the data set division text)
OpenPCDet
├── pcdet
├── tools
├── checkpoints
├── data
│ ├── kitti
│ │ ├── ImageSets
│ │ ├── testing
│ │ │ ├── calib
│ │ │ ├── image_2
│ │ │ ├── velodyne
│ │ ├── training
│ │ │ ├── calib
│ │ │ ├── image_2
│ │ │ ├── label_2
│ │ │ ├── velodyne
2.2 Preprocessing of data sets
# 这一步后会生成下方所示的gt_database(里面是根据label分割好的点云)及.pkl文件
python -m pcdet.datasets.kitti.kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/kitti_dataset.yaml
kitti
├── ImageSets
│ ├── test.txt
│ ├── train.txt
├── testing
│ ├── calib
│ ├── image_2
│ ├── velodyne
├── training
│ ├── calib
│ ├── image_2
│ ├── label_2
│ ├── velodyne
├── gt_database
│ ├── xxxxx.bin
├── kitti_infos_train.pkl
├── kitti_infos_val.pkl
├── kitti_dbinfos_train.pkl
├── kitti_infos_trainval.pkl
- After generating the corresponding file, you can start training.
cd tools
python train.py --cfg_file cfgs/kitti_models/pointpillar.yaml --batch_size=1 --epochs=10 --workers=1
- Test the performance of the generated model test.py
# 测试生成模型的性能 test.py
python test.py --cfg_file cfgs/kitti_models/pointpillar.yaml --batch_size 1 --ckpt ../output/kitti_models/pointpillar/default/ckpt/checkpoint_epoch_9.pth
# 可视化模型预测效果 demo.py
python demo.py --cfg_file cfgs/kitti_models/pointpillar.yaml --data_path ../data/kitti/testing/velodyne/000001.bin --ckpt ../output/kitti_models/pointpillar/default/ckpt/checkpoint_epoch_9.pth
2.3OpenPCDet training KITTI error: KeyError: 'road_plane'
- Modify the following code, the number of lines may be different, mainly plane related code
- tools/cfgs/dataset_configs/kitti_dataset.yaml
- 23行 USE_ROAD_PLANE: true 将true改为False
- pcdet/datasets/augmentor/data_augmentor.py
- 225-228行 注释掉
if 'road_plane' in data_dict:
data_dict.pop('road_plane')
- pcdet/datasets/augmentor/database_sampler.py
- 161-167行 注释掉
if self.sampler_cfg.get('USE_ROAD_PLANE', False):
sampled_gt_boxes, mv_height = self.put_boxes_on_road_planes(
sampled_gt_boxes, data_dict['road_plane'], data_dict['calib']
)
data_dict.pop('calib')
data_dict.pop('road_plane')
- 186-189行 注释掉
if self.sampler_cfg.get('USE_ROAD_PLANE', False):
# mv height
obj_points[:, 2] -= mv_height[idx]
3.OpenPCDet trains its own data set
- My own data set only has point clouds and annotation information, but no pictures and parameter calibration information.
- Use point-cloud-annotation-tool to annotate point cloud data. Since point-cloud-annotation-tool can only annotate point clouds in PCD and bin formats, and my point cloud format is in PLY format, I use it first. The script will PLY->PCD (wheel 1) the point cloud (actually it would be better to convert it directly into bin, so I will record my not very clever operation process first).
- After labeling, since the software label categories are limited, most of them are traffic-related categories, so I use the wheel to modify the categories in all the text in the saved directory (wheel 2) . Since I only set one category and marked one category, Modification is quite easy. The label information I have obtained so far is (class, x, y, z, l, w, h, θ) (category, x, y, z, length, width, height, angle with the y-axis)
- I naively thought that my data set was ready, and I found several methods on the Internet to train my own data set. Because I was too good at it, I couldn't implement any of the big guys' methods. Here are some training methods from the big guys. Tutorial links may provide some inspiration for training.
- There is no label box in the prediction after training. If it is not very successful and you are short on time, don't read it. If you have time, you can take a look at the processing ideas. If there may be deviations in your understanding, please correct me.
- 3D target detection (4): OpenPCDet training chapter – custom data set
- openPCdet implements custom point cloud data set training
- Openpcdet-(2) Self-dataset training data set training
3.1 Do not use KITTI format
- Under GitHub's issues Training using our own dataset, the tutorial and code issued by 155cannon are relatively detailed and in Chinese, and the processing idea does not require converting the dataset label format to KITTI format (I really feel that the parameters in the KITTI label are too Many, observe the angle between the angle of view and the y-axis, etc., and the center point coordinates are the center point of the bottom surface instead of the center point of the frame, and the order is still hwlxyz (height, width, length xyz)), so I am going to follow his ideas. Training, let’s briefly describe his processing flow. You can also go to issues to look at the code of the boss, but I didn’t succeed. If you are anxious to train, you can just skip to 3.2 Using KITTI format annotation.
- The data set point cloud file is converted to txt format and only xyz is stored. The label file only stores x, y, z, dx, dy, dz, heading, and heading are uniformly set to 1.57, which is 90°.
- Add three files under openPCDet, namely
- pcdet/datasets/kitti/dataset.py will be used when preprocessing the data set, mainly to generate the .pkl files and gt_database needed for training.
- tools/cfgs/dataset_configs/coils_dataset.yaml is used together with the previous file during data set preprocessing. It mainly modifies the point cloud range POINT_CLOUD_RANGE and some settings of label category information.
- tools/cfgs/coils_models/pointpillar.yaml Network information used during training, mainly modifying category information and the used dataset configuration file path
- So I started looking for a wheel on the Internet to convert the point cloud I just converted from PLY->PCD to PCD->TXT (Wheel 1) (these are all wheels that traverse the entire folder to modify, and wait until I have time to convert these point cloud formats later) Integrate it)
- Then use the wheel to modify the category name in 1.1 to directly remove the category information in the label, so that the format of my data set is almost the same as his.
- However, maybe my operation was wrong, and the error reported by Baidu failed to be corrected. Well, then I will use another method to process it, and convert my own data set to KITTI mode for processing.
3.2 Use KITTI format annotation
- Sharing and brief analysis of KITTI data set
- In the latest article under issues Training using our own dataset on GitHub , I saw that the boss shared his tutorial on converting annotation labels to KITTI format for training . Following his method, my data set preprocessing and training were successfully completed. The corresponding .pth file was also generated, but during testing it was found that the label box was not generated, and it may need to be changed. The following is the processing flow
- The input point cloud format needs to be in bin format, so the point clouds that have been converted to PCD format before (fortunately, they have not been overwritten or deleted before) are batch converted to bin format ( wheel 1) . When converting, it reminds that there is no intensity information, but still The conversion is successful and can be visualized, and there seems to be no problem with the training. I wonder if this is the reason why the test box is not generated.
- Because it was marked with point-cloud-annotation-tool before, there is information about the angle between the xyz length, width, and height and the y-axis, so 7 0s (wheel 2) were added after the category information of the label (these parameters are suitable for images without images and The camera parameters are of little significance and can be directly set to 0), and the preparation of the data set is almost complete.
- The next step is to create the following three files. It is recommended to go directly to the boss's GitHub to save them.
- Change the category information, data set path, and point cloud range POINT_CLOUD_RANGE in the above files. No other major changes are required.
- Change the pcdet/datasets/ init.py file and add the following code
from .custom.custom_dataset import CustomDataset # 在__all__ = 中增加 'CustomDataset': CustomDataset- After completing the above, you can start preprocessing and training the data set.
custom
├── ImageSets
│ ├── test.txt
│ ├── train.txt
├── testing
│ ├── velodyne
├── training
│ ├── label_2
│ ├── velodyne
├── gt_database
│ ├── xxxxx.bin
├── custom_infos_train.pkl
├── custom_infos_val.pkl
├── custom_dbinfos_train.pkl
python -m pcdet.datasets.custom.custom_dataset create_custom_infos tools/cfgs/dataset_configs/custom_dataset.yaml
python tools/train.py --cfg_file tools/cfgs/custom_models/pointrcnn.yaml --batch_size=1 --epochs=10 --workers=1
-
Let me talk about the mistakes I encountered during training to avoid pitfalls for others.
-
After the initial test, I checked my label data and found that the data of my middle digits was <xyz length, width, and height> while the middle digits of KITTI were <height, width, and length xyz>, so the .bin file generated in my gt_database was initially There are no points in the data in 0kb (I don’t know why it can be trained like this). The label data is in the wrong position and the points cannot be cropped from the point cloud.
-
After correcting the label data, re-preprocess and train the data set. The .bin file generated in the gt_database folder is correct and can also be trained (if there is an error when training, please comment out the line and run it) and generate the corresponding pth file, but there is still no box when testing.
-
Since my labels were initially generated using PCD file annotation in 3.1, I was wondering if the test would have no frames for this reason, so I re-labeled all the .bin point clouds. As a result, a ValueError was reported during training: Caught ValueError in DataLoader worker process 0. After changing the worker value to 0, this error was not reported, but the error was still reported ValueError: Cannot take a larger sample than population when 'replace=False'. I followed several blogs and changed it, but it still failed. The change was successful.
-
Update the error ValueError: Cannot take a larger sample than population when 'replace=False' and find a solution.
- I saw tutorials on the Internet because the size in numpy.random.choice(a, size=None, replace=True, p=None) is larger than the sample length of a and needs to be smaller, but at that time I directly modified the value of size. too successful
- Today I saw the same <issues> on GitHub that can be solved. The method is as follows
# 将pcdet/datasets/processor/data_processor.py大概第161行的 extra_choice = np.random.choice(choice, num_points - len(points), replace=False) # 修改为以下代码 try: extra_choice = np.random.choice(choice, num_points - len(points), replace=False) except ValueError: extra_choice = np.random.choice(choice, num_points - len(points), replace=True) -
- But a new error occurred during Evaluation: ZeroDivisionError: float division by zero
- Thanks to netizen wangwhan for sharing his debugging experience. This is his solution ["The data set was not read correctly during evaluation, resulting in the dataloader being empty. In fact, the name of the generated pkl file was inconsistent with the info path in the data set configuration file. Yes, it will be solved by changing the data set configuration file (yaml)"]
- Here are some more detailed code comment tutorials for easy viewing.
4. Convert point-cloud-annotation-tool annotation format to KITTI format
- Traverse the folder to modify each annotation file (make a backup copy first to prevent errors)
- I have added 7 zeros after the category (wheel 2) before doing this script conversion
- This script mainly changes <xyz length, width and height> to <height, width and height xyz>
- Secondly, through the source code of point-cloud-annotation-tool, we can know that its xyz is the geometric center coordinate of the annotation box, and KITTI is the coordinate of the bottom surface of the annotation box, so use the z-0.5 height to modify the z-axis coordinate to the KITTI form.
# -*- coding:utf-8 -*-
import os
def process(path):
files = os.listdir(path)
file_names = set([file.split('.')[0] for file in files])
file_names = list(file_names)
# 替换字符
print(file_names)
for filename in file_names:
file_data = ''
dir_path = os.path.join(path, filename + ".txt") # TXT文件
# f = open('E:/DataSet/label_2/cloudnorm_00001.txt') # 打开txt文件
print(type(dir_path))
print(dir_path)
f = open(dir_path) # 打开txt文件
line = f.readline() # 以行的形式进行读取文件
list1 = []
while line:
a = line.split()
b = a[0:15] # 这是选取需要读取/修改的列
list1.append(b) # 将其添加在列表之中
line = f.readline()
f.close()
# path_out = dir_path # 新的txt文件
with open(dir_path, 'w+', encoding='utf-8') as f_out:
for i in list1:
# print(len(i))
l0 = i[0]
l1 = i[1]
l2 = i[2]
l3 = i[3]
l4 = i[4]
l5 = i[5]
l6 = i[6]
l7 = i[7]
l8 = i[8]
l9 = i[9]
l10 = i[10]
l11 = i[11]
l12 = i[12]
l13 = i[13]
l14 = i[14]
a=round( (float(i[10])-0.5*float(i[13])),6)
# print(fir)
# print(str(sec))
f_out.write(
l0 + ' ' + l1 + ' ' + l2 + ' ' + l3 + ' ' + l4 + ' ' + l5 + ' ' + l6 + ' ' + l7 + ' ' + l13 + ' ' + l12 + ' ' + l11 + ' '+ l8 + ' ' + l9 +' ' + str(a) + ' ' + l14 + '\n')
print('trans has done!!!')
if __name__ == '__main__':
# 这里路径修改为你自己point-cloud-annotation-tool标注文件路径
path = 'E:/DataSet/text2/'
process(path)
- If you modify the label according to the above operation, you need to modify the function around line 127 of the pcdet/datasets/custom/custom_dataset.py file.
# 将原本的
loc_lidar = np.concatenate([np.array((float(loc_obj[2]), float(-loc_obj[0]), float(loc_obj[1]-2.3)), dtype=np.float32).reshape(1,3) for loc_obj in loc])
return loc_lidar
# 修改为
loc_lidar = np.concatenate(
[np.array((float(loc_obj[0]), float(loc_obj[1]), float(loc_obj[2])), dtype=np.float32).reshape(1, 3) for loc_obj in loc])
return loc_lidar
