1. Reference materials
2. Tracking dataset of KITTI dataset
1. Introduction to tracking dataset
trackingTasks are divided into three types, namely Multi-Object Tracking（多目标跟踪）, , Multi-Object Tracking and Segmentation（多目标跟踪和分割）and Segmenting and Tracking Every Pixel（分割和像素跟踪）. This article takes as Multi-Object Tracking（多目标跟踪）an example to introduce.
2. Download label annotation data
Label label data download: Download training labels of tracking data set (9 MB)
The directory structure of label label data is as follows:
├── training
│ └── label_02
│ ├── 0000.txt
│ ├── 0001.txt
│ ├── 0002.txt
│ ├── 0003.txt
│ ├── 0004.txt
│ ├── 0005.txt
│ ├── 0006.txt
│ ├── 0007.txt
│ ├── 0008.txt
│ ├── 0009.txt
│ ├── 0010.txt
│ ├── 0011.txt
│ ├── 0012.txt
│ ├── 0013.txt
│ ├── 0014.txt
│ ├── 0015.txt
│ ├── 0016.txt
│ ├── 0017.txt
│ ├── 0018.txt
│ ├── 0019.txt
│ └── 0020.txt
The label label data is xxxx.txtnamed after , taking as 0000.txtan example, the label format is as follows:
0 -1 DontCare -1 -1 -10.000000 219.310000 188.490000 245.500000 218.560000 -1000.000000 -1000.000000 -1000.000000 -10.000000 -1.000000 -1.000000 -1.000000
0 -1 DontCare -1 -1 -10.000000 47.560000 195.280000 115.480000 221.480000 -1000.000000 -1000.000000 -1000.000000 -10.000000 -1.000000 -1.000000 -1.000000
0 0 Van 0 0 -1.793451 296.744956 161.752147 455.226042 292.372804 2.000000 1.823255 4.433886 -4.552284 1.858523 13.410495 -2.115488
0 1 Cyclist 0 0 -1.936993 737.619499 161.531951 931.112229 374.000000 1.739063 0.824591 1.785241 1.640400 1.675660 5.776261 -1.675458
0 2 Pedestrian 0 0 -2.523309 1106.137292 166.576807 1204.470628 323.876144 1.714062 0.767881 0.972283 6.301919 1.652419 8.455685 -1.900245
1 -1 DontCare -1 -1 -10.000000 228.120000 183.030000 258.830000 217.340000 -1000.000000 -1000.000000 -1000.000000 -10.000000 -1.000000 -1.000000 -1.000000
1 -1 DontCare -1 -1 -10.000000 59.210000 191.300000 137.370000 227.430000 -1000.000000 -1000.000000 -1000.000000 -10.000000 -1.000000 -1.000000 -1.000000
...
...
...
3. Parsing label annotation data
For the field explanation of the label label data, please refer to: https://github.com/pratikac/kitti/blob/master/readme.tracking.txt.
kitti dataset The original dataset is divided into Road, City, Residential, Campusand Person. For 3D object detection, the label is subdivided into car, van, truck, pedestrian, pedestrian(sitting), cyclist, tramand misccomponents. Note DontCarelabelthat the area is not marked, for example, the target object is too far away from the lidar. In order to prevent false positives (false positives) from areas that are originally target objects but not marked for some reason during the evaluation process (mainly calculating precision), the evaluation script will automatically ignore the prediction results of the DontCare area.
4. Read label annotation data
Use the jupyter notebook tool to read label annotation data.
# 安装jupyter
pip install jupyter
# 启动jupyter
jupyter notebook
import sys
sys.path.remove('/opt/ros/kinetic/lib/python2.7/dist-packages')
import cv2
import pandas as pd
import numpy as np
import os
LABEL_NAME = ["frame", "track id", "type", "truncated", "occluded", "alpha", "bbox_left", "bbox_top", "bbox_right", "bbox_bottom", "dimensions_height", "dimensions_width", "dimensions_length", "location_x", "location_y", "location_z", "rotation_y"]
BASE_PATH = "/media/yoyo/U/2011_10_03/2011_10_03_drive_0027_sync"
data = pd.read_csv(os.path.join(BASE_PATH, "training/label_02/0000.txt"), header=None, sep=" ")
# 修改列名
data.columns = LABEL_NAME
data
3. Load the kitti dataset and display it in rivz
1. Prepare the KITTI dataset
Download and unzip the RawData data of the KITTI dataset.
2011_10_03_drive_0027_sync.zip
2011_10_03_calib.zip
file directory structure
.
├── 2011_10_03
│ ├── 2011_10_03_drive_0027_sync
│ │ ├── image_00
│ │ ├── image_01
│ │ ├── image_02
│ │ ├── image_03
│ │ ├── oxts
│ │ └── velodyne_points
│ ├── calib_cam_to_cam.txt
│ ├── calib_imu_to_velo.txt
│ └── calib_velo_to_cam.txt
2. Convert KITTI to bag file
Use kitti2bag to convert the KITTI dataset into a ROS bag file.
2.1 Installation dependencies
Before installing kitti2bag, ROS needs to be installed. Install whatever is missing.
pip install -U numpy
pip install opencv-python
pip install pycryptodomex
pip install gnupg
2.2 Install kitti2bag
# 安装pykitti
pip install pykitti
# 安装kitti2bag
pip install kitti2bag
kitti2bag installed successfully
yoyo@yoyo:~$ kitti2bag
usage: kitti2bag [-h] [-t DATE] [-r DRIVE]
[-s {
00,01,02,03,04,05,06,07,08,09,10,11,12,13,14,15,16,17,18,19,20,21}]
{
raw_synced,odom_color,odom_gray} [dir]
kitti2bag: error: too few arguments
2.3 Convert bag file
# 转换文件
kitti2bag -t 2011_10_03 -r 0027 raw_synced
yoyo@yoyo:/media/yoyo/U$ kitti2bag -t 2011_10_03 -r 0027 raw_synced
Exporting static transformations
Exporting time dependent transformations
Exporting IMU
Exporting camera 0
100% (4544 of 4544) |#####################| Elapsed Time: 0:00:34 Time: 0:00:34
Exporting camera 1
100% (4544 of 4544) |#####################| Elapsed Time: 0:00:38 Time: 0:00:38
Exporting camera 2
100% (4544 of 4544) |#####################| Elapsed Time: 0:01:29 Time: 0:01:29
Exporting camera 3
100% (4544 of 4544) |#####################| Elapsed Time: 0:01:29 Time: 0:01:29
Exporting velodyne data
100% (4544 of 4544) |#####################| Elapsed Time: 0:08:31 Time: 0:08:31
## OVERVIEW ##
path: kitti_2011_10_03_drive_0027_synced.bag
version: 2.0
duration: 7:50s (470s)
start: Oct 03 2011 12:55:34.00 (1317617735.00)
end: Oct 03 2011 13:03:25.83 (1317618205.83)
size: 24.0 GB
messages: 63616
compression: none [18184/18184 chunks]
types: geometry_msgs/TwistStamped [98d34b0043a2093cf9d9345ab6eef12e]
sensor_msgs/CameraInfo [c9a58c1b0b154e0e6da7578cb991d214]
sensor_msgs/Image [060021388200f6f0f447d0fcd9c64743]
sensor_msgs/Imu [6a62c6daae103f4ff57a132d6f95cec2]
sensor_msgs/NavSatFix [2d3a8cd499b9b4a0249fb98fd05cfa48]
sensor_msgs/PointCloud2 [1158d486dd51d683ce2f1be655c3c181]
tf2_msgs/TFMessage [94810edda583a504dfda3829e70d7eec]
topics: /kitti/camera_color_left/camera_info 4544 msgs : sensor_msgs/CameraInfo
/kitti/camera_color_left/image_raw 4544 msgs : sensor_msgs/Image
/kitti/camera_color_right/camera_info 4544 msgs : sensor_msgs/CameraInfo
/kitti/camera_color_right/image_raw 4544 msgs : sensor_msgs/Image
/kitti/camera_gray_left/camera_info 4544 msgs : sensor_msgs/CameraInfo
/kitti/camera_gray_left/image_raw 4544 msgs : sensor_msgs/Image
/kitti/camera_gray_right/camera_info 4544 msgs : sensor_msgs/CameraInfo
/kitti/camera_gray_right/image_raw 4544 msgs : sensor_msgs/Image
/kitti/oxts/gps/fix 4544 msgs : sensor_msgs/NavSatFix
/kitti/oxts/gps/vel 4544 msgs : geometry_msgs/TwistStamped
/kitti/oxts/imu 4544 msgs : sensor_msgs/Imu
/kitti/velo/pointcloud 4544 msgs : sensor_msgs/PointCloud2
/tf 4544 msgs : tf2_msgs/TFMessage
/tf_static 4544 msgs : tf2_msgs/TFMessage
After execution, the file is generated kitti_2011_10_03_drive_0027_synced.bag.
3. rviz visualization bag file
Use rviz to visualize bag files.
# 打开新终端，启动roscore
roscore
# 打开新终端，启动rviz
rosrun rviz rviz
yoyo@yoyo:~/catkin_ws$ rosrun rviz rviz
[ INFO] [1690354820.980297976]: rviz version 1.12.17
[ INFO] [1690354820.980329927]: compiled against Qt version 5.5.1
[ INFO] [1690354820.980337661]: compiled against OGRE version 1.9.0 (Ghadamon)
[ INFO] [1690354821.111348690]: Stereo is NOT SUPPORTED
[ INFO] [1690354821.111407271]: OpenGl version: 3 (GLSL 1.3).
4. Play the bag file
rosbag play kitti_2011_10_03_drive_0027_synced.bag
5. Check the bag
3. FAQ
Q：ImportError: /opt/ros/kinetic/lib/python2.7/dist-packages/cv2.so: undefined symbol: PyCObject_Type
ImportError: /opt/ros/kinetic/lib/python2.7/dist-packages/cv2.so: undefined symbol: PyCObject_Type
Solve the problem that python3.5 cannot import cv2.so
(demo) yoyo@yoyo:~$ python
Python 3.9.6 (default, Aug 18 2021, 19:38:01)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pykitti
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/yoyo/miniconda3/envs/demo/lib/python3.9/site-packages/pykitti/__init__.py", line 5, in <module>
from pykitti.tracking import tracking
File "/home/yoyo/miniconda3/envs/demo/lib/python3.9/site-packages/pykitti/tracking.py", line 12, in <module>
import cv2
ImportError: /opt/ros/kinetic/lib/python2.7/dist-packages/cv2.so: undefined symbol: PyCObject_Type
>>>
KeyboardInterrupt
错误原因：
安装ROS时，ROS的环境变量覆盖了Anaconda的环境变量
ROS默认的PYTHONPATH为python2.7
具体参见 /opt/ros/kinetic/_setup_util.py文件中的PYTHONPATH
默认使用 /opt/ros/kinetic/lib/python2.7/dist-packages/cv2.so
Method 1 (pro-test is effective)
退出Anaconda虚拟环境，在python2.7环境中安装kitti2bag
Method Two
In the python file, clear the environment variables of ROS.
import sys
sys.path.remove('/opt/ros/kinetic/lib/python2.7/dist-packages')
(demo) yoyo@yoyo:~$ python
Python 3.9.6 (default, Aug 18 2021, 19:38:01)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.remove('/opt/ros/kinetic/lib/python2.7/dist-packages')
>>> import cv2
>>>