sfMLearning视觉无人驾驶项目 KITTI数据集结构与可视化

RawDATA原始网站
http://www.cvlibs.net/datasets/kitti/raw_data.php

1:数据采集平台

如图-1所示,KITTI数据集的数据采集平台装配有2个灰度摄像机,2个彩色摄像机,一个Velodyne 64线3D激光雷达,4个光学镜头,以及1个GPS导航系统。具体的传感器参数如下[2] :

2 × PointGray Flea2 grayscale cameras (FL2-14S3M-C), 1.4 Megapixels, 1/2” Sony ICX267 CCD, global shutter
2 × PointGray Flea2 color cameras (FL2-14S3C-C), 1.4 Megapixels, 1/2” Sony ICX267 CCD, global shutter
4 × Edmund Optics lenses, 4mm, opening angle ∼ 90◦, vertical opening angle of region of interest (ROI) ∼ 35◦
1 × Velodyne HDL-64E rotating 3D laser scanner, 10 Hz, 64 beams, 0.09 degree angular resolution, 2 cm distance accuracy, collecting ∼ 1.3 million points/second, field of view: 360◦ horizontal, 26.8◦ vertical, range: 120 m
1 × OXTS RT3003 inertial and GPS navigation system, 6 axis, 100 Hz, L1/L2 RTK, resolution: 0.02m / 0.1◦
在这里插入图片描述
如图-2所示为传感器的配置平面图。为了生成双目立体图像,相同类型的摄像头相距54cm安装。由于彩色摄像机的分辨率和对比度不够好,所以还使用了两个立体灰度摄像机,它和彩色摄像机相距6cm安装。为了方便传感器数据标定,规定坐标系方向如下[2] :
• Camera: x = right, y = down, z = forward
• Velodyne: x = forward, y = left, z = up
• GPS/IMU: x = forward, y = left, z = up
在这里插入图片描述

2.Dataset详述

图-3展示了KITTI数据集的典型样本,分为 ’Road’, ’City’, ’Residential’, ’Campus’ 和’Person’五类。原始数据采集于2011年的5天,共有180GB数据。 其中三天进行了重新标定:

- S_rect_xx: 1x2 size of image xx after rectification校正后的图片尺寸这里我们下载的部分是同步+矫正之后的图片,calibration的信息是传感器信息融合转换信息使用。tracklets是跟踪三维物体

在这里插入图片描述

http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_calib.zip

http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0001_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0002_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0005_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0009_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0011_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0013_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0014_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0015_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0017_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0018_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0019_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0020_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0022_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0023_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0027_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0028_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0029_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0032_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0035_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0036_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0039_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0046_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0048_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0051_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0052_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0056_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0057_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0059_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0060_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0061_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0064_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0070_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0079_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0084_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0086_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0087_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0091_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0093_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0095_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0096_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0101_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0104_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0106_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0113_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0117_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_26_drive_0119_sync.zip

http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_calib.zip 

http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0001_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0002_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0016_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0021_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0034_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0035_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0037_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0038_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0039_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0043_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0045_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0047_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0053_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0054_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0057_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0065_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0066_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0068_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0070_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0071_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0075_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0077_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0078_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0080_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0082_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0086_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0087_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0089_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0090_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0094_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0095_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0096_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0098_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0100_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0102_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0103_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0104_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0106_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0108_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0110_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0113_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0117_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0119_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0121_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0122_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0125_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0126_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0128_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0132_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0134_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0135_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0136_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0138_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0141_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0143_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0145_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0146_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0149_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0153_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0154_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0155_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0156_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0160_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0161_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0162_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0165_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0166_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0167_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0168_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0171_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0174_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0177_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0179_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0183_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0184_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0185_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0186_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0187_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0191_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0192_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0195_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0198_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0199_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0201_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0204_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0205_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0208_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0209_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0214_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0216_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0220_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0222_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_28_drive_0225_sync.zip

http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_29_calib.zip 

http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_29_drive_0004_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_29_drive_0026_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_29_drive_0071_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_29_drive_0108_sync.zip

http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_30_calib.zip.zip 

http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_30_drive_0016_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_30_drive_0018_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_30_drive_0020_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_30_drive_0027_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_30_drive_0028_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_30_drive_0033_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_30_drive_0034_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_09_30_drive_0072_sync.zip

http://datadownload.f3322.net:666/share/kitti/raw_data/2011_10_03_calib.zip 

http://datadownload.f3322.net:666/share/kitti/raw_data/2011_10_03_drive_0027_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_10_03_drive_0034_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_10_03_drive_0042_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_10_03_drive_0047_sync.zip
http://datadownload.f3322.net:666/share/kitti/raw_data/2011_10_03_drive_005_sync.zip

在这里插入图片描述

3数据组织形式

论文[2] 中提及的数据组织形式,可能是早期的版本,与目前KITTI数据集官网公布的形式不同,本文稍作介绍。
如图-4所示,一个视频序列的所有传感器数据都存储于data_drive文件夹下,其中date和drive是占位符,表示采集数据的日期和视频编号。时间戳记录在Timestamps.txt文件。
在这里插入图片描述
从图1和图2中可以看到不同传感器的坐标系定义(上下左右以驾驶员为基准):
相机:x轴——向右,y轴——向下,z轴——向前;
激光雷达:x轴——向前,y轴——向左,z轴——向上;
组合导航:同激光雷达。
注: 此处坐标系的定义对后续数据的可视化和分析,以及标定矩阵的理解和使用至关重要,需要熟悉。

其中"date"和"drive"是占位符,根据当时采集的日期命名,"image_0x"指的是4个相机序列。

所有数据都是同步采集的,按照激光雷达的频率(10hz)采集,所有传感器的数据序列都是以image_00作为同步参考系。
包括:

原始的(没有同步和没有矫正的)和处理过的(同步和矫正过)的双目灰度图像序列(50万像素、png格式)
原始的(没有同步和没有矫正的)和处理过的(同步和矫正过)的双目彩色图像序列(50万像素、png格式)
3D激光雷达数据(每帧10万+个点,.bin格式float矩阵存储)
标定文件(相机与相机之间、相机与GPS/IMU之间、相机与激光雷达之间,txt文本文件存储)
3D目标跟踪标签(汽车、卡车、火车、行人、自行车人,xml文件)

此处的非同步和非矫正的含义是指原始的带有畸变并且没有同步的数据,而同步和矫正的图片是指经过消除畸变并经过双目矫正的图片,并且图片序列也是和其他传感器对应好的。所有的数据都提供时间戳。一般我们用的是同步和矫正的数据(synced+rectified data)。

例如我们下载2011_09_26_drive_0005这个数据序列,最终解压得到的文件夹如下图所示:
------------------- 注意:这里可能是以前的结构,现在的结构是分开下载的,五次的Calibration是分开下载对应的,而且tracklet_labels也是分开的下载的。!!!现在要获取这样的结构,需要我们分布下载。
在这里插入图片描述
image_00:左侧灰度相机图像序列
image_01:右侧灰度相机图像序列
image_02:左侧彩色相机图像序列
image_03:右侧彩色相机图像序列
oxts:GPS/IMU位姿相关信息序列
velodyne_points:激光雷达数据序列

数据处理部分:

C:\Users\Administrator\Desktop\workspace\myProject3\data\devkit、readme.txt 解读
This file gives more information about the KITTI raw data recordings.

General information about streams and timestamps

0-灰机1,1-灰机2,2-彩机1,3-彩机2(同时由GPS出发同步采集,0,1相机系统标定一样,2,3相机系统标定一样,时间戳依据标定值)
Each sensor stream is stored in a single folder. The main folder contains
meta information and a timestamp file, listing the timestamp of each frame
of the sequence to nanosecond precision. Numbers in the data stream correspond
to each numbers in each other data stream and to line numbers in the
timestamp file (0-based index), as all data has been synchronized. All
cameras have been triggered directly by the Velodyne laser scanner, while
from the GPS/IMU system (recording at 100 Hz), we have taken the data
information closest to the respective reference frame. For all sequences
‘image_00’ has been used as the synchronization reference stream.

//校正后的颜色和灰度立体序列
Rectified color + grayscale stereo sequences

Our vehicle has been equipped with four cameras: 1 color camera stereo pair
and 1 grayscale camera stereo pair. The color and grayscale cameras are
mounted close to each other (~6 cm), the baseline of both stereo rigs is
approximately 54 cm. We have chosen this setup such that for the left and
right camera we can provide both color and grayscale information. While the
color cameras (obviously) come with color information, the grayscale camera
images have higher contrast and a little bit less noise.
//时间戳间隔都是10Hz,100ms左右
All cameras are synchronized at about 10 Hz with respect to the Velodyne
laser scanner. The trigger is mounted such that camera images coincide
roughly with the Velodyne lasers facing forward (in driving direction).
//初次采集得到的都是无损压缩的图片,经过校正后微微尺寸减小
All camera images are provided as lossless compressed and rectified png
sequences. The native image resolution is 1382x512 pixels and a little bit
less after rectification, for details see the calibration section below.
The opening angle of the cameras (left-right) is approximately 90 degrees.

The camera images are stored in the following directories:

  • ‘image_00’: left rectified grayscale image sequence
  • ‘image_01’: right rectified grayscale image sequence
  • ‘image_02’: left rectified color image sequence
  • ‘image_03’: right rectified color image sequence

//三维激光扫描数据点云用N*4的矩阵保存在velodyne_points/data 文件夹的.bin 文件
Velodyne 3D laser scan data

The velodyne point clouds are stored in the folder ‘velodyne_points’. To
save space, all scans have been stored as Nx4 float matrix into a binary
file using the following code:

**//matlab加载方法
  stream = fopen (dst_file.c_str(),"wb");
  fwrite(data,sizeof(float),4*num,stream);
  fclose(stream);
//data/.bin de 的数据结构是每一列代表x,y,z,reflectance,,xyz米为单位**

Here, data contains 4*num values, where the first 3 values correspond to
x,y and z, and the last value is the reflectance information. All scans
are stored row-aligned, meaning that the first 4 values correspond to the
first measurement. Since each scan might potentially have a different
number of points, this must be determined from the file size when reading
the file, where 1e6 is a good enough upper bound on the number of values:

//存储写入的方法
  // allocate 4 MB buffer (only ~130*4*4 KB are needed)
  int32_t num = 1000000;
  float *data = (float*)malloc(num*sizeof(float));
 // pointers
  float *px = data+0;
  float *py = data+1;
  float *pz = data+2;
  float *pr = data+3;
//读取的方法
  // load point cloud
  FILE *stream;
  stream = fopen (currFilenameBinary.c_str(),"rb");
  num = fread(data,sizeof(float),num,stream)/4;
  for (int32_t i=0; i<num; i++) {
    point_cloud.points.push_back(tPoint(*px,*py,*pz,*pr));
    px+=4; py+=4; pz+=4; pr+=4;
  }
  fclose(stream);

x,y and y are stored in metric (m) Velodyne coordinates.

IMPORTANT NOTE: Note that the velodyne scanner takes depth measurements
continuously while rotating around its vertical axis (in contrast to the cameras,
which are triggered at a certain point in time). This means that when computing
point clouds you have to ‘untwist’ the points linearly with respect to the velo-
dyne scanner location at the beginning and the end of the 360° sweep. The time-
stamps for the beginning and the end of the sweeps can be found in the time-
stamps file. The velodyne rotates in counter-clockwise direction.

Of course this ‘untwisting’ only works for non-dynamic environments.

The relationship between the camera triggers and the velodyne is the following:
We trigger the cameras when the velodyne is looking exactly forward (into the
direction of the cameras).

GPS/IMU 3D localization unit

The GPS/IMU information is given in a single small text file which is
written for each synchronized frame. Each text file contains 30 values
which are:

  • lat: latitude of the oxts-unit (deg)维度

  • lon: longitude of the oxts-unit (deg)经度

  • alt: altitude of the oxts-unit (m)高度

  • roll: roll angle (rad), 0 = level, positive = left side up (-pi…pi)

  • pitch: pitch angle (rad), 0 = level, positive = front down (-pi/2…pi/2)

  • yaw: heading (rad), 0 = east, positive = counter clockwise (-pi…pi)

  • vn: velocity towards north (m/s)

  • ve: velocity towards east (m/s)

  • vf: forward velocity, i.e. parallel to earth-surface (m/s)

  • vl: leftward velocity, i.e. parallel to earth-surface (m/s)

  • vu: upward velocity, i.e. perpendicular to earth-surface (m/s)

  • ax: acceleration in x, i.e. in direction of vehicle front (m/s^2)

  • ay: acceleration in y, i.e. in direction of vehicle left (m/s^2)

  • az: acceleration in z, i.e. in direction of vehicle top (m/s^2)

  • af: forward acceleration (m/s^2)

  • al: leftward acceleration (m/s^2)

  • au: upward acceleration (m/s^2)

  • wx: angular rate around x (rad/s)

  • wy: angular rate around y (rad/s)

  • wz: angular rate around z (rad/s)

  • wf: angular rate around forward axis (rad/s)

  • wl: angular rate around leftward axis (rad/s)

  • wu: angular rate around upward axis (rad/s)

  • posacc: velocity accuracy (north/east in m)

  • velacc: velocity accuracy (north/east in m/s)

  • navstat: navigation status

  • numsats: number of satellites tracked by primary GPS receiver

  • posmode: position mode of primary GPS receiver

  • velmode: velocity mode of primary GPS receiver

  • orimode: orientation mode of primary GPS receiver

To read the text file and interpret them properly an example is given in
the matlab folder:

  1. oxts = loadOxtsliteData(‘2011_xx_xx_drive_xxxx’)::to read in the GPS/IMU data.
  2. pose = convertOxtsToPose(oxts):: to transform the oxts data into local euclidean poses, specified by 4x4 rigid
    transformation matrices. For more details see the comments in those files.

Coordinate Systems

//三种传感器的坐标器标定(都满足右手定则)
The coordinate systems are defined the following way, where directions
are informally given from the drivers view, when looking forward onto
the road:

  • Camera: x: right, y: down, z: forward
  • Velodyne: x: forward, y: left, z: up
  • GPS/IMU: x: forward, y: left, z: up

All coordinate systems are right-handed.
//三种传感器的系统校正标定
Sensor Calibration

按行读取标定参数
The sensor calibration zip archive contains files, storing matrices in
row-aligned order, meaning that the first values correspond to the first
row:
1)calib_cam_to_cam.txt
2)calib_imu_to_velo.txt
3)calib_velo_to_cam.txt

calib_cam_to_cam.txt: Camera-to-camera calibration

//XX表示相机的编号

  • S_xx: 1x2 size of image xx before rectification (矫正前的图片尺寸)
  • K_xx: 3x3 calibration matrix of camera xx before rectification(标定矩阵)
  • D_xx: 1x5 distortion vector of camera xx before rectification(方向矢量)
  • R_xx: 3x3 rotation matrix of camera xx (extrinsic)(旋转矩阵标定)外在需要
  • T_xx: 3x1 translation vector of camera xx (extrinsic)(转换矩阵)外在需要
  • S_rect_xx: 1x2 size of image xx after rectification校正后的图片尺寸
  • R_rect_xx: 3x3 rectifying rotation to make image planes co-planar(校正旋转共面)
  • P_rect_xx: 3x4 projection matrix after rectification(投影映射矩阵)
    //实际应用中P_rect_xx居多对于校正后的图片进行投影操作
    Note: When using this dataset you will most likely need to access only
    P_rect_xx, as this matrix is valid for the rectified image sequences.

calib_velo_to_cam.txt: Velodyne-to-camera registration

  • R: 3x3 rotation matrix
  • T: 3x1 translation vector
  • delta_f: deprecated
  • delta_c: deprecated

R|T takes a point in Velodyne coordinates and transforms it into the
coordinate system of the left video camera. Likewise it serves as a
representation of the Velodyne coordinate frame in camera coordinates.

calib_imu_to_velo.txt: GPS/IMU-to-Velodyne registration

  • R: 3x3 rotation matrix
  • T: 3x1 translation vector

R|T takes a point in GPS/IMU coordinates and transforms it into the
coordinate system of the Velodyne scanner. Likewise it serves as a
representation of the GPS/IMU coordinate frame in Velodyne coordinates.

example transformations

//laser 坐标转换到相机坐标,作为深度信息的一部分的方法
As the transformations sometimes confuse people, here we give a short
example how points in the velodyne coordinate system can be transformed
into the camera left coordinate system.

In order to transform a homogeneous point X = [x y z 1]' from the velodyne
coordinate system to a homogeneous point Y = [u v 1]' on image plane of
camera xx, the following transformation has to be applied:

Y = P_rect_xx * R_rect_00 * (R|T)_velo_to_cam * X

To transform a point X from GPS/IMU coordinates to the image plane:

Y = P_rect_xx * R_rect_00 * (R|T)_velo_to_cam * (R|T)_imu_to_velo * X

The matrices are:

- P_rect_xx (3x4):         rectfied cam 0 coordinates -> image plane
- R_rect_00 (4x4):         cam 0 coordinates -> rectified cam 0 coord.
- (R|T)_velo_to_cam (4x4): velodyne coordinates -> cam 0 coordinates
- (R|T)_imu_to_velo (4x4): imu coordinates -> velodyne coordinates

Note that the (4x4) matrices above are padded with zeros and:
R_rect_00(4,4) = (R|T)_velo_to_cam(4,4) = (R|T)_imu_to_velo(4,4) = 1.

Tracklet Labels

Tracklet labels are stored in XML and can be read / written using the
C++/MATLAB source code provided with this development kit. For compiling
the code you will need to have a recent version of the boost libraries
installed.

Each tracklet is stored as a 3D bounding box of given height, width and
length, spanning multiple frames. For each frame we have labeled 3D location
and rotation in bird’s eye view. Additionally, occlusion / truncation
information is provided in the form of averaged Mechanical Turk label
outputs. All tracklets are represented in Velodyne coordinates.

Object categories are classified as following:

  • ‘Car’
  • ‘Van’
  • ‘Truck’
  • ‘Pedestrian’
  • ‘Person (sitting)’
  • ‘Cyclist’
  • ‘Tram’
  • ‘Misc’

Here, ‘Misc’ denotes all other categories, e.g., ‘Trailers’ or ‘Segways’.
//编译’cpp/tracklets.h’的方法和如何把不同坐标系的数据转换到相机的image平面上
Reading the Tracklet Label XML Files

This toolkit provides the header ‘cpp/tracklets.h’, which can be used to
parse a tracklet XML file into the corresponding data structures. Its usage
is quite simple, you can directly include the header file into your code
as follows:

  #include "tracklets.h"
  Tracklets *tracklets = new Tracklets();
  if (!tracklets->loadFromFile(filename.xml))
    <throw an error>
  <do something with the tracklets>
  delete tracklets;

In order to compile this code you will need to have a recent version of the
boost libraries installed and you need to link against
‘libboost_serialization’.
1.1.6 Tracklet Labels
对于部分raw data中数据,KITTI对其中出现在相机视野范围内的有效目标(主要是Car、Van、Truck、Pedestrian、Person、Cyclist、Tram等)进行了人工标注,可以作为其中目标的检测或跟踪的真值。

每一个tracklet保存了目标三维包围盒的高度、宽度和长度以及目标在激光雷达坐标系下的旋转和平移量等信息,结果保存为一个xml格式文件,可以用提供的cpp/tracklets.h文件进行解析:

‘matlab/readTrackletsMex.cpp’ is a MATLAB wrapper for ‘cpp/tracklets.h’.
It can be build using make.m. Again you need to link against
‘libboost_serialization’, which might be problematic on newer MATLAB
versions due to MATLAB’s internal definitions of libstdc, etc. The latest
version which we know of which works on Linux is 2008b. This is because
MATLAB has changed its pointer representation.

Of course you can also directly parse the XML file using your preferred
XML parser. If you need to create another useful wrapper for the header file
(e.g., for Python) we would be more than happy if you could share it with us).

Demo Utility for projecting Tracklets into Images

In ‘matlab/run_demoTracklets.m’ you find a demonstration script that reads
tracklets and projects them as 2D/3D bounding boxes into the images. You
will need to compile the MATLAB wrapper above in order to read the tracklets.
For further instructions, please have a look at the comments in the
respective MATLAB scripts and functions.
举例:
1.2.1 run_demoVelodyne.m
该文件用来示例如何将激光雷达点云投影到相机图像上实现传感器融合的目的。

首先使用Matlab(我使用的是Matlab 2019a版本)打开run_demoVelodyne.m文件,修改你下载好的KITTI raw data数据集所在路径,例如我修改的结果是:
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述
从图可以看出,程序将一帧完整的在相机视野范围内的点云通过KITTI提供的标定矩阵,投影到相机图像上了,从而为某些图像像素赋予了点云深度信息。投影结果中不同颜色表示点云的不同距离,颜色越红,表示目标离雷达坐标系原点越近。

sfMLearning数据下载:

raw_data_downloader.sh

文件内容,修改镜像文件!!!

#!/bin/bash

files=(2011_09_26_calib.zip
2011_09_26_drive_0001
2011_09_26_drive_0002
2011_09_26_drive_0005
2011_09_26_drive_0009
2011_09_26_drive_0011
2011_09_26_drive_0013
2011_09_26_drive_0014
2011_09_26_drive_0015
2011_09_26_drive_0017
2011_09_26_drive_0018
2011_09_26_drive_0019
2011_09_26_drive_0020
2011_09_26_drive_0022
2011_09_26_drive_0023
2011_09_26_drive_0027
2011_09_26_drive_0028
2011_09_26_drive_0029
2011_09_26_drive_0032
2011_09_26_drive_0035
2011_09_26_drive_0036
2011_09_26_drive_0039
2011_09_26_drive_0046
2011_09_26_drive_0048
2011_09_26_drive_0051
2011_09_26_drive_0052
2011_09_26_drive_0056
2011_09_26_drive_0057
2011_09_26_drive_0059
2011_09_26_drive_0060
2011_09_26_drive_0061
2011_09_26_drive_0064
2011_09_26_drive_0070
2011_09_26_drive_0079
2011_09_26_drive_0084
2011_09_26_drive_0086
2011_09_26_drive_0087
2011_09_26_drive_0091
2011_09_26_drive_0093
2011_09_26_drive_0095
2011_09_26_drive_0096
2011_09_26_drive_0101
2011_09_26_drive_0104
2011_09_26_drive_0106
2011_09_26_drive_0113
2011_09_26_drive_0117
2011_09_26_drive_0119
2011_09_28_calib.zip
2011_09_28_drive_0001
2011_09_28_drive_0002
2011_09_28_drive_0016
2011_09_28_drive_0021
2011_09_28_drive_0034
2011_09_28_drive_0035
2011_09_28_drive_0037
2011_09_28_drive_0038
2011_09_28_drive_0039
2011_09_28_drive_0043
2011_09_28_drive_0045
2011_09_28_drive_0047
2011_09_28_drive_0053
2011_09_28_drive_0054
2011_09_28_drive_0057
2011_09_28_drive_0065
2011_09_28_drive_0066
2011_09_28_drive_0068
2011_09_28_drive_0070
2011_09_28_drive_0071
2011_09_28_drive_0075
2011_09_28_drive_0077
2011_09_28_drive_0078
2011_09_28_drive_0080
2011_09_28_drive_0082
2011_09_28_drive_0086
2011_09_28_drive_0087
2011_09_28_drive_0089
2011_09_28_drive_0090
2011_09_28_drive_0094
2011_09_28_drive_0095
2011_09_28_drive_0096
2011_09_28_drive_0098
2011_09_28_drive_0100
2011_09_28_drive_0102
2011_09_28_drive_0103
2011_09_28_drive_0104
2011_09_28_drive_0106
2011_09_28_drive_0108
2011_09_28_drive_0110
2011_09_28_drive_0113
2011_09_28_drive_0117
2011_09_28_drive_0119
2011_09_28_drive_0121
2011_09_28_drive_0122
2011_09_28_drive_0125
2011_09_28_drive_0126
2011_09_28_drive_0128
2011_09_28_drive_0132
2011_09_28_drive_0134
2011_09_28_drive_0135
2011_09_28_drive_0136
2011_09_28_drive_0138
2011_09_28_drive_0141
2011_09_28_drive_0143
2011_09_28_drive_0145
2011_09_28_drive_0146
2011_09_28_drive_0149
2011_09_28_drive_0153
2011_09_28_drive_0154
2011_09_28_drive_0155
2011_09_28_drive_0156
2011_09_28_drive_0160
2011_09_28_drive_0161
2011_09_28_drive_0162
2011_09_28_drive_0165
2011_09_28_drive_0166
2011_09_28_drive_0167
2011_09_28_drive_0168
2011_09_28_drive_0171
2011_09_28_drive_0174
2011_09_28_drive_0177
2011_09_28_drive_0179
2011_09_28_drive_0183
2011_09_28_drive_0184
2011_09_28_drive_0185
2011_09_28_drive_0186
2011_09_28_drive_0187
2011_09_28_drive_0191
2011_09_28_drive_0192
2011_09_28_drive_0195
2011_09_28_drive_0198
2011_09_28_drive_0199
2011_09_28_drive_0201
2011_09_28_drive_0204
2011_09_28_drive_0205
2011_09_28_drive_0208
2011_09_28_drive_0209
2011_09_28_drive_0214
2011_09_28_drive_0216
2011_09_28_drive_0220
2011_09_28_drive_0222
2011_09_28_drive_0225
2011_09_29_calib.zip
2011_09_29_drive_0004
2011_09_29_drive_0026
2011_09_29_drive_0071
2011_09_29_drive_0108
2011_09_30_calib.zip
2011_09_30_drive_0016
2011_09_30_drive_0018
2011_09_30_drive_0020
2011_09_30_drive_0027
2011_09_30_drive_0028
2011_09_30_drive_0033
2011_09_30_drive_0034
2011_09_30_drive_0072
2011_10_03_calib.zip
2011_10_03_drive_0027
2011_10_03_drive_0034
2011_10_03_drive_0042
2011_10_03_drive_0047
2011_10_03_drive_0058)

for i in ${files[@]}; do
        if [ ${i:(-3)} != "zip" ]
        then
                shortname=$i'_sync.zip'
                fullname=$i'/'$i'_sync.zip'
        else
                shortname=$i
                fullname=$i
        fi
	echo "Downloading: "$shortname
        wget 'http://datadownload.f3322.net:666/share/kitti/raw_data/'$fullname
        unzip -o $shortname
        rm $shortname
done
命令行1:

python data/prepare_train_data.py --dataset_dir=/path/to/raw/kitti/dataset/ --dataset_name='kitti_raw_eigen' --dump_root=/path/to/resulting/formatted/data/ --seq_length=3 --img_width=416 --img_height=128 --num_threads=4

参考:https://blog.csdn.net/lgczym/article/details/76474350

未完待续:。。。

猜你喜欢

转载自blog.csdn.net/zjguilai/article/details/90168564