3D point cloud recognition

Lidar point cloud

• Generated by lidar, it is divided into mechanical Lidar:TOF, N independent laser units, which rotate to generate a 360-degree field of view.

In the image above, the lidar is located in the center of the circle, and the light spots around it are the resulting 3D point clouds. Its center is typically scanned 360 degrees by a car carrying a lidar

In the above two pictures, one is normal angle and one is top view. The device with the blue axis is a mechanical lidar, which rotates 360 degrees and emits 64 laser lines.

There is also a non-rotating MEMS lidar, which has a component that rotates inside, and the effect of scanning is achieved through the rotation of this component.

• Data characteristics

1. Simple: consists of x, y, z, i, x, y, z are coordinates, i can be intensity or distance.
2. Sparse: Compared with the image, it is much sparser, if the lidar point cloud is projected into the image, it may only occupy 7% of the image .
3. Disorder: N! The point cloud has no order. Different angles, vibrations, and different scanning orders will cause the order of the point clouds to be different.
4. Accurate: +/-2cm

• Image vs point cloud

1. Point Clouds: Simple and Precise Fit for Geometry Perception
2. Image: Rich and Varied for Semantic Perception

These two perception methods have their own uses in ADAS (autonomous driving) scenarios. For example, traffic signals cannot be perceived by 3D point clouds, but can only be perceived through image recognition.

• 3D point cloud data source

1. CAD models: The dataset for general classification/segmentation problems is CAD.
2. LiDAR sensor (Lidar)
3. RGBD cameras, binocular cameras, they are all dense point clouds

• Common datasets

1. Classification (Feature Respresentation), all based on CAD models
   1. ModelNet40/10
   2. ShapeNetPart
2. Driverless scene: Detection/ Segmentation/ Tracking
   1. http://www.cvlibs.net/datasets/kitti/
   2. http://apolloscape.auto
   3. https://www.cityscapes-dataset.com
   4. https://bdd-data.berkeley.edu

Here we mainly use the kitti dataset. Let's take a look at the datasets in kitti first.

This is a binocular dataset

Optical flow dataset

Deep dataset

Odometer dataset

Detection data set 2d, 3d, we are going to use this 3d data set

tracking dataset

Split the dataset

These are all algorithms that process the original data set to deal with various segmentations.

• KITTI dataset

Computer Vision Algorithm Evaluation Dataset in Autonomous Driving Scenarios

1. Algorithm evaluation categories: stereo (stereo), optical flow (optical flow), visual odometry (visual odometer), object dectection (object detection), tracking (tracking)
2. Data collection scenarios: Road (road), City (city), Residential (residential area), Campus (campus), Person (person)
3. 3D object detection: car (car), van (lorry), truck (truck), pedestrian (pedestrian), sitting (sitting), cyclist (cyclist), tram (tram), misc ( Miscellaneous)

Let's still take a look at this collection truck

In the above image, there are two grayscale cameras cam 0 and cam 1, forming a binocular; and two color cameras cam 2 and cam 3, also forming a binocular. It's a GPS navigation, it's a lidar.

• Coordinate System

For the camera, it is the red coordinate system in the above figure, X corresponds to the right side of the car, Y corresponds to the bottom of the car, and Z corresponds to the front of the car . The lidar coordinate system is the blue coordinate system in the above figure, X corresponds to the front of the car, Y corresponds to the left of the car, and Z corresponds to the top of the car . The GPS coordinate system is the green coordinate system in the picture above, which is the same as the lidar .

• Aerial view of lidar point cloud

The above picture takes the point in the upper left corner of the picture as the origin of the coordinates of the picture, to the right is the x coordinate of the picture, and downward is the y coordinate of the picture, then it forms the following relationship with the coordinates of the lidar

• Project a point cloud onto an image

We still take the upper left corner of the picture as the origin of the picture, the x coordinate of the picture to the right, and the y coordinate of the picture downward, then it forms the following relationship with the coordinates of the lidar

This means that the picture we are facing is the front of the lidar, the left side of the picture is the y-axis of the lidar, and the top of the picture is the z-axis of the lidar.

 

{{o.name}}

{{m.name}}