Deep Instance Segmentation Based on Automotive Radar Detection Points
One difference:
automotive radar car radar:
Low resolution, sparse point clouds, semantically ambiguous, not suitable for direct use of methods for dense LiDAR point development;
Object images returned are not as accurate as LIDAR and can be returned in rain, snow, sleet, hail, fog, mud and dust;
It is also more accurate than lidar at night and in cloudy conditions;
emits radio waves instead of laser beams
LiDAR lidar:
Laser waves are shorter than radio waves, so lidar can produce more detailed images of objects; expensive
Left LiDAR ; Right RADAR
============================================
Ask a question:
Starting from the practical problem of radio radar on point cloud instance segmentation:
Problem 1: Due to the lower resolution of the radar measurement point, it is more sparse and noisy than the lidar measurement point cloud, and lacks semantic and geometric information. Some geometric information, such as the shape of the object, cannot be reflected by the distribution of points, resulting in local Lack of information.
Problem 2: Car driving requires algorithms for real-time processing of radar data, and the Micro Central Unit (MCU) in the radar processing system limits the storage space and available computing power for the model.
analyse problem:
For problem 1: global feature information can help segment tasks, how to design a deep learning algorithm to solve the sparsity and ambiguity of radar detection points, because the mainstream pointnet++ imitates convolutional thinking and is designed according to the thinking of extracting local features Yes, it takes a deep network to learn the global interaction
Aiming at problem 2: while improving the performance, the actual scene should be taken into account, and the weight should be as light as possible
solution:
1. Aiming at the instance segmentation task of sparse detection points of automotive radar, a clustering method based on semantic segmentation is designed. The model is designed based on the semantic segmentation version of PointNet++, with the newly introduced head-estimated point-to-center offset vector (CSV) -- which represents the offset from each detected point to the geometric center of its corresponding instance. During the clustering process, use the predicted CSV to move each point to the center of its instance, so that the points belonging to the same instance are closer and the clustering accuracy is improved
2. Use the sum of cosine similarity loss and normalized inner product loss as the loss during semantic segmentation training of sparse radar detection points to improve the performance of CSV-guided clustering----for the center offset proposed above Adaptation loss to minimize the distance between the predicted CSV and the ground-truth CSV
3. Proposed to use gMLP to further improve the performance of the model
The effect obtained:
The average coverage (mCov) and average precision (mAP) are 9.0% and 9.2% higher than cluster-based classification methods, and 8.6% and 9.1% higher than end-to-end instance segmentation methods, respectively. At the same time, the memory consumption of this method and its lightweight version is kept at about 1MB, and the inference time is less than 40 ms, which is suitable for automotive radar microcontrollers
specific methods:
N is the sample size; C is the number of output channels of the backbone network; Nclass is the number of classes; Ndim is the size of the original radar detection point. The input points first obtain their predicted class labels through the point-wise classification branch. Then move its predicted csv according to the csv prediction branch, so that the points belonging to the same instance are more concentrated. Then, points with the same class label are grouped into clusters (i.e., instances). In the instance segmentation results of the sample frame, different colors represent different classes, and points in the same circle belong to the same instance.
(a) PointNet++ backbone network, using two SA levels and two FP levels to extract point features
(b) The structure of the prediction head, which is a two-layer MLP, the semantic segmentation branch predicts the score of each class for each point, and for the csv prediction branch, it predicts the difference between each point and its instance center.
(c) Diagram of csv, points in the same circle belong to the same instance
(d) MLP-enhanced PointNet++ backbone for visualization of the overall structure. The visualized MLP block adds gMLP after each collection abstraction level and feature propagation level
Learning reference:
The gmlp structure depends on the original paper: Pay Attention to MLPs
experimental narrative