Classic literature reading--LBEVPlace (LiDAR position recognition in bird's-eye view)

0. Introduction

We have talked a lot about how to complete relocation through frame and map matching in previous blogs, so as to complete location recognition. At the same time, there is also a blog " Positioning Analysis and Thinking " dedicated to introducing relocation . Current location recognition methods based on LiDAR are usually based on the representation of point clouds such as unordered points or range images. These methods achieve high retrieval recall, but their performance may degrade under view changes or scene changes. In BEVPlace : Learning LiDAR-based Place Recognition using Bird's Eye View Images , we explored the potential of a different representation for place recognition, namely Bird's Eye View (BEV) images. We observe that the structural content of BEV images is less affected by the rotation and translation of the point cloud. The code for this paper is also open source on Github

Figure 1. (a) Two range images from the KITTI dataset. The images were projected from two point clouds about 5 meters apart. Small translations of point clouds will introduce structural distortions such as scale changes and occlusions from objects in the scene. (b) Corresponding BEV image. The scale and location distribution of objects on the road remain almost unchanged. ©Performance on various datasets. The recall of the simple NetVALD network based on BEV representation at Top1 is comparable to the state-of-the-art methods. Our BEVPlace further improves the performance of the baseline.

1. Main contributions

The contributions of this article are as follows:

1. A new LiDAR-based location recognition method BEVPlace is proposed. In this method, we extract rotation-equivalent local features from Bev images based on grouped convolution, which facilitates the design of rotation-invariant global features.

2. The statistical correlation between characteristic and geometric distances of point cloud pairs is explored. Based on this correlation prior, we calculate the geometric distance between the query point cloud and the matching point cloud and use it for position estimation.

3. We evaluate our method on three large-scale public datasets and show that our method is robust to view changes, has strong generalization ability, and achieves the best performance in terms of recall .

2. Preliminary preparation (explaining concepts)

设$m_i$For the sensor in the pose $T_i=（R_i，t_i)$ , where$R_i$is the rotation matrix, $t_i$It's the location. by $The\; database\; formed\; by\; n$ point clouds and their related poses can be expressed as$M=\{(m_i，T_i){\}}_{i=1,2，\dots ，n}$. Given a query point cloud $m_q$, venue identification aims at MM from a pre-built databaseFind the point cloud with the most structural similarity in$M.$In LiDAR-based place recognition problems, it is generally considered that if two point clouds are geometrically close, they are structurally similar. To achieve this goal, we design a network$f(\cdot )$ maps the point cloud to a unique compact global feature vector such that if$m_q$Structurally similar to $m_i$But with $m_j$不相似，则$||f(m_q)-f(m_i)|{|}^2<||f(m_q)-f(m_j)|{|}^2$ . Based on network$f$ , we perform place retrieval by finding the point cloud with the smallest feature distance from the query point cloud.

In this work, we train our network based on BEV images of point clouds. In addition to place retrieval, we also develop an extended use for estimating the location of a query point cloud.

3. Summary of methods

The method in this article consists of two modules, as shown in Figure 2. In the BEVPlace network, we project the query point cloud into the BEV image. We then extract rotation-invariant global features through a set of convolutional networks and NetVLAD [2]. In the position estimator, we retrieve the closest features of global features from a pre-built database. We recover the geometric distance between query and matching point clouds based on a mapping model. Based on the recovered distance, the location of the query is estimated.

Figure 2: Two modules of our approach. In the BEVPlace network, we project point clouds into BEV images and extract rotation-invariant global features. In the position estimator module, we recover the geometric distance from the feature space and estimate the position of the query point cloud.

4. BEVPlace network (important part)

In road scenarios, LiDAR sensors on cars or robots can only move on the ground plane. Since we generate BEV images by projecting point clouds onto the ground plane, changes in the sensor's perspective will cause rotational transformations on the image. To achieve robust place recognition, we aim to design a network f that extracts rotation-invariant features from BEV images. Will BEV Image IIRotation transformation R ∈ SO (2) R∈SO(2)on$I$$R\in SO\left(2\right)$ is expressed as$R◦I$，$The\; rotation\; invariance\; of\; f$ can be expressed as:

A straightforward way to achieve this invariance is to use data augmentation [17] to train the network. However, data augmentation typically requires the network to have a larger set of parameters to learn rotations, and may fail to generalize to combinations of rotations and scenes not present in the training set. In this work, we use a cascade of group convolutional networks and NetVLAD to achieve rotation invariance. Our BEVPlace has strong generalization ability because the network is inherently rotation invariant.

4.1 Bird’s-eye image generation

We follow the approach of BVMatch [20] to construct images using point densities. We divide the ground space into grids with a grid size of 0.4 m. For point cloud $m$ , we count the number of points in each grid and use the normalized point density as the BEV imageIIThe pixel intensity of$I.$

4.2 Group convolutional network (rotation transformation)

Group convolutions treat feature maps as functions of corresponding symmetric groups [30]. Consider the 2D rotation group $SO\left(2\right)$ , in BEV imageIIApply group convolutionfgc f_{gc}$on\; I$$f_{gc}$will produce rotational equivariant characteristics, which can be written as

That is, the input $I$ transform RRby rotation$R$ transforms and then maps$f_{gc}$Passing it should be done with $f_{gc}$Projection $I$ , then use$R^{\prime }\in The\; results\; of\; SO\left(2\right)$ transformed features are the same. Usually, the design$f$ to make$R^{\prime }=R$。

Group convolution has been developed for several years, and there are some mature group convolution designs [30, 18, 32]. We implemented our network based on GIFT [18]. GIFT was originally designed for image matching and can produce unique local features. Our main modification to GIFT is to remove the scale feature since there is no scale difference between BEV images. More details of our network implementation are attached in the supplementary material.

4.3 Loss function

For the LiDAR-based place recognition problem, there are some loss functions [1, 33]. In this work, we use the simple and commonly used lazy triplet loss [1] to train our network, whose formula is:

…For details, please refer to Gu Yueju