Loopback detection Loopback correction (2): Principle of loopback correction
1. Overview of loopback calibration
Reference: [1] Xu Kuan. Research on binocular vision SLAM fused with IMU information [D]. Harbin Institute of Technology, 2018.
What the loopback calibration needs to do is how to use the loopback information to eliminate the accumulated errors. The loopback correction is mainly divided into two steps:
1. First, perform loop fusion. Loop fusion is to fuse the map points associated with the feature points matched by the loop detection and remove the duplicate map points.
2. Then perform the pose correction of the current frame, and use the basic matrix obtained by the loop detection to calculate the pose conversion relationship between the current frame and the loop frame, and correct the pose of the current frame.
3. Finally, perform posture correction of all frames, and use BA (Bundle Adjustment, beam adjustment method) to correct all postures of the image in the loop.
Second, several pictures
1. Common view (Covisibility Graph)
The common view is an undirected weighted graph. Each vertex of the graph represents the state of the camera at a certain moment, namely the pose. The edge of the graph connects two vertices, indicating the transformation relationship between the camera poses in these two states. When two cameras see similar spatial points, their corresponding poses will be connected (we can calculate the motion between the two cameras based on the projection of these spatial points on the photo). Based on the number of observed spatial points, a weight is added to this edge to measure the credibility of this edge. As shown in the figure is a common view, where the blue vertex represents the camera status point, and the green line connects the two blue vertices, indicating the transformation relationship of the camera pose at these two moments.
2. Essential Graph
The essential graph is a concept proposed by ORB-SLAM, which reduces the number of edges as much as possible, thereby reducing the calculation amount of the global loopback. Because, if the two vertices in the common view can observe more map points together (evaluate the movement based on the common map points), it means that they have a stronger connection relationship. Therefore, using this feature, some common views can be simplified to obtain the essential picture. The specific process is as follows:
Set an appropriate threshold. If the number of map points that two vertices can observe is greater than this threshold, the edge between them will be saved, otherwise it will be rejected. However, if all the vertices of a group and the edges of another group of vertices are all eliminated, then the two groups of vertices will be optimized separately when optimizing, and the pose of the camera group that does not contain the loopback information will not It is optimized, which is undesirable. Therefore, before excluding the edges of the weak connection relationship, first build a spanning tree of the original common view, and then remove some unnecessary edges on this basis, so that the final result is also a spanning tree of the original common view, and the vertices Most of them can observe a sufficient number of map points. The loop correction is carried out on the essence map (ORB-SLAM).
三、BA（Bundle Adjustment）
Reference: Bundle Adjustment—that is, to minimize reprojection errors
BA, Bundle Adjustment, translated as beam adjustment method, its essence is an optimization model, the purpose is to minimize the re-projection error.
beam
The light beam refers to the light beam projected from a point in the three-dimensional space onto the image plane, and the reprojection error is constructed using these light beams, so it is called the beam method, emphasizing that the light beam also describes how the optimization model is established.
Adjustment
When measuring, in order to reduce the measurement error as much as possible, the number of observations is often more than the number that must be observed to determine the unknown quantity, that is, to make redundant observations. With redundant observations, there will inevitably be contradictions between the observation results. The purpose of the measurement adjustment is to eliminate these contradictions to obtain the most reliable results of the observational measurements and assess the accuracy of the measurement results. The principle used for adjustment is "least squares".
Reprojection error
Reprojection refers to the second projection:
The first projection: refers to the projection point of the real three-dimensional space point on the image plane when the camera is taking a picture; the
second projection (re-projection): we use the image to calculate some frames by polar geometry first Camera movement between pictures. After obtaining the camera motion, the spatial position of the feature point is estimated by triangulation. Then use the coordinates of the three-dimensional point we calculated (note that it is not real) and the camera pose we calculated (of course not true) to perform the second projection, that is, re-projection.
The difference between the projection of a real three-dimensional space point on the image plane (that is, the pixel point on the image) and the reprojection (actually a virtual pixel point obtained from our calculation) is the reprojection error.
BA
1. Construct the error function
BA is a graph optimization model, as follows:
2. Non-linear optimization
The process of nonlinear optimization is to use an algorithm to optimize the pose of the camera to minimize the error function.
BA is basically using the LM (Levenberg-Marquardt) algorithm and based on this calculation using the sparse nature of the BA model. The combination of descent method (gradient descent method) and Gauss-Newton.
3. Solve the equation (omitted)
