3D Reconstruction of Computer Vision—Chapter 6: Multi-view Geometry "Sfm and SLAM Core Algorithm in Simple Ways (Lu Peng)"

Chapter 6: Multiple View Geometry

1. Sports recovery structural problems

Three typical structural tasks for motor recovery include:

• European structure reply (the internal parameters of the camera are known, but the external parameters are unknown)
• Affine structure restoration (the camera is an affine camera, and the internal and external parameters are unknown)
• Perspective structure recovery (camera is a perspective camera, internal and external parameters are unknown)

2. Restoration of European structure

2.1 Definition of European structure recovery problem

2.2 Two-view European structure recovery

• Taking the left view as the reference coordinate system, then the left view $[RT]=[I0]$ , and the internal parameters of the camera are known, then the left view projection matrix$M_1$A known;
• When the projection matrix of the right view is also known, the coordinates of the three-dimensional points can be obtained by using triangulation (linear solution or nonlinear solution);
• However, we only know the camera internal parameters of the right view, and do not know the external parameters of the right view relative to the left view $\left[RT\right]$ , so it cannot be directly solved by triangulation, and$[RT]$。

The specific solution steps are as follows:

(1) Solve the fundamental matrix F

(2) Solve the essential matrix

(3) Decompose the essential matrix - solve $[RT]$

(4) Triangulation

2.3 Continental structure restores ambiguity

Ambiguity: a similar change (rotation, translation, scaling) between the restored structure and the real scene;

Metric reconstruction: This kind of reconstruction that only has similar transformations between the restored scene and the real scene is called metric reconstruction;

3. Affine Structure Restoration

3.1 Affine Structure Restoration Problem Definition

3.2 Affine Structure Restoration Based on Factorization

(1) Data centralization

(2) Factorization

3.3 Affine structure recovery ambiguity

$$D=MS=\left(MH\right)\left(H^{-1}S\right)=M^{\ast }{S}^{\ast }$$

The affine structure recovery decomposition is not unique, there is ambiguity, and the ambiguity can have any $3\times 3$ Reversible matrix H expression;

4. Perspective Structure Restoration

4.1 Algebraic method (two views)

(1) Solve the fundamental matrix F

(2) Estimated camera matrix $M_1$、$M_2$

(3) Triangulation

4.2 Algebraic methods (multiple views)

Calculate the motion and structure for each image pair separately, and then perform incremental construction:

4.3 Bundle adjustment

• In the factorization method used for affine transformation structure restoration, it is necessary to assume that all 3D points are visible to all cameras, so when there is occlusion, or the establishment of corresponding point relationship fails, the points that can be used to construct the observation matrix D Very few, the reconstructed points are very few;
• The restoration of perspective structure is solved by algebraic method, which is prone to error accumulation;

Bundle adjustment restores motion and structure by minimizing reprojection errors, and is a nonlinear solution;