Colmap learning three: backend Initialization part (basic matrix F, essential matrix E and homography matrix H)

The basic knowledge of calculating the matrix,
analyzing the essence, foundation, and degrees of freedom of the homography matrix

1 Find the initial image pair

1.1 Manually select ID

1.2 Automatic screening

①If prior focal length exists, start screening

• Matching points are sorted from small to large
• The number of matches for the second candidate image is greater than init_min_num_inliers (100)

②Two-view Triangulation

• R and t meet the startup threshold

2 Fundamental Matrix F

2.1 Degrees of freedom of F matrix = 7

• From the perspective of cv, 3*3 matrix with 9 parameters
  • Because up2scale degrees of freedom -1
  • And because rank(F)=2 degrees of freedom-1
• From a photogrammetry perspective:
  • 5pose
    • determine the ray pose)
  • +2 intrinsic
    • fx and fy, usually s=0, the principal point coordinates (cx, cy) do not participate in the optimization

2.2 Solution method (both require normalization):

• ① The eight-point method has a unique solution. Considering the noise, it is actually transformed into: constructing an overdetermined equation and solving it by least squares .
  • Note: Due to the error, we need to force the constraint rank=2
    • Specific method: Set the last item (minimum item) of the eigenvalue decomposed by SVD to 0 and then reconstruct, the code is as follows:

// Enforcing the internal constraint that two singular values must non-zero
  // and one must be zero.
  Eigen::JacobiSVD<Eigen::Matrix3d> fmatrix_svd(
      ematrix_t.transpose(), Eigen::ComputeFullU | Eigen::ComputeFullV);
  Eigen::Vector3d singular_values = fmatrix_svd.singularValues();
  singular_values(2) = 0.0;
  const Eigen::Matrix3d F = fmatrix_svd.matrixU() *
                            singular_values.asDiagonal() *
                            fmatrix_svd.matrixV().transpose();

• ②Seven-point method: use 7 points to solve 9 unknown parameters, and the number of unknowns > the number of equations. New equations can be constructed to solve:
  • There are 9-7=2 nullspaces, rewrite the equation as:
    • F=λF1+μF2，λ+μ=1

3 Essence matrix E

3.1 The degree of freedom of the essential matrix is ​​5

• cv angle: 6-1=5
  • Camera pose 6pose (3 positions, 3 angles)
  • scale up2scale -1
• Photogrammetry Angle: Direct 5pose Orientation Elements

3.2 Understand the essential matrix and fundamental matrix :

• The fundamental matrix connotates the internal parameters of the camera, and the intrinsic matrix has nothing to do with the internal parameters

3.3 The solution of the essential matrix:

• The 8-point method is the same as the 8-point method for finding F, and constructs an overdetermined equation to solve the least squares
  • Pay attention to the mandatory constraints, usually let λ1=(λ1+λ2)/2, λ2=λ1, λ3=0
• The 5-point method , the nonlinear solution process is complicated, generally calculates the fundamental matrix F and combines the internal parameter K to solve the essential matrix E (such as orbslam2)
• Generally speaking, the 5-point method is worse than the 8-point method. The 8-point method is suitable for the forward-facing of the vehicle, but it is not as good as the 5-point method in sideways-facing.

4 Homography matrix H: DLT solution

5 Recover R and t from matrix

5.1 Essential matrix Essential restores Rt

• SVD solves 4 solutions
• Triangulation to find the set of attitude parameters with the most 3D points ahead

5.2 Homography recovery Rt

Reference: syllabus reference online course