"Computer Vision 3D Reconstruction" Note 5 - Binocular Vision

Notes for the course "3D Reconstruction of Computer Vision (Sfm and SLAM Core Algorithms)" taught by Beiyou teacher Lu Peng

5. Binocular vision

5.1, Parallel view

引理1：$[a_{\times }]B=B^T[(B^{-1}a{)}_{\times }]$
Lemma 2:$e^{\prime }=K^{\prime }T$

Proof:
$e^{\text{'}}$ is$O_1$In camera $O_2$According to the
camera model: $e^{\prime }=K^{\prime }[R,T]\left[\begin{array}{c}0\\ 0\\ 0\\ 1\end{array}\right]=K^{\prime }T$

Theorem : The fundamental matrix can be written as: $F=K^{\prime -T}[T_{\times }]R{K}^{-1}=[e_{\times }^{\prime }]K^{\prime }R{K}^{-1}$

Proof:
According to Lemma 1,2, $\left[e_{\times }^{\prime }\right]K^{\prime }=K^{\prime T}[(K^{\prime -1}{e}^{\prime }{)}_{\times }]=K^{\prime T}[\left(K^{\prime -1}{K}^{\prime }T{)}_{\times }\right]=K^{\prime T}[T_{\times }]$
$\therefore [e_{\times }^{\prime }]K^{\prime }R{K}^{-1}=K^{\prime T}[T_{\times }]R{K}^{-1}=F$

Corollary 1 : Fundamental matrix for parallel views: $F=\left[e_{\times }^{\prime }\right]=\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& -1\\ 0& 1& 0\end{array}\right]$

Proof:
In parallel view: $K=K^{\prime },R=I,T=(T,0,0{)}^{Bring\; T}$
into the camera model to get:$e^{\prime }=(1,0,0{)}^T$ , that is, the pole is at infinity and the epipolar line is parallel to the X-axis.
Bring it into the theorem:$F=[e_{\times }^{\prime }]KI{K}^{-1}=[e_{\times }^{\prime }]$

Deduction 2 : In the parallel view, the image point $p=(p_u,p_v,1{)}^T，p^{\prime }=(p_u^{\prime },p_v^{\prime },1{)}^T$ , then$p_v=p_v^{\prime }$

Proof:
Bring in $p^{T}F{p}^{\prime }=0$ , we can get$p_v=p_v^{\prime }$

Triangulation problem:

Known $p,p^{\prime },K,K^{\prime },R,T$ , findPPThe three-dimensional coordinates of$P.$

Triangulation of parallel views : $$p_u-p_u^{\prime }=\frac{fB}{z}$$

A depth map can be obtained. Application: 3D Movies
Question :

1. How to ensure parallel views
2. How to establish point correspondence

5.2, image correction

For problem 1, image correction can be used to transform the two images into parallel views
. Image correction in 5 steps:

1. Enclosure $p_i\leftrightarrow p_i^{\prime }$, more than 8 pairs
2. Find the fundamental matrix $F$ , seeking points$e_i,e_i^{\prime }$

Find $F$ , get epipolar line$l_i=F^T{p}_i^{\prime }$, solve the system of equations { li T e = 0 l_i^Te=0 $l_i^{T}e=0$ } get pole$e$ , similarly, e ′ e'can be obtained$e^{\prime }$

3. Transformation $H^{\prime }=T^{-1}GRT$, put$e^{\prime }$ maps to infinity$(f,0,0)$

$T=\left[\begin{array}{ccc}1& 0& -width/2\\ 0& 1& -heigth/2\\ 0& 0& 1\end{array}\right],width,height$ are the width and height of the$image$
afterTTe ′ e'after$T\; change$$e^{\prime }$ coordinates are marked as$(e_1^{\prime },e_2^{\prime },1)$
$R=\left[\begin{array}{ccc}a\frac{e_1^{\prime }}{\sqrt{e_1^{\prime 2}+e_2^{\prime 2}}}& a\frac{e_2^{\prime }}{\sqrt{e_1^{\prime 2}+e_2^{\prime 2}}}& 0\\ -a\frac{e_2^{\prime }}{\sqrt{e_1^{\prime 2}+e_2^{\prime 2}}}& a\frac{e_1^{\prime }}{\sqrt{e_1^{\prime 2}+e_2^{\prime 2}}}& 0\\ 0& 0& 1\end{array}\right]$
where $a=sign(e_1^{\prime })$ , after$R$ changed$e^{\prime }$ The coordinates are:$(f,0,1)$
$G=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ -\frac{1}{f}& 0& 1\end{array}\right]$
through $G$ changed$e^{\prime }$ The coordinates are:$(f,0,0)$

4. Solve for $H$，$\min {\sum }_{i}d(H{p}_i,H^{\prime }{p}_i^{\prime })$
5. with $H,H^{\text{'}}$ Transform the image to a parallel view
   

3. Corresponding point search

After image rectification, parallel views are obtained, and matching points are found on the scanning line;
correlation matching/normalized correlation matching, the effect is average, and existing problems: occlusion, perspective shortening, baseline selection, homogeneous area, repeated pattern Constraints
:

• uniqueness
• monotonicity
• smoothness