【3D Reconstruction】Epipolar Geometry

Polar geometry describes the geometric relationship between two viewpoint images of the same scene or object.

It can be found that the projection points of P on the left and right cameras must be on their respective epipolar lines. If the epipolar lines are found, the range of corresponding points can be narrowed down.

The essential matrix algebraically describes the polar geometric relationship between the two viewpoint images captured by the normalized camera.

Normalized camera means that the internal parameter matrix of the camera is an identity matrix

$P'=\begin{bmatrix} x\\y \\ z \end{bmatrix}=\begin{bmatrix} 1 &0 & 0 & 0\\ 0&1 &0 &0 \\ 0& 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} x\\y \\ z \\ 1 \end{bmatrix} \ \ P'=MP$

Suppose $p'^*$ it is the coordinate $p'$ in $O_1$ the coordinate system, then:

$p'=Rp'^*+T \rightarrow p'^*=R^Tp'-R^TT(R^{-1} = R^T)$

Therefore, the coordinates $p'$ in $O_1$ the coordinate system are　 $R^Tp'-R^TT$

In the same way: the coordinates $O_2$ in $O_1$ the coordinate system are　 $-R^TT$

get two vectors　 $O_1p' \ , O_1O_2$

$O_1p' \times O_1O_2 = R^TT \times (R^Tp'-R^TT)=R^TT \times R^Tp'$ (perpendicular to the polar plane)

$[R^TT \times R^Tp']^T \cdot p = 0$

$a \times b$ It can be written in the form of a matrix product, such as:

$a = (x_1,y_1,z_1),b=(x_2,y_2,z_2)$

$a \times b = (y_1z_2-z_1y_2,z_1x_2-x_1z_2,x_1y_2-y_1x_2)\\ = \begin{bmatrix} 0& -z_1 &y_1 \\ z_1&0 &-x_1 \\ -y_1&x_1 &0 \end{bmatrix}\begin{bmatrix} x_2\\y_2 \\ z_2 \end{bmatrix}$

therefore,

$[R^TT \times R^Tp']^T \cdot p = [T \times p']^TRp=[T_xp']^TRp \\ \\ =p'^TT_x^TRp=p'^TT_xRp=p'^T[T \times R]p = 0$

(　 $T_x$ 　is a symmetric matrix of rank 2)

$E = T \times R = T_{\times}R$

E is the essential matrix, which describes the relationship between two viewpoints under the normalized camera.

$p^T \cdot l = 0 ,\ p'^TEp =0\rightarrow l =E^Tp' \\ p'^T \cdot l' = 0 ,\ p'^TEp =0\rightarrow l' =Ep$

e is also a point on the polar line l, so

$l^Te = p'^TEe=0\rightarrow Ee =0$

The fundamental matrix algebraically describes the polar geometric relationship between the images from two viewpoints of a general perspective camera.

The core idea is to transform what is seen under a general camera $p,p'$ into what is seen under a standardized camera $p,p'$

$p=K[I\O]P\rightarrow K^{-1}p=K^{-1}K[I\O]P\rightarrow \begin{bmatrix} 1 &0 &0 &0 \\ 0& 1& 0&0 \\ 0 & 0&1 & 0 \end{bmatrix}P$

$let \\begin{bmatrix}1&0&0&0\\0&1&0&0\\0&0&1&0\end{bmatrix}P = p_c\rightarrow p_c = K^{-1}p\, p_c '= K'^{-1}p'$

$p_c',p_c$ Comply with specification camera replacement requirements, therefore:

$p_c'Ep_c=p_c'^T[T_{\times}]Rp_c=(K'^{-1}p' )^T [T_{\times}]RK^{-1}p \\ \\ =p'^TK'^{-T}[T_{\times}]RK^{-1}p = 0$

$let F=K'^{-T}[T_{\times}]RK^{-1},\ p'^TFp=0$

F is the fundamental matrix, which reflects the relationship between two viewpoints under a general camera.

Solve for the fundamental matrix

Solve F using corresponding points

What is obtained $\hat{F}$ is often full rank, and the basic matrix we require has a rank of 2

homography matrix

If the collected corresponding points are on the same plane, the correspondence between the two viewpoints is represented by a homography matrix

$M = K(I \ O),\ M' = K'(I\ O),\ P= (\tilde{P^T},1)^T \\ \\ p = M\begin{pmatrix} \tilde{P}\\1 \end{pmatrix} =K \tilde{P} \\ \\ p' = M'\begin{pmatrix} \tilde{P}\\1 \end{pmatrix}=K'(R \ t)\begin{pmatrix} \tilde{P} \\1 \end{pmatrix}=K'(R \tilde{P} + t)$

$n^T \tilde{P} = d \rightarrow n^T \tilde{P} / d = 1 \\ \\ let \ n / d = n_d , n_d^T \tilde{P} = 1$

$p'=K'(R\tilde{P} + t \cdot n_d^T \tilde{P}) = K'(R + t \cdot n_d^T)\tilde{P} =K'(R + t \cdot n_d^T)K^{-1}p$

essential matrix $H = K'(R + t \cdot n_d^T)K^{-1}$

Reference content: Three-dimensional reconstruction of computer vision (an in-depth introduction to the core algorithms of SfM and SLAM) - 4. Basics of three-dimensional reconstruction and polar geometry_bilibili_bilibili

【3D Reconstruction】Epipolar Geometry

Guess you like