Article directory
Chapter 5: Binocular Stereo Vision
1. Binocular stereo vision based on parallel view
1.1 Parallel View Fundamental Matrix
Since e ′ e'e' isO 2 O_2O2In the projection of the right view, the following equations can be listed:
Given the cross-product property (for any vector α \alphaα , ifBBB is reversible, with a difference of one scale):
[ a × ] B = B − T [ ( B − 1 α ) × ] [a_{\times}]B=B^{-T}[(B^{-1 }\alpha)_{\times}][a×]B=B−T[(B− 1 a)×]
Another α = T \alpha=Ta=T， B = K ′ − 1 B=K^{'-1} B=K′−1，则：
[ T × ] K ′ − 1 = K ′ T [ ( K ′ T ) × ] [ T × ] = K ′ T [ ( K ′ T ) × ] K ′ \begin{aligned} [T_\times]K^{'-1} & =K^{'T}[(K'T)_\times] \\ [T_\times] & =K^{'T}[(K'T)_\times]K' \\ \end{aligned} [T×]K′−1[T×]=K′T[(K′T)×]=K′T[(K′T)×]K′
Will [ T × ] [T_\times][T×] into the fundamental matrix F:
F = K ′ − T [ T × ] RK − 1 = K ′ − TK ′ T [ ( K ′ T ) × ] K ′ RK − 1 = [ ( K ′ T ) × ] K ′ RK − 1 = [ e × ′ ] K ′ RK − 1 \begin{aligned} F & =K^{'-T} [T_{\times}] RK^{-1} \\ & =K ^{'-T} K^{'T}[(K'T)_\times]K' RK^{-1} \\ & = [(K'T)_\times]K' RK^{- 1}\\ & = [e'_\times]K' RK^{-1} \\ \end{aligned}F=K′−T[T×]RK−1=K′−TK′T[(K′T)×]K′RK−1=[(K′T)×]K′RK−1=[e×′]K′RK−1
The final derivation is:
F = [ e × ′ ] K ′ RK − 1 F = [e'_\times]K' RK^{-1}F=[e×′]K′RK−1
In a parallel view, the following conditions are met:
- Same for cameras: K = K ′ K=K'K=K′；
- No rotation: R = IR=IR=I；
- xx onlyTranslation in x direction:T = [ T 0 0 ] T=\begin{bmatrix} T\\ 0\\ 0\\ \end{bmatrix}T=⎣ ⎡T00⎦ ⎤；
- Pole at infinity: e ′ = [ 1 0 0 ] e'=\begin{bmatrix} 1\\ 0\\ 0\\ \end{bmatrix}e′=⎣ ⎡100⎦ ⎤;
so:
1.2 Parallel view polar geometry
(1) Parallel view polar lines
Polar lines are horizontal, parallel to uu轴!
(2) Parallel view corresponding point search
Corresponding point ppp andp 'p'p' vv_The v coordinates are the same;
therefore, p ′ p'p′ points only need to be inppvvat point pYou can find the line where the v coordinate is located;
1.3 Parallel view triangulation
Using the triangle similarity principle, we can get
pu − pu ′ f = BZ \frac{p_u-p_u'}{f}=\frac{B}{Z}fpu−pu′=ZB
p u − p u ′ = B ⋅ f Z p_u-p_u'=\frac{B \cdot f}{Z} pu−pu′=ZB⋅f
It can be obtained from this: the parallax is inversely proportional to the depth Z!
That is to say, the farther the object is from the human eye, the more similar the images observed by the left and right eyes;
The conclusion that "disparity is inversely proportional to depth Z" can facilitate us to derive the depth map from the disparity map. (As shown in the figure below, the darker the disparity map is, the farther it is from the binocular camera)
2. Image Correction
In the parallel view, it is very convenient to use the parallax to obtain the depth map. However, in the actual construction of the binocular stereo vision system, how to ensure that the two views are completely parallel, which requires image correction.
The calibration results are as follows:
3. Correspondence point search
image rectification, p ′ p’p' The point can be searched directly along the scanning line;
3.1 Correlation matching
The correlation method is not suitable for cases with obvious brightness changes;
3.2 Normalized correlation matching
The influence of the window size used when matching:
- When the window is small, the details are richer, but the noise is more;
- When the noise is large, the disparity map is smoother and less noisy, but the details are lost;
3.3 Problems with related laws
(1) perspective shortening
(2) Blocking
(3) Baseline selection
- In order to reduce the impact of perspective shortening and occlusion, it is desirable to have a smaller B/Z (baseline depth ratio) ratio;
- However, when B/Z is too small, a small error in corresponding point matching means a large error in estimated depth;
(4) Correlation methods fail when there are homogeneous regions or repeating patterns;