Notes on "Multiple View Geometry in Computer Vision" (2)

2 Projective Geometry and Transformations of 2D

This chapter mainly introduces the necessary geometric knowledge and symbols for this book.

2.1 Planar geometry

Plane geometry is briefly introduced, and the book will be taught in a mixed way of algebra and geometry.

2.2 The 2D projective plane

Row vectors and column vectors. By default, all vectors in this book are column vectors, such as $x$ , then$x^T$ is the row vector. For a row vector$(x,y)$ , we have$x=(x,y{)}^T$。

2.2.1 Points and lines

Homogeneous coordinates of a line segment The line is given by the equation $ax+by+c=0$ , so we use$(a,b,c{)}^T$ represents the line segment. But$(a,b,c{)}^T$ cannot uniquely represent a line segment because$(a,b,c{)}^T$与$k(a,b,c{)}^T$ represents the same line segment ($k$ is not 0). Then$k(a,b,c{)}^T$ is actually the expression of a type of vector. All vectors in this class are secondary vectors. Then we put all the classes together to form a projection space. There is a special point in space$(0,0,0{)}^T$ , which does not belong to any straight line.

The second coordinate of the point and the equation of the straight line are $ax+by+c=0$ , then it can be written as$(x,y,1)(a,b,c{)}^T$ , then this$(x,y,1)$ is the homogeneous coordinate of the point.

Conclusion 2.1 A point is on a straight line if and only if $x^{T}l=0$

Degree of freedom The degree of freedom means that this geometry can be expressed by several freely changing variables. For example, the degree of freedom of a point is 2, because $x,y$ is enough, and the degree of freedom of the line is also 2. This is because although the line has three variables, the ratio between them is$a:b:c$ . For example, in a non-homogeneous representation, these two parameters can be chosen as the gradient and y-intercept of the line.

Intersection of straight lines Two straight lines $l=(a,b,c)，l^{\prime }=(a^{\prime },b^{\prime },c^{\prime })$, their intersection point is$l\times l^{\prime }$。

A straight line determined by two points Two points $x,x^{\prime }$ , the straight line they determine is$x\times x^{\prime }$。

2.2.2 Ideal points and the line at infinity

Intersection point of parallel lines If we consider the intersection point of two parallel lines $ax+by+c=0，ax+by+c^{\prime }=0$ , doing the cross product, we will get$(c^{\prime }-c)(b,-a,0{)}^T$ , if we ignore the scale factor$(c^{\prime }-c)$ , then the intersection of the parallel lines is at$(b,-a,0{)}^T$ , we then change this homogeneous coordinate into non-homogeneous, then we get$(b/0,-a/0{)}^T$ , this point is at infinity, so we say that parallel lines intersect at infinity.

Ideal point and straight line at infinity We consider any point $(x_1,x_2,x_3{)}^T$ , set$x_3=0$ , 那么$(x_1,x_2,0{)}^T$ is all points at infinity. These points all fall on a straight line, that is,$l_{\infty }=(0,0,1{)}^T$。

We then consider any straight line $l=(a,b,c{)}^T$，$l$与$l_{\infty }$The intersection point is $(b,-a,0{)}^{T.}$ _ Then one and$l$ parallel straight line$l^{\prime }$ will sum with$l_{\infty }$Also intersects with $(b,-a,0{)}^T$。$(b,-a)$ and the normal vector of the straight line$(a,b)$ is vertical, so it is the direction of the line. Note that$(a,b)$ is not the direction of a straight line,$(a,b)$ is perpendicular to the straight line. $(b,-a,0)$ This point is on the straight line at infinity, then the straight line at infinity can be regarded as a set of straight line directions.

Geometric model of a two-dimensional projection plane A projection plane can be imagined as a collection of rays in three-dimensional space. You can pick three points from three rays and make them coplanar, then all other rays will have intersections with the plane. So the plane is made up of points on the ray. A line on the projective plane is the intersection of the plane passing through the origin and the projective plane. Any two different rays lie on the same plane, and any two different planes are compared to a ray. It can be compared to two straight lines intersecting at one point, and the two points determine a straight line.

As shown in the figure, the ideal point at infinity and the straight line are parallel to the plane $x_3=1$。

Duality between Line Segments and Points The roles of points and line segments can actually be interchanged. For example, $l^{T}x=0$ can be written as$x^{T}l=0$。

2.2.3 Conics and dual conics

A cone describes a quadratic equation in a plane. There are three main types of Euclidean geometry: parabola, hyperbola, and ellipse. In two-dimensional photographic geometry, these three are equivalent.

We first write the cone as and then the expression $x=(x_1,x_2,x_3)$

$$\left[\begin{array}{ccc}a& b/2& d/2\\ b/2& c& e/2\\ d/2& e/2& f\end{array}\right]$$

The cone can be written as $x^{T}Cx=0$ A cone has 5 degrees of freedom.

Five points determine a cone. Let’s write the cone in another way, using $x=x_1/x_3,y=x_2/x_3$，可以得到：
$$\begin{gathered} (x_i^2,x_i{y}_i,y_i^2,x_i,y_i,1)c=0 \\ c=(a,b,c,e,d,f) \end{gathered}$$

Five equations are needed to solve $c$ , because$The\; degrees\; of\; freedom\; of\; c$ are 5.

Line tangent to ellipse and ellipse $C$ is at pointxxtangent linell$at\; x$$l$ is$l=Cx$。

Dual Conic In the previous articleccThe cone defined by$c\; is\; a\; cone\; composed\; of\; points.$We can define a cone c ∗ c^*consisting of straight lines$c^{\ast }$ , this cone is composed of all and$c$ consists of tangent straight lines. $c^{\ast }=c^{-1}$

2.3 Projective transformations

Definition 2.9 Projection transformation $h$ is a transformation from two-dimensional projection space to two-dimensional projection space, which satisfies a property: if$x_1,x_2,x_3$On a straight line before transformation, if and only if they are after transformation $(h(x_1),h(x_2),h(x_3))$ is still on a straight line.

According to this definition, projection transformation is also called collinearity, and projection transformation and homography have the same meaning.

The book also introduces another way to define it from an algebraic perspective. In layman's terms, it is any $3\times$The non-singular matrixHH$of\; 3$$H$ both defines a projection transformation.

Explained from a geometric perspective, projection transformation actually defines a plane-to-plane mapping, because we know that projection geometry is defined by a plane. Moreover, the projection transformation maintains collinearity. If the coordinate systems on these two planes are both Euclidean coordinate systems, then this projection transformation becomes a perspective transformation with 6 degrees of freedom.

2.3.1 Transformations of lines and conics

A point has the following transformation $x^{\prime }=Hx$ , then the line transformation is$l^{\prime }=H^{-T}l$, the conic transformation is$x^{T}Cx=x^{\prime T}{H}^{-T}C{H}^{-1}{x}^{\prime }$ , so$C=H^{-T}C{H}^{-1}$ , its dual circle transformation is$C^{{\ast }^{\prime }}=H{C}^{\ast }{H}^T$。

2.4 A hierarchy of transformations

This section is a key chapter, which introduces the definitions and properties of various transformations in a layer-by-layer manner.

2.4.1 Isometries rigid body transformation

Determine:
$$\left(\begin{array}{c}{x}^{\prime }\\ y^{\prime }\\ 1\end{array}\right)=\left[\begin{array}{ccc}ϵ\cos i& -\sin & t_x\\ ϵ\sin i& -\cos & t_y\\ 0& 0& 1\end{array}\right]\left(\begin{array}{c}x\\ y\\ 1\end{array}\right)$$
It can be abbreviated as:
$$x^{\prime }=H_{E}x=\left[\begin{array}{cc}R& t\\ 0^T& 1\end{array}\right]x$$
$R$ is a$2\times An\; orthogonal\; matrix\; of\; 2$ , the entire large matrix has three degrees of freedom: one for rotation and two for translation. The invariants are: the length of the line segments, the angle between the line segments, and the area of ​​the figure.

2.4.2 Similarity transformations Similarity transformations

Determine:
$$\left(\begin{array}{c}{x}^{\prime }\\ y^{\prime }\\ 1\end{array}\right)=\left[\begin{array}{ccc}s\cos i& -s\sin & t_x\\ s\sin i& s\cos & t_y\\ 0& 0& 1\end{array}\right]\left(\begin{array}{c}x\\ y\\ 1\end{array}\right)$$
It can be abbreviated as:
$$x^{\prime }=H_{S}x=\left[\begin{array}{cc}sR& t\\ 0^T& 1\end{array}\right]The\; entire\; x$$
matrix has 4 degrees of freedom: one for scaling factor, one for rotation, and two for translation. The invariants are: the angle between the line segments, whether the lines are parallel or parallel, and the ratio between the line segments does not change. Because the entire graphic is scaled, the area ratio between different areas does not change.

2.4.3 Affine transformationsAffine transformations

其形式如下：
$$\left(\begin{array}{c}{x}^{\prime }\\ y^{\prime }\\ 1\end{array}\right)=\left[\begin{array}{ccc}{a}_{11}& a_{12}& t_x\\ a_{21}& a_{22}& t_y\\ 0& 0& 1\end{array}\right]\left(\begin{array}{c}x\\ y\\ 1\end{array}\right)$$
可以简写成：
$$x^{\prime }=H_{A}x=\left[\begin{array}{cc}A& t\\ 0^T& 1\end{array}\right]The\; entire\; matrix\; x$$
has 6 degrees of freedom, and the upper left corner$Four\; for\; A$ , two for translation.

$A$ can be decomposed into the following form:
$$A=R\left(\theta \right)R(-\varphi )DR\left(\varphi \right)$$
$$\left[\begin{array}{cc}{l}_1& 0\\ 0& l_2\end{array}\right]$$

So $A$ can be interpreted as first rotating by an angle$\varphi$ , and then from$x,Scaling\; is\; performed\; in\; both\; directions\; of\; y$ , and the scaling factors are$l_1,l_2$, then press $-\phi$ rotates back and then rotates$i$ .

Due to the compression, the angle between the line segments changes, and the invariant can only maintain the parallelism between the straight lines, the ratio between the straight lines, and the ratio of the area.

2.4.4 Projective transformationsProjective transformations

Projective transformation is a general non-singular linear transformation of homogeneous coordinates. It actually generalizes affine transformation. We have seen the role of projective transformation before.
Its form is as follows:
$$x^{\prime }=H_{P}x=\left[\begin{array}{cc}A& t\\ v^T& v\end{array}\right]xThe$$
entire large matrix has 8 degrees of freedom. Its invariant: a straight line is still a straight line after transformation.

2.4.5 Summary and comparison

2.4.6 Decomposition of a projective transformation

The entire projection transformation matrix can be decomposed into three small matrices:
$$\begin{gathered} H=H_S{H}_A{H}_P \\ =\left[\begin{array}{cc}sR& t\\ 0^T& 1\end{array}\right]\left[\begin{array}{cc}K& 0\\ 0^T& 1\end{array}\right]\left[\begin{array}{cc}I& 0\\ v^T& v\end{array}\right]=\left[\begin{array}{cc}A& t\\ v^T& v\end{array}\right] \end{gathered}$$
$H_P$Moving a straight line to infinity, $H_A$is an affine transformation, $H_S$is a generalized similarity transformation.

2.4.7 The number of invariants

Earlier we discussed how many invariants a geometry has under certain transformations. So how to calculate this invariant? We have the following conclusions:

Conclusion 2.16 The invariant of a geometry is greater than or equal to the degrees of freedom of the geometry minus the degrees of freedom of the transformation.

For example, 4 points in space have 8 degrees of freedom, because each point has 2. Then the invariant of the geometry is: the degrees of freedom of the geometry, 8, minus the degrees of freedom of the transformation. If we assume that the transformation is a similarity transformation, then the answer is 8-4=4 (similarity transformations have 4 degrees of freedom). Assuming that the transformation is an affine transformation, the answer is 8-6=2 (affine transformation is 6 degrees of freedom).

2.5 The projective geometry of 1D

One-dimensional space projection geometry is point $\bar{x}=(x_1,x_2{)}^T$ , where$x_2=0$ , the one-dimensional homography matrix is:
$${\bar{x}}^{\prime }=H_{2\times x}\bar{x}$$
$H_{2\times 2}$There are 3 degrees of freedom.

The cross ratio
Given 4 points on a one-dimensional plane, we define a cross ratio:
$$Cross(\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4})=\frac{|\overline{x_1}\overline{x_2}||\overline{x_3}\overline{x_4}|}{|\overline{x_1}\overline{x_3}||\overline{x_2}\overline{x_4}|}$$

其中
$$|\overline{x_i}\overline{x_j}|=\mathrm{the}\left[\begin{array}{cc}{x}_i& x_j\\ x_{i2}& x_j\end{array}\right]$$

The crossover ratio has several properties:

1. The cross ratio has nothing to do with what coordinate system is used, because the ratio between the numerator and the denominator cancels each other out.
2. If every point is a point at a finite distance and $x_2=1$ , then∣$|{\bar{x}}_i{\bar{x}}_j|$ represents the starting point from${\bar{x}}_i$to ${\bar{x}}_j$the signed distance of
3. If one point is an ideal point, the crossover ratio still holds
4. The crossover ratio is invariant under any projection transformation

Concurrent lines
are lines with a common starting point, and then find one more line to intersect all the common lines, so that the intersection ratio can be defined.

2.7 Recovery of affine and metric properties from images

This section is mainly to remove the loss of properties caused by projection transformation and restore the image from projection transformation to similarity transformation, so properties such as parallel lines, line segments, and area ratios are retained.

Because we know that the projection transformation only has 4 more degrees of freedom than the similarity transformation, then we only need to restore 4 degrees of freedom. Where do these 4 degrees of freedom come from? The line at infinity provides 2, and two absolute points at infinity, since they are invariant under similarity transformations. It can also be called a cone point, because any cone intersects the line at infinity at these two points.

2.7.1 The line at infinity

Under projection transformation, lines at infinity are projected to non-infinity.
A line at infinity is unchanged under affine transformation, which means that after affine transformation, it is still at infinity. But the positions of the points on the line have changed, except that the points are all at infinity.

2.7.2 Recovery of affine properties from images

We know that to restore the affine properties we need to find the line at infinity. So we first make it clear that the camera is a projection transformation, then the line will be mapped to a certain place in the image coordinate system. We first find this place and then use the properties of 2.7.1 to establish an equation.

Assume that the line at infinity is mapped to $l=(l_1,l_2,l_3{)}^T$ , we know that the coordinates of the line at infinity are$(0,0,1{)}^T$ and the line is invariant under affine transformation, then we construct a matrix:
$$H=H_A\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ l_1& l_2& l_3\end{array}\right]$$

$H_A$is any affine transformation, $H$ can put$l$ transforms to$(0,0,1)$ , then we take$H$ is multiplied by the entire image, so that the entire image regains its affine properties.

Then the next question is how to find $l$ , we find two parallel lines from the image and extend them. They must intersect, so it is a point. Again, so there are two points. These two points determine$l$ . There is another method in the book, Example 2.20 on P51.

2.7.3 The circular points and their dual

We've already covered what an elliptical point is, so now let's take a look at what it looks like.
We use $I,J$来表示，$I=(1,i,0{)}^T,J=(1,-i,0{)}^T$ ,find one of the
following equations
$$\begin{gathered} I^{\prime }=H_{s}I \\ =\left[\begin{array}{ccc}s\cos i& -s\sin i& t_x\\ s\sin i& s\cos i& t_x\\ s0& 0& 1\end{array}\right]\left(\begin{array}{c}1\\ i\\ 0\end{array}\right) \\ =with{e}^{-i\theta }\left(\begin{array}{c}1\\ i\\ 0\end{array}\right) \\ =I \end{gathered}$$

According to the above formula, we have the following conclusion:
Conclusion 2.21 Ellipse point $I,J$ remains unchanged under projection transformation if and only if the projection transformation is a similarity transformation.

How were these two points found? is the conic equation with $l_{\infty }$intersection point.

* Dual cones defined by cone points
We can use $I,J$ to define a cone
$$C_{\infty }^{\ast }=I{J}^T+J{I}^{TthisC\infty }$$
∗$C_{\infty }^{\ast }$It is a cone composed of straight lines, which is the degeneration of the cone of 2.2.3 knot lines. So who is it about? It is the dual of a cone point.

$C_{\infty }^{\ast }$It is also invariant under similar transformations. So we can have the following conclusions:

Conclusion 2.22 Cone point $C_{\infty }^{\ast }$is invariant under a projective transformation if and only if the projective transformation is a similarity transformation.

$C_{\infty }^{\ast }$There are two more properties. 1. There are four degrees of freedom 2. $l_{\infty }$is $C_{\infty }^{\ast }$The zero vector of

2.7.4 Angles on the projective plane

Suppose there are two straight lines $l=(l_1,l_2,l_3{)}^T$, $m=(m_1,m_2,m_3{)}^T$ , the angle between them is:
$$\cos i=\frac{l_1{m}_1+l_2{m}_2}{\sqrt{(l_1^2+l_2^2)(m_1^2+m_2^2)}}$$

If for $l,m$ applies projection transformation, the above formula is no longer applicable. In order to calculate the angle after projection transformation, we have the following formula:
$$\cos i=\frac{l^T{C}_{\infty }^{\ast }m}{\sqrt{(l^T{C}_{\infty }^{\ast }l)(m^T{C}_{\infty }^{\ast }m)}}$$

So we know $C_{\infty }^{\ast }$You can calculate the angle between line segments or planes ( Conclusion 2.23 ).

There is also an obvious conclusion in the book: if $l^T{C}_{\infty }^{\ast }m=0$ , then$l,m$ vertical.

2.7.5 Restoring the metric properties of images

Metric properties refer to angles, ratios between line segments, etc. Mainly use $C_{\infty }^{\ast }$, this is because under the projection transformation, the following formula exists:
$$\begin{gathered} C_{\infty }^{{\ast }^{\prime }}=(H_P{H}_A{H}_S)C_{\infty }^{\ast }(H_P{H}_A{H}_S{)}^T \\ =\left[\begin{array}{cc}K{K}^T& K{K}^{T}v\\ v^{T}K{K}^T& v^{T}K{K}^{T}v\end{array}\right] \end{gathered}$$

$K$ is the upper left component of the affine transformation,$v$ is the component of the projective transformation. From the above formula, we can see that as long as we know$C_{\infty }^{\ast }$You can find $C_{\infty }^{{\ast }^{\prime }}$, and then do SVD decomposition, you can find $K,v$。

For specific details, please refer to P56 Example2.26.

2.8 More properties of conics

This chapter is also the focus. It introduces the relationship between points, lines, and cones, which is the basis of epipolar geometry.

2.8.1 The pole–polar relationship

One point $x$ and a cone$C$ can determine a straight line$l=Cx$ , this$l$ is called the polar line. Pay attention to this$x$ is not in the cone$on\; C$ , but onCCoutside of$C.$Passed$x$ can submit to$C$ makes two lines (note that this lineis not a polar line, I call ita tangent line). Everytangent lineis tangent to the cone, as shown in the figure below. We can imagine$x$ gradually moves toward the cone, so the angle between the two tangent lines gradually increases. When$x$ is located on the cone, and the two tangent lines become one tangent line. Another concept is introduced below: Definition 2.29

Correlation between points and linesCorrelation is a reversible mapping from points in two-dimensional projection space to lines in two-dimensional projection space. It is a3 × 3 3 \times 3
$3\times The\; non-singular\; matrix\; of\; 3$ (non-singular so reversible), we denote it as$A$ , then the entire correlation can be expressed as$l=Ax$。

This $A$ provides the relationship between points and lines, but$A$ is not symmetrical. So if$A$ is symmetrical, what will happen? This leads to the concept of conjugate points:

Conjugate point point $y$ in byxxOn the polar line determined by$x$$y$ and$x$ is the conjugate point, expressed as$y^{T}l=y^{T}Cx=0$。

So $C$ describes the relationship between points and lines.

In addition, the conjugate point has a property: $x$ if in$On\; the\; polar\; line\; of\; y$ ,$y$ will also be in$On\; the\; extreme\; line\; of\; x$ .

2.8.2 Classification of conics

Cones can identify three categories: hyperbola, parabola, and ellipse. They are formed by the intersection of planes and cones respectively. Then if we consider it from the perspective of projective geometry and use a straight line at infinity to intersect an ellipse, if there is no real intersection point, then an ellipse is formed. If there is an intersection point, it is a parabola. If there are two intersection points, that is Hyperbola, as shown in the figure below:

If we think about it from an algebraic point of view, $Decompose\; C$ using SVD to get$C=U^{T}DU$, where$D$ is the eigenvalue of the matrix, let$D$ is decomposed again by SVD, ensuringDDThe eigenvalue of$D$$Different\; D$ eigenvalues ​​result in different types of cones, as shown in the following table:

2.9 Fixed points and lines

We know that $l_{\infty }$and elliptical points are invariant under projection transformation. So, if a transformation is regarded as a matrix, and points and lines are regarded as vectors, then what kind of vectors remain unchanged under the action of the matrix? The vector corresponding to the eigenvalue. So those unchanged points and lines are the eigenvectors of the projection matrix.

The fixed points in different transformations are introduced below:

The eigenvalues ​​of Euclidean transformation (rigid body transformation) are { ei θ , e − i θ e^{i \theta}, e^{-i \theta} $e^{i\theta },e^{-i\theta }$ }, the two fixed points are the circular points (ellipse points) mentioned above.

Similar changes and special expeditions {1, $with{e}^{i\theta },with{e}^{-i\theta }$ }, the two fixed points are the elliptical points mentioned above.

The two fixed points of affine transformation can be real points or complex points, but the fixed line $l_{\infty }$It is true in any case.