Binocular vision practice--single view measurement method

Binocular vision uses two cameras or image sensors to simulate the distance difference (ie, parallax) between human eyes to achieve the acquisition of object depth information and three-dimensional reconstruction. By calculating the difference in object position between two viewing angles, the distance of the object relative to the camera can be obtained, thereby achieving depth estimation and three-dimensional reconstruction. It is widely used in machine vision, robotics, virtual reality, augmented reality and other fields.

Single-view measurement refers to a method that uses an image to estimate the geometric dimensions of objects in it. It is commonly used in engineering, manufacturing, construction and other fields to quickly measure and analyze the size and shape of objects.

Single view measurements typically involve the following steps:

Select an object to measure and capture an image: Select the features to be measured from the object to be measured and capture an image containing those features.
Estimating object size: Use existing camera parameters and basic geometric knowledge (such as triangulation, proportional relationships, etc.) to infer the geometric size of objects in the image. Relatively advanced computer vision technologies (such as structured light, image matching, etc.) can also be used to improve measurement accuracy and efficiency.
Verify measurement results: Verify and calibrate measurement results according to actual measurement needs, such as comparing with actual on-site measurement values, or using measurement results from multiple perspectives for fusion and correction.

The advantage of single-view measurement is that it is simple and intuitive and does not require additional sensors and measurement equipment. However, since only one image is used for estimation, there are certain errors and uncertainties, especially when the shape of the object and the illumination change greatly. Therefore, in practical applications, it is usually necessary to combine other measurement methods and technologies to improve accuracy and reliability.

2. 2D transformation

1. Isometric transformation (Euclidean transformation)

Isometric Transformation, also known as Rigid Transformation, refers to a transformation that keeps the shape, size, and length of an object unchanged in two- or three-dimensional space.

Isometric transformation includes three basic operations: translation, rotation and reflection. Translation is to move the entire object a certain distance in a certain direction, rotation is to transform the rotation angle around a certain point, and mirroring is to flip the object to make it symmetrical.

In two-dimensional space, the isometric transformation can be expressed as:

Translation transformation:

[x']   [1 0 tx] [x]
[y'] = [0 1 ty] [y]

Rotation transformation:

[x']   [cosθ -sinθ] [x]
[y'] = [sinθ cosθ]  [y]

Mirror transformation:

Mirror the x-axis:

[x']   [1  0] [x]
[y'] = [0 -1] [y]

Mirror the y-axis:

[x']   [-1 0] [x]
[y'] = [0  1] [y]

Mirror the origin:

[x']   [-1  0] [x]
[y'] = [ 0 -1] [y]

In three-dimensional space, isometric transformation is expressed in a similar way, except that more dimensions and parameters need to be added to describe the movement, rotation and mirroring operations of objects.

Features:

Invariants:Length, area
3 degrees of freedom
Suitable for describing the motion of rigid objects

2. Similarity transformation

Similarity transformation refers to a transformation that maintains the shape and proportion of an object, including the three basic operations of translation, rotation and scaling.

In two-dimensional space, similarity transformation can be expressed as:

Translation transformation:

[x']   [1 0 tx] [x]
[y'] = [0 1 ty] [y]

Rotation transformation:

[x']   [cosθ -sinθ] [x]
[y'] = [sinθ cosθ]  [y]

Scale transformation:

[x']   [s 0] [x]
[y'] = [0 s] [y]

Among them, (x, y) is the coordinate of the original point, (x', y') is the coordinate of the new point obtained after similarity transformation, (tx, ty) is the translation vector, θ is the rotation angle, and s is the scaling factor.

Similar transformation does not change the shape and proportion of the object, but only moves, rotates and scales the object's position. They are all linear transformations, which maintain the properties of straight lines, but do not necessarily maintain other geometric properties such as angles and curvature.

Features:

Invariant:Length ratio, angle
4 degrees of freedom

3. Affine transformation

Affine Transformation refers to a transformation that maps a point or object to another space through linear transformation and translation transformation. It can maintain the parallelism, linearity and proportional relationship of the object, but not necessarily the angle and curvature of the object.

Affine transformations can be expressed using linear algebra matrix operations, usually using a 2x2 matrix to represent the rotation and scaling parts, plus a 2x1 translation vector. On a two-dimensional plane, affine transformation can be expressed in the following matrix form:

[x']   [a b] [x]   [tx]
[y'] = [c d] [y] + [ty]

where(x, y) is the coordinate of the original point, (x', y') is the coordinate of the new point obtained after affine transformation, (a, b, c, d) is the parameter of rotation and scaling, (tx, ty) is the vector of translation.

Affine transformation is widely used in computer vision and image processing. For example, image rotation, scaling, translation, flipping and other operations can be achieved through affine transformation. In addition, affine transformation can also be used in tasks such as image registration, image correction, feature alignment, and face recognition.

It is worth noting that affine transformation is a linear transformation that maintains the properties of straight lines but not curves. If you need to deal with non-linear transformations, such as perspective projection, you need to use a more complex projective transformation (Projective Transformation).

Features:

Invariant:The ratio of parallel lines and areas
Unable to maintain the angle relationship between the original lines
6 degrees of freedom

4. Projective transformation (perspective transformation)

Projective Transformation refers to a transformation that projects a point or object in one space into another plane or space through the mapping of a perspective camera or perspective view. It can usually be represented by a matrix, so it is also called Homogeneous Coordinates Transformation.

Projective transformation is a nonlinear transformation that can be represented by a 3x3 matrix, and because it has transformation effects such as rotation, translation, scaling, and projection, it can complete more complex geometric transformation tasks. In computer vision, the most commonly used projective transformation is that under the Pinhole Camera Model, which assumes that the relationship between the camera and the scene is perspective projection. In image processing, projective transformation can be used to correct and correct various images, such as perspective distortion correction, tilt correction, etc.

Features:

Invariant:Collinearity, intersection ratio of four collinear points
Cannot maintain the original parallel relationship
8 degrees of freedom

5 Conclusion

Euclidean transformation is a special case ofsimilar transformation
Similar transformation is a special case ofaffine transformation
affine transformation is a special case ofperspective transformation

3. Shadow cancellation point and shadow cancellation line

1. Lines on a plane

2. Intersection point of straight lines

3. 2D infinity point

4. Infinity straight line

The line at infinity is the geometry of a point at infinity and is also a collection of "directions" of lines on a plane.

5. Perspective transformation and affine transformation of infinity point

The point at infinity is not the point at infinity after perspective transformation

An infinite point is still an infinite point after affine transformation

6. Perspective transformation and affine transformation of infinity line

The line at infinity is not the line at infinity after perspective transformation

An infinite line is still an infinite line after affine transformation

7. Surfaces and straight lines in space

A straight line in space is defined as the intersection of two planes. Generally used to describe the direction of a straight line.

8. Infinity point in three-dimensional space (intersection point of parallel lines in space)

The intersection point of parallel lines in space

9. Shadow vanishing point

Definition:The projection point of the infinity point on the image plane

Assume that the straight line direction in space is

Then the shadow vanishing point and the straight line direction satisfy the following relationship:

10. Shadow elimination line (eye level)

The cancellation line is a collection of cancellation points obtained by perspective transformation of multiple infinite points on the same plane in space. It is also a plane straight line obtained by perspective transformation of an infinite line in space.

The relationship between the shadow cancellation line and the straight line at infinity:

The relationship between the shadow cancellation line and the plane normal vector:

11. Infinity plane

Parallel planes intersect at a common line at infinity, which is called an infinity straight line.

The set of two or more infinite straight lines is defined as the infinite plane

12. The relationship between the angle between two sets of parallel lines and the vanishing point

Based on the relationship between the shadow vanishing point and the direction of the straight line,

We can get the relationship between the angle between two sets of parallel lines in space and the shadow cancellation point:

In addition, it has the following properties:

application:

Estimating camera parameters (single view calibration)
Restore 3D scene structure

3. Single view reconstruction

1. Single view calibration

Select three mutually perpendicular planes and find the shadow cancellation points on the three planes. The following three equations can be listed:
There are 5 degrees of freedom, but only three equations can be listed. If we assume zero tilt and square pixels, then there are only three degrees of freedom left. In this way, the camera internal parameter matrix can be solved based on the three shadow elimination points.

2. Single view reconstruction

The shadow cancellation line of this plane can be drawn from two shadow cancellation points. According to the previously calculated internal parameter matrix /(K/), the normal vector of this plane can be obtained:
In the same way, the normal vectors of the other two planes can be found. According to the equations of the three planes, the picture can be reconstructed.

Notice:

The actual proportion of the scene cannot be restored;
Need to manually select the shadow cancellation point and shadow cancellation line;
Need a priori information of the scene, such as points Corresponding relationships, line and surface geometric information, etc.
The mapping from 2D to 3D in a single view is ambiguous