3D reconstruction - camera geometry
1. Pinhole camera
Principle: We will find that in the small hole imaging, there may be many points mapped by the object corresponding to a point on the film, so we can add a partition with a pinhole between the object and the film. In this case, each object A dot corresponds to only one dot on the film. (However, the pinhole imaging is equal and reversed!)
As shown in the figure below:
3D reconstruction generally analyzes the virtual phase plane similar to the real picture, and the phase plane is a reverse, which basically makes no difference. All it takes is proper adjustments.
Next, we analyze the positional relationship between the real position and the image plane:
(The image plane is two-dimensional, and the space is three-dimensional. We use similarity to replace the position on the z-axis with two-dimensional coordinates.)
Here we use It is a simple similarity problem in mathematics. The image plane is mainly formed at the focal length position, and we can obtain the image x, y position by using the similarity. The above picture only expresses the expression of y on the image plane , as for the expression of x , I think it is just a picture.
Aperture effect of pinhole camera: :
The smaller the aperture, the clearer but also darker, the larger the aperture, the blurrier but also brighter. In this case, when we consider clarity, we will find a way to increase the brightness of the image. At this time we need to use the lens.
2. Lens
**Principle of action:** Focus the light emitted by the object onto a point on the film. In this way, the amount of light increases and the brightness of the light increases. This can effectively solve the problem of reducing the aperture. There is a problem with reduced brightness.
Features: 1. All rays parallel to the optical axis will converge to one point—the focus, and the distance from the focus to the center of the lens becomes the focal length.
2. The direction of the light rays passing through the center does not change.
Paraxial refraction model; 
f : focal length, distance from **z'** pinhole to image plane.
The conversion relationship between the real position and the image plane is shown in the figure.
Problems:
1. Out of focus : There is a specific distance for the object to focus - the focal length. If your object is projected beyond or below it, then it may not be clearly displayed on the film.
2. Radial distortion : The image pixel point takes the distortion center as the center point, and the position deviation along the radial direction
causes the image formed in the image to be distorted . Curved
pincushion distortion : offset radially outward
Barrel distortion : offset radially towards the center
3. The relationship between image plane, pixel and world coordinate system
1. Image plane ------> pixel plane
The unit of the image plane: meter (m), the unit of the pixel plane: pixel (pixel) The
coordinate centers of the two planes are generally different,
the above picture is an offset The problem is to convert the three-dimensional coordinates in a space into the two-dimensional coordinates of pixels, and the unit and the origin of the coordinates have changed. (Cx, Cy are internal parameters of the camera).
Here, we are talking about the conversion of units. There may be doubts about the situation of one fk and one fl . This is related to the craftsmanship of the camera. If the craftsmanship is good, then k may be equal to l, but generally Deviation, this specific reference to the parameters of the camera.
For general simplification, we represent fk by α and fl by β , as shown in the figure below:
This is a mapping formula between spatial coordinates and pixel coordinates.
By the way, the above formula looks linear, but if you look closely, x, z, and y are all variables, so this is not a linear relationship.
Conversion of homogeneous coordinates and Euclidean coordinates:
As can be seen from the picture, this is the conversion through some basic knowledge of linear algebra, so as to facilitate subsequent calculations and expressions.
In this way, we can convert the points in the space into points on the surface of the object and express them with homogeneous coordinates: it
can also be written like this:
here M represents the projection matrix , as mentioned before, the two planes are different , In this case, a relationship is needed to connect the two of them and build a relationship for transformation. We can see that the above α, β, cx, and cy are all fixed, so he said it is a fixed matrix. However, this is not necessarily fixed, as shown in the figure below:
We found that the camera is tilted, causing the two coordinate axes of the pixel coordinate system to be non-perpendicular. At this time, it is necessary to consider the case where the included angle is not 90°.
At this time, the projection matrix can be written in this form. Of course, it has a new name at this time: the internal parameter matrix of the camera (because it is determined by the process of the camera).
This internal parameter matrix consists of α, β, Angle, cx, cy5 degrees of freedom.
World coordinate system: When we describe the motion of an object, we all know that if we use different reference systems, the description we may get may be different. At this time, we need a reference system that can be described uniformly. The world coordinate system is used to solve such problems.
At this time, we have to consider how to establish a connection between the pixel coordinate system and the world coordinate system.
Pw is a point in the world coordinate system, P' is the homogeneous coordinate of a point in the pixel coordinate system, and P is a point on the camera coordinate. This It is necessary to transform, replace, etc., such as bringing the homogeneous coordinate system P into P'.
Intrinsic parameters: the mapping from camera points to pixel coordinate system.
External parameters: the relationship between the camera and the world.
R and T in the three-dimensional internal parameter matrix each have three degrees of freedom, plus five degrees of freedom for internal parameters, we can find that the projection matrix M has 11 degrees of freedom.
The European coordinates are as follows:
m1, m2, and m3 above are all projection matrices.
The picture data comes from the video of station B.