【Reprint】The principle of monocular camera calibration

The purpose of camera calibration is to establish the relationship between the pixel coordinate system and the world coordinate system. The principle is to solve the model parameters of the camera from the image coordinates of the known feature points according to the camera model, so that the three-dimensional coordinates of the spatial points can be recovered from the image. The parameters to be solved include 4 internal parameters and 5 distortion parameters, and the external parameters are three rotation parameters of the rotation matrix and three parameters of the translation vector.

1. Pinhole camera model

There are generally three camera imaging models: lens projection model (pinhole camera model), orthographic projection model and perspective projection model. As follows:

What is the focal length? The focal length of a lens is basically the distance from the center point of the lens to the film plane or digital CCD that forms a clear image.

The relationship between object distance u, focal length f, and phase moment v $\frac{1}{f}=\frac{1}{u}+\frac{1}{v}$ is as follows:

2. Four major coordinate systems

1. Pixel coordinate system and image coordinate system (CCD plane)

The origin of the pixel coordinate system is the upper left corner of the image, that is, the uv coordinate system; the origin of the image coordinate system is the center of the image, that is, the xy coordinate system. The mapping relationship is as follows:

Express the above formula in the form of homogeneous coordinates and matrix, as follows:

Assuming that the unit in the image coordinate system is mm, then the unit of dx is mm/pixel, and the unit of x/dx is pixel. The inverse relationship of the above formula is as follows:

2. Camera coordinate system

Point O is the optical center of the camera, and $X_{c},Y_{c},Z_{c}$ the rectangular coordinate system composed of point O and the axis is the camera coordinate system, which $OO_{1}$ is the focal length of the camera, $Z_{c}$ and the axis is the optical axis of the camera, and is perpendicular to the image plane.

3. World coordinate system

The world coordinate system is introduced to describe the position of the camera, $O_{w}X_{w}Y_{w}Z_{w}$ that is, the world coordinate system. The translation vector t and the rotation matrix R are used to represent the relationship between the camera coordinate system and the world coordinate system. Assume that the homogeneous coordinates of the space point P in the world coordinate system are $(X_{w},Y_{w},Z_{w},1)^{T}$ , and the homogeneous coordinates in the camera coordinates are $(X_{c},Y_{c},Z_{c},1)^{T}$ . As follows:

Among them, R is a 3×3 orthogonal identity matrix (that is, a rotation matrix), and t is a three-dimensional translation vector. Vector $\vec{O}=(0,0,0)$ , $M_{1}$ which is a 4×4 matrix.

The relationship between the camera coordinate system and the world coordinate system is as follows:

(1) Rotation matrix R

In three-dimensional space, rotations can be decomposed into two-dimensional rotations about the respective coordinate axes, where the metric of the axis of rotation remains unchanged. If the angles ψ, φ, and θ are rotated around the x, y, and z axes in turn, then the total rotation matrix R is the product of the three matrices Rx(ψ), Ry(φ), and Rz(θ). As follows:

(2) Translation vector t

The translation vector is the offset of the origin of the first coordinate from the origin of the second coordinate. To move from the coordinate system with the object center as the origin to another coordinate system with the camera as the origin, the corresponding translation vector is t=object origin-camera origin. Then from the coordinates in the world coordinate system $P_{o}$ to the coordinates in the camera coordinate system $P_{c}$ , as follows:

$P_{c}=R(P_{o}-T)$

The three-dimensional rotation is represented by angles ψ, φ and θ, and the three-dimensional translation is represented by parameters $(x,y,z)$ , with a total of 6 parameters. The internal parameters (camera matrix) are $f_{x},f_{y},c_{x},c_{y}$ and the external parameters are R and t.

3. Camera model and camera calibration

1. Ideal camera model (linear model)

According to the pinhole camera model, the $P(X_{c},Y_{c},Z_{c})$ projection point of any point on the image plane $p(x,y)$ is the line connecting C (optical center, that is, the center of projection) and $P(X_{c},Y_{c},Z_{c})$ . By proportional relationship, as follows:

Express the above formula in homogeneous coordinate and matrix form (camera coordinate system and image coordinate system), as follows:

Pixel coordinate system and image coordinate system, world coordinate system and camera coordinate system, as follows:

Putting it all together, it looks like this:

The above formula completes the conversion from the world coordinate system (unit is millimeter) to the pixel coordinate system (unit is pixel), and has experienced the transformation from the world coordinate system to the camera coordinate system, from the camera coordinate system to the image coordinate system, and from the image coordinate system to the pixel coordinate system. convert. Among them, $f_{x}=f/d_{x}$ and $f_{y}=f/d_{y}$ . $f_{x},f_{y},c_{x},c_{y}$ M is a 3×4 projection matrix. M1 is completely determined by the internal parameters of the camera , $f_{x},f_{y}$ which respectively represent the scale factors on the u-axis and v-axis of the image, which $(c_{x},c_{y})$ are the coordinates of the principal point. M2 is completely determined by the external parameters of the camera, and camera calibration is to determine The internal and external parameters of the camera. The pixel coordinate system can be obtained from the world coordinate system, but the world coordinate system cannot be obtained from the pixel coordinate system, because the projection matrix M is a 3×4 irreversible matrix. Intuitively, the projection points of all spatial points located on the ray OP are point p.

2. Actual camera model (non-linear model)

Camera calibration is the process of determining the internal or external parameters of the camera. The internal parameters refer to the internal geometric and optical characteristics of the camera, and the external parameters of the camera refer to the translation vector and rotation matrix from the world coordinate system to the camera coordinate system. The actual camera model, as follows:

Among them, m represents the physical coordinates in the image plane coordinate system of the actual projection point, and m' represents the physical coordinates in the image plane coordinate system of the ideal projection point. In the following, only radial distortion and tangential distortion are considered, and eccentric distortion and thin prism distortion are not considered.

(1) Radial distortion

Radial distortion occurs because light rays are more curved away from the center of the lens than closer to the center. As follows:

(2) Tangential distortion

Tangential distortion is caused by defects in camera manufacturing that make the lens itself non-parallel to the imaging plane. As follows:

Among them, $k_{1},k_{2},k_{3}$ is the radial distortion coefficient, $p_{1},p_{2}$ is the tangential distortion coefficient, $r$ is the radial radius, $r^{2}=x^{2}+y^{2}$ .

4. Implementation of OpenCV monocular camera calibration [3]

references:

[1] The principle of monocular vision calibration: http://www.cnblogs.com/polly333/p/4994408.html

[2] Detailed camera model: http://blog.csdn.net/lsh_2013/article/details/47615309

[3] OpenCV monocular camera calibration: http://blog.csdn.net/t247555529/article/details/47836233