1. Definition and function of camera calibration
Camera calibration refers to the calculation of internal parameters, external parameters and lens distortion parameters of a single or multiple cameras with the help of calibration boards.
effect:
Restore the distorted image to a normal image, laying the foundation for subsequent stitching, SLAM, etc.
Multi-camera calibration can transform all camera outputs to the same coordinate system.
Camera calibration is an essential step in 3D vision applications, and is widely used in robotic arms, robot positioning and mapping, self-driving car/smartphone calibration, image stitching, and 3D reconstruction.
2. Why camera calibration is needed
The real world is three-dimensional, while the image captured by the camera is two-dimensional, in which distance/depth information is bound to be lost. Abstracting into a mathematical problem is that the camera is similar to a mapping function, which maps the input three-dimensional scene into a two-dimensional image (maybe a grayscale image or a color image).
Camera calibration is the process of using mathematical models and mathematical methods to approximate this complex mapping function. The calibrated camera has the ability to describe this process, so it can be used for various computer vision tasks, such as depth restoration, three-dimensional reconstruction, etc., which are essentially the recovery of lost distance information.
3. What exactly is camera calibration?
In practical applications, camera calibration can be divided into single-eye camera calibration and multi-eye (binocular) camera calibration
3.1 Monocular camera calibration
Such applications usually require the imaging model parameters of the camera itself and the relative pose of the camera relative to a certain coordinate system.
(1) For example, in the simple scene shown in the figure below, if you know the orientation of the camera and the height from the ground, you can roughly estimate the distance to the object corresponding to a certain pixel of the camera (actually, you can measure the height of the object or the object is close to the ground). This model is more common in some simple vehicle tasks, such as estimating the distance of the vehicle ahead.
(2) Another example is the relatively common robot control in the industry, which needs to establish the relative positional relationship between the robot coordinate system and its visual coordinate system (that is, hand-eye calibration)
(3) Another example is monocular depth recovery. Some traditional monocular depth recovery usually use fixed-size objects as markers, and then move the object to judge the actual shooting distance by the size of the object in the camera.
(4) Another example is some SLAM methods. At present, some self-supervised deep learning methods also use relative posture information as supervision (such as ManyDepth)
3.2 Multi-eye (binocular) camera calibration
This type of application is more common, and usually needs to obtain the information of the camera itself, the relative pose information between each camera, and sometimes the relationship between it and a fixed coordinate system.
(1) For example, in a vehicle-mounted panoramic view, it is necessary to obtain the relative poses of the four cameras and their relationship with the ground coordinate system in order to obtain the final panoramic stitching result.
(2) Another example is AR applications, or more complex camera arrays.
