I hope that 2022 will be able to study independently.
This article borrows heavily from: Camera Calibration Zhang Zhengyou Calibration Mathematical Principles Detailed Explanation (including python source code) - Zhihu and the most detailed and complete camera calibration explanation_a083614's column-CSDN blog_Camera calibration , thank you very much, just For self-study, not for other use.
After we know how the camera images, we need to calibrate it. Why do we need to calibrate the camera? For example, when we get a picture and recognize it, how many pixels are the distance between the two parts, but how many meters do these pixels correspond to in the actual world? This requires the use of camera calibration results to convert pixel coordinates to physical coordinates to calculate distances (of course, it is worth explaining here that only using the results of monocular camera calibration cannot directly convert from pixel coordinates to physical coordinates, because perspective projection is lost. coordinates in one dimension, so the distance measurement actually requires a binocular camera). Therefore, the first purpose of camera calibration is to obtain the camera's internal reference matrix and external reference matrix.
There is still some distortion in the pictures taken by the camera, so let's take a look at the distortion first.
1. Distortion
Distortion is an inherent characteristic of the camera itself, which is the same as the camera's internal parameters, and it can be calibrated once.
1.1 Radial distortion (barrel distortion and pincushion distortion)
Radial distortion comes from the shape of the lens. The lens of an actual camera always produces significant distortion at the edge of the imager. This phenomenon comes from the effect of "barrel" or "fisheye".
As shown in the figure below, light rays are more curved at the center of the principle lens than near the center. For commonly used ordinary lenses, this phenomenon is more serious. Barrel distortion is very severe in cheap webcams, but not noticeable in high-end cameras, because these lens systems do a lot of work to remove radial distortion.
For radial distortion, the distortion is zero at the center of the imager (optical center) and becomes more and more severe as one moves towards the edge.
The radial distortion can be corrected by the following Taylor series expansion:
Here, x and y are the distorted position coordinates in the image, and the real coordinates are obtained through correction. r is the distance of the point from the imaging center.
1.2 Tangential distortion (thin lens distortion and centrifugal distortion)
The tangential distortion comes from the entire camera assembly process. The tangential distortion is caused by defects in lens manufacturing that make the lens itself not parallel to the image plane. This phenomenon occurs when the imager is glued to the camera.
Tangential distortion can be corrected by the following formula:
Among them, (x, y) is the ideal undistorted normalized image coordinate, (x tip, y tip) is the normalized image coordinate after distortion, and is the distance from the image pixel point to the image center point, that is =
+
.
The second purpose of camera calibration is to obtain the distortion parameters of the camera, such as k1, k2, k3, p1, p2, etc. in the above formula, and then perform de-distortion processing on the captured pictures.
2. Calibration method
Introduction to Zhang Zhengyou's calibration (Zhang Zhengyou's calibration only considers radial distortion, not tangential distortion)
Zhang Zhengyou’s calibration method uses the checkerboard calibration board as shown in the figure below. After obtaining an image of the calibration board, the corresponding image detection algorithm can be used to obtain the pixel coordinates (u, v) of each corner point.
Zhang Zhengyou’s calibration method fixes the world coordinate system on the checkerboard, so the physical coordinates of any point on the checkerboard are W=0. Since the world coordinate system of the calibration board is artificially defined in advance, the size of each grid on the calibration board is known. Yes, we can calculate the physical coordinates of each corner point in the world coordinate system.
We will use this information: the pixel coordinates (u, v) of each corner point, and the physical coordinates of each corner point in the world coordinate system to calibrate the camera and obtain the camera's internal and external parameter matrix and distortion parameters.
Camera calibration steps:
1. Print a checkerboard, the size of the checkerboard is known, and paste it on a plane as a calibration object.
2. By adjusting the direction of the calibration object or the camera, take some photos of the calibration object in different directions.
3. Extract the corner points of the checkerboard from the photo for detection, and obtain the pixel coordinates of the corner points of the calibration board. According to the known size of the checkerboard and the origin of the world coordinate system, calculate the physical coordinates of the corner points of the calibration board; 4.
Estimation In an ideal undistorted situation, five internal references and six external references.
5. Apply the least square method to estimate the distortion coefficient under the actual radial distortion.
6. Maximum likelihood method, optimized estimation, and improved estimation accuracy.
3. Source code
The source code of the detailed explanation of the mathematical principles of Zhang Zhengyou's calibration method (including python source code) for camera calibration: Click here
After execution, the internal and external parameters and distortion parameters of the camera can be obtained, and the distorted picture can be corrected.
distortion:
After output:
okk This is an article written a month ago, add a little more, the recent epidemic has been blocked again, and nothing has been done.