1. Principle of camera calibration
1.1 How the camera produces images:
The camera imaging system contains a total of four coordinate systems: world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system.
1.1.1 World coordinate system:
World coordinate system (world coordinate), also called measurement coordinate system, is a three-dimensional rectangular coordinate system, based on which the spatial position of the camera and the object to be measured can be described. The position of the world coordinate system can be freely determined according to the actual situation.
1.1.2 Camera coordinate system:
The camera coordinate system is also a three-dimensional rectangular coordinate system. The origin is located at the optical center of the lens. The x and y axes are parallel to both sides of the phase plane respectively. The z axis is the optical axis of the lens and is aligned with the image. Plane vertical
1.1.3 Pixel coordinate system, image coordinate system:
The pixel coordinate system (pixel coordinate)
is as shown in the figure above. The pixel coordinate system is a two-dimensional rectangular coordinate system that reflects the arrangement of pixels in the camera CCD/CMOS chip. The origin is located at the upper left corner of the image, and the axes and axes are parallel to both sides of the image plane. The unit of the coordinate axis in the pixel coordinate system is pixels (integers).
The pixel coordinate system is not conducive to coordinate transformation, so an image coordinate system needs to be established. The unit of its coordinate axis is usually millimeters (mm). The origin is the intersection of the camera optical axis and the phase plane (called the principal point), which is the center point of the image. axes and axes are parallel to axes and axes respectively. Therefore, the two coordinate systems are actually in a translational relationship, which can be obtained through translation.
1.2 The purpose of camera calibration:
to find the internal and external parameters of the camera, as well as the distortion parameters.
After calibrating a camera, you usually want to do two things: one is that since the degree of distortion of each lens is different, camera calibration can correct the lens distortion and generate a corrected image; the other is to reconstruct the image based on the obtained image. Construct a three-dimensional scene.
1.2 The overall principle of camera calibration:
Camera calibration (Camera calibration) is simply the process of changing from the world coordinate system to the image coordinate system, which is the process of finding the final projection matrix.
2. Camera calibration steps
1) Prepare a checkerboard for Zhang Zhengyou’s calibration method. The size of the checkerboard is known, and use a camera to shoot it from different angles to obtain a set of images;
2) Detect the feature points in the image such as the corner points of the calibration board to obtain the pixel coordinates of the corner points of the calibration board. Based on the known checkerboard size and the origin of the world coordinate system, calculate the physical coordinates of the corner points of the calibration board. ;
3) Solve the internal parameter matrix and external parameter matrix.
According to the relationship between physical coordinate values and pixel coordinate values, calculate the H matrix, then construct the V matrix, solve the B matrix, use the B matrix to solve the camera internal parameter matrix A, and finally solve the camera external parameter matrix corresponding to each picture:
4) Solve for the distortion parameters.
Construct the D matrix and calculate the radial distortion parameters;
5) Use LM (Levenberg-Marquardt) algorithm to optimize the above parameters
3. Implement
3.1 Data Preparation
Take photos of the calibration plate at different angles and get 10 pictures. Get a picture similar to the following.
3.2 Code (Python+OpenCV)
import cv2
import numpy as np
import glob
# 设置寻找亚像素角点的参数，采用的停止准则是最大循环次数30和最大误差容限0.001
criteria = (cv2.TERM_CRITERIA_MAX_ITER | cv2.TERM_CRITERIA_EPS, 30, 0.001)
# 获取标定板角点的位置
objp = np.zeros((4 * 4, 3), np.float32)
objp[:, :2] = np.mgrid[0:4, 0:4].T.reshape(-1, 2) # 将世界坐标系建在标定板上，所有点的Z坐标全部为0，所以只需要赋值x和y
obj_points = [] # 存储3D点
img_points = [] # 存储2D点
images = glob.glob(r"D:\software\pycharm\PycharmProjects\computer-version\biaoding\images\*.jpg")
i = 0
for fname in images:
img = cv2.imread(fname)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
size = gray.shape[::-1]
ret, corners = cv2.findChessboardCorners(gray, (4, 4), None)
# print(corners)
if ret:
obj_points.append(objp)
corners2 = cv2.cornerSubPix(gray, corners, (5, 5), (-1, -1), criteria) # 在原角点的基础上寻找亚像素角点
# print(corners2)
if [corners2]:
img_points.append(corners2)
else:
img_points.append(corners)
cv2.drawChessboardCorners(img, (4, 4), corners, ret) # 记住，OpenCV的绘制函数一般无返回值
i += 1
cv2.imwrite('conimg' + str(i) + '.jpg', img)
cv2.waitKey(1500)
print(len(img_points))
cv2.destroyAllWindows()
# 标定
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(obj_points, img_points, size, None, None)
print("内参数矩阵:\n", mtx) # 内参数矩阵
print("畸变系数:\n", dist) # 畸变系数 distortion cofficients = (k_1,k_2,p_1,p_2,k_3)
print("旋转向量:\n", rvecs) # 旋转向量 # 外参数
print("平移向量:\n", tvecs) # 平移向量 # 外参数
print("-----------------------------------------------------")
img = cv2.imread(images[2])
h, w = img.shape[:2]
# 获取新的相机矩阵和ROI
newcameramtx, roi = cv2.getOptimalNewCameraMatrix(mtx, dist, (w, h), 1, (w, h))
# 进行畸变校正
dst = cv2.undistort(img, mtx, dist, None, newcameramtx)
# 裁剪校正后的图片，去掉黑边
x, y, w, h = roi
dst = dst[y:y + h, x:x + w]
# 保存校正后的图片
cv2.imwrite('undistorted.jpg', dst)
print("校正后的图片已保存到文件 'undistorted.jpg'")The result is as follows:
D:\software\anaconda3\envs\homework\python.exe D:\software\pycharm\PycharmProjects\computer-version\biaoding\biaoding.py
10
内参数矩阵:
[[2.96560599e+04 0.00000000e+00 1.14649373e+03]
[0.00000000e+00 2.27193976e+04 2.23086118e+03]
[0.00000000e+00 0.00000000e+00 1.00000000e+00]]
畸变系数:
[[-2.89172774e+00 -7.08266723e+02 -8.71069820e-02 9.23282091e-03
2.69077473e+04]]
旋转向量:
(array([[ 0.00393121],
[-0.55572922],
[ 0.00146949]]), array([[-0.24713914],
[-0.62161368],
[ 1.12164248]]), array([[0.03655035],
[0.72426273],
[0.58207192]]), array([[ 0.03267646],
[-0.81493845],
[ 0.33111165]]), array([[ 0.10136185],
[-0.8188412 ],
[ 0.0126507 ]]), array([[ 0.07815116],
[-0.45667052],
[ 0.04574239]]), array([[-0.06538785],
[-0.45146705],
[ 0.29559633]]), array([[0.19950015],
[0.59439631],
[1.05909245]]), array([[-0.61847323],
[-0.65382223],
[ 1.44780297]]), array([[-0.56075885],
[-0.65910126],
[ 1.21835979]]))
平移向量:
(array([[-1.88349377],
[-2.34826569],
[79.07381781]]), array([[-3.71964945e-03],
[-2.09696872e+00],
[ 9.57830776e+01]]), array([[ 2.82236969],
[-10.16609008],
[105.50710577]]), array([[ 2.4498244 ],
[ -9.91592803],
[113.69245822]]), array([[ 1.94032193],
[ -9.41660327],
[118.34213194]]), array([[ 3.29323168],
[-10.37799624],
[150.0005035 ]]), array([[ 1.90004234],
[ -9.89181093],
[136.81120801]]), array([[ 3.81109485],
[ -9.80118203],
[131.55206487]]), array([[ 0.12140321],
[ -3.0523607 ],
[114.3591686 ]]), array([[ 0.17173674],
[-2.75540486],
[80.26443015]]))
-----------------------------------------------------
校正后的图片已保存到文件 'undistorted.jpg'
Process finished with exit code 0
Image before distortion correction:
Image after distortion correction:
analyze:
After running the program, I successfully calibrated the parameters of my mobile phone and calculated the internal and external parameters of the camera. Through this experiment, I further understood the wonders of Zhang Zhengyou’s calibration method.