1. opencv monocular camera calibration process
- According to the size of the calibration plate, determine the coordinates of the calibration plate in the real world
- Take pictures of the calibration board from different angles
- Use opencv's findChessboardCorners to calculate the corner coordinates of each calibration board
- Use the calibrateCamera interface of opencv to calibrate the camera parameters according to the 3D coordinates and pixel coordinates of the corner points
2. Define the coordinates of the calibration plate in the real world
There are two ways to shoot the calibration board:
- Fix the camera and move the calibration plate to achieve shooting from different angles
- Fix the calibration board, move the camera, and shoot the calibration board from different angles
Here we assume to use the second method, fix the calibration board, fix the world coordinate system, select any corner point on the calibration board as the origin of the world coordinate system, the x and y axes are along the plane of the calibration board, and the z axis is perpendicular to the plane of the calibration board. Since the calibration plate is always on the xy plane, z=0 for all points on the calibration plate plane.
Next, you need to shoot the calibration board from different angles. When shooting the calibration board, you need to pay attention to the following:
- The distance between the calibration plate and the camera should ensure that at least 20% of the imaging plane is covered by the calibration plate
- Take at least 10~20 pictures
- Try to use uncompressed image formats, such as PNG, to obtain higher precision in one pass
- Shoot the calibration plate in different directions relative to the camera, and the deflection angle of the calibration plate relative to the camera plane should be less than 45° as much as possible
- Do not modify the picture, such as cropping the picture taken
- During the shooting process, do not adopt the method of auto focus, and do not change the focal length during the process
- The calibration plate pattern taken occupies as much of the frame as possible, and the lens distortion will increase radially with the distance from the center, which is not consistent across the frame. In order to capture lens distortion, the pattern must cover the edge of the captured image.
3. Calibration solution
- First find the 2D coordinates of the calibration board, the corresponding API is:
retval, corners = cv2.findChessboardCorners(image, patternSize, flags)
image: A single checkerboard calibration image, must be 8bit grayscale or color image
patternSize: checkerboard grid column, the number of interior corner points per row (points_per_row, points_per_column)
corners: output the detected corner array
flags: defines the way to find corners
cv2.cornerSubPix(image, corners,winsize, zeroZone, criteria)
image: A single checkerboard calibration image, must be 8bit grayscale or color image
corners: Input the initial coordinates of the corner points
winSize: half the length of the search window
criteria: Termination criteria for corner point optimization iterations, one is the maximum number of iterations of corner point optimization cv2.TERM_CRITERIA_MAX_ITER, and the other is that the movement displacement of corner points is less than cv2.TERM_CRITERIA_EPS
- In the second step of camera calibration, the corresponding API is:
retval , cameraMatrix, distCoeffs, rvecs, tvecs = cv2. CalibrateCamera(objectPoints, imagePoints,imageSize)
objectPoints: a vector of three-dimensional corner points
imagePoints: a vector of 2D point coordinates
ImageSize: image resolution
cameraMatrix: camera internal parameter matrix
distCoefs: distortion parameters
rvecs: rotation matrix
tvecs: translation matrix
retval: RMS reprojection error
4. Distortion Correction
After calibration, image distortion correction can be performed, including the following API:
newCameraMatrix, validPixROI = cv2. getOptimalNewCameraMatrix(cameraMatrix,distCoeffs, imageSize, alpha[, newImgSize[, centerPrinciplePoint]])
cameraMatrix: camera internal reference
distCoeffs: input vector of distortion parameters
imageSzie: The resolution of the original image
alpha: Controls the pixel retention method of distortion correction, alpha = 0, indicating that all pixels of the corrected image are valid, alpha = 1, the corrected image retains all pixels of the original image
newCameraMatrix: Based on free scaling parameters, optimized camera internal parameters
validPixROI: The valid pixel region of the corrected image
dist = cv2.undist(src, cameraMatrix, disCoeffs[, newCameraMatrix]), this function is equivalent to the superposition of the following two functions.
src: the original image to be corrected
cameraMatrix: camera internal reference
distCoeffs: vector of distortion parameters
dst: output image after distortion correction
newCameraMatrix: The internal camera reference corresponding to the image to be corrected
map1, map2 = cv2.initUndistortRectifymap(cameraMatrix, distCoeffs, R, newCameraMatrix, size,m1type)
cameraMatrix: camera internal reference
distCoeffs: distortion parameters
R: optional, 3*3 rectified transformation matrix
newcameraMatrix: new camera internal parameters, generally equal to cameraMatrix in monocular
size: corrected image size
map1: the first mapping matrix, mapx(x,y)
map2: the first mapping matrix, mapy(x,y)
dst = cv.remap(src, map1,map2, interpolation[,dst[,borderMode[,borderValue]]])
src: original image
dst: the mapped image, with the same size and data type as src
map1: the first mapping matrix, the x coordinate of src corresponding to dst (x, y)
map2: The second mapping matrix, the y coordinate of src corresponding to dst (x, y)
interpolation: interpolation method, linear interpolation, etc.
borderMode: the processing method of border value
borderValue: borderMode is the daily recharge when the fixed value is filled, the default is 0