opencv image distortion correction: source code learning

Reference materials: Camera calibration (4) Correction of distortion undistort() and initUndistortRectifyMap()

Background: opencv provides code for direct distortion correction. Since distortion correction is needed in the project, study the relevant interfaces and codes of distortion correction in opencv to facilitate learning, improvement and secondary development.

1. Principle of distortion correction

Opencv has a brief introduction to the principles of camera calibration and distortion correction in the document. You can refer to the link: Opencv's calibration and 3D reconstruction module document link . In addition, there is also a lot of content on the Internet that explains the principle of distortion correction, so this article is here No detailed introduction.

 2. Distortion correction interface provided by opencv

opencv provides a correction algorithm that can be used directly, that is,  to obtain the intrinsic distortion coefficient through cv::calibrateCamera , and then use the cv::undistort  function to complete the distortion correction of the image in one go; it can also be done through the combination of initUndistortRectifyMap() and remap() deal with.

1）initUndistortRectifyMap()

void cv::initUndistortRectifyMap	(	InputArray 	cameraMatrix,
                                        InputArray 	distCoeffs,
                                        InputArray 	R,
                                        InputArray 	newCameraMatrix,
                                        Size 	size,
                                        int 	m1type,
                                        OutputArray 	map1,
                                        OutputArray 	map2 
                                    )	

• cameraMatrix: Camera matrix, containing the internal parameters of the camera (focal length, optical center, etc.).
• distCoeffs: Distortion coefficient, used to describe the distortion of the camera (radial distortion and tangential distortion).
• R: Correction transformation matrix, used to correct the input image to the corrected output image.
• newCameraMatrix: The corrected camera matrix, which can be customized or generated using the correction transformation matrix.
• size: The size of the output image.
• m1type: The data type of the output mapping table map1.
• map1and map2: output mapping table, which stores the pixel coordinate mapping relationship from the input image to the output image.

Function description: The function of the function is to calculate a mapping table based on the camera's internal parameters, distortion coefficient, correction transformation matrix and output image size, which contains the output image position corresponding to each input image pixel. Through this mapping table, the pixels in the input image can be reconstructed into the corrected output image according to the corrected mapping relationship, thereby realizing the correction of camera distortion.

2）remap()

cv::remap is a remapping function commonly used to correct calibrated stereo images. Reference: OpenCV image processing - remapping remap function

3）cv::undistort

void cv::undistort	(	InputArray 	src,
                    OutputArray 	dst,
                    InputArray 	cameraMatrix,
                    InputArray 	distCoeffs,
                    InputArray 	newCameraMatrix = noArray() 
                    )	

• src: Input image, which can be a single-channel or multi-channel image.
• dst: Output image, the corrected image will be stored here.
• cameraMatrix: Camera matrix, containing the internal parameters of the camera (focal length, optical center, etc.).
• distCoeffs: Distortion coefficient, used to describe the distortion of the camera (radial distortion and tangential distortion).
• newCameraMatrix: The corrected camera matrix, which can be customized or use the default value noArray(). If a new camera matrix is ​​specified, the function will correct the image according to the new camera matrix; otherwise, the original camera matrix will be used for correction.

Function description: The function of the function is to directly perform camera distortion correction on the input image based on the provided camera parameters.

4）cv::undistortPoints
 

The above introduction is to correct the distortion of the entire image, which is actually relatively slow. What I actually encountered in the project was the distortion correction of a few pixels. After searching, I found this function.

void cv::undistortPoints	(	InputArray 	src,
                            OutputArray 	dst,
                            InputArray 	cameraMatrix,
                            InputArray 	distCoeffs,
                            InputArray 	R = noArray(),
                            InputArray 	P = noArray() 
                            )	

• src: The input pixel coordinates can be a N x 1or 1 x Nmatrix, representing Npixels.
• dst: The output corrected pixel coordinates. The corrected pixels will be stored here and have the same dimensions and type as the input matrix.
• cameraMatrix: Camera matrix, containing the internal parameters of the camera (focal length, optical center, etc.).
• distCoeffs: Distortion coefficient, used to describe the distortion of the camera (radial distortion and tangential distortion).
• R: Rotation matrix, optional parameter, default value is noArray(). If a rotation matrix is ​​specified, the function will correct the pixels according to the rotation matrix; otherwise, the rotation transformation will not be considered.
• P: Projection matrix, optional parameter, default value is noArray(). If a projection matrix is ​​specified, the function will correct the pixels according to the projection matrix; otherwise, the projection transformation will not be considered.

3. Source code learning

I'll take a rest now and continue writing tomorrow.