In multi-sensors, we need to calibrate the equipment before use. First, we calibrate a single device (later use requires joint calibration)
Joint calibration tools such as Apollo, Autoware and other automatic labeling tools, as well as some open source methods, etc., can refer to some classic papers (see the link below for the catalog).
imu camera calibration_multi-sensor fusion calibration method summary_Gamer42's Blog-CSDN Blog Click on the "New Machine Vision" above, choose to add "Star" or "Top" heavy dry goods, and the author will be delivered to the author丨Tom Hardy@知道Almost source丨https://zhuanlan.zhihu.com/p/268908000Edited丨3D Vision Workshop This article summarizes some common single-sensor and multi-sensor calibration fusion papers and engineering codes, which are applied to automatic driving, 3D vision, and SLAM And other fields, for your reference~1. Camera calibration Online Calibration of Ext... https://blog.csdn.net/weixin_31831459/article/details/112066574 First of all, we calibrate the camera. Most of the camera calibration methods are Based on the "Zhang's" calibration method, it adopts the checkerboard method. The main calibration method is to use the ros calibration tool, you can refer to the following (see the second link for the operation method and calibration board download):
camera_calibration - ROS Wiki 
https://wiki.ros.org/camera_calibration camera_calibration/Tutorials/MonocularCalibration - ROS Wiki https://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration calibration board is printed with A0 (ROS official cannot Put it on A0 paper and need to cut), here is a cut calibration board for printing, link:
Extraction code of the trimmed calibration board (108) : g14u
Of course, everyone designs the calibration board according to the needs of their own cameras (the size of the focal length) (the calibration board pdf of the required specifications can be generated below):
Camera Calibration Pattern Generator – calib.io https://calib.io/pages/camera-calibration-pattern-generator here introduces a method of using MATLAB to calibrate the camera (version R2021b)
Open MATLAB APP->Image Processing and Computer Vision->Camera Calibration
We only need to throw the collected pictures into this tool. The pictures can be collected by using the method of moving the calibration board with a fixed camera (recommended) or moving the camera without moving the calibration board ( 20~30 pictures are enough ).
Add the picture you collected (the picture is a test, not the calibration board of the specification mentioned above)
MATLAB will automatically remove images that do not meet the requirements, and the next step is to select the camera type and output.
Ordinary camera is in 1 place, keep the basic settings, keep it as standard, choose fish eye for fisheye camera; click option to choose radial distortion, generally choose two parameters, two or three represent the number of de-distortion parameters (k1, k2, whether there is k3), we don’t choose skew under normal circumstances. After setting, click calibration to generate, and the blue picture on the right side of the above picture will be generated. Let’s drag the straight line of outlier within 0.8 (or according to the demand , if the number of photos is large and the number of changes can be adjusted according to needs), the outlier image will be automatically selected on the left, right-click to delete, and it will automatically run again. At this time, click the green Export camera calibration above to complete the calibration; click on the upper left corner The save session will generate a .mat file to save your output.
Our calibration results will be displayed (output) in the command line window. It is worth noting that the camera intrinsics are the internal parameters of the camera. In this version, it is not displayed as a 3*3 matrix, but as a 1*1 cameraIntrinsics . And the content of this parameter is more abundant.
We can use pylon_01.xx (the parameter name you output. Project) to call in the command window, or double-click the parameter directly on the right to view the result:
Its top-to-bottom content is: image size, radial distortion parameter (corresponding to two or three parameters), tangential or distortion parameter (two parameters), world coordinate point, world unit, skew, radial distortion parameter The number, whether to calculate tangential distortion (bool), ..., internal parameters, focal length, cx, cy (Principepoint), etc.
For other calling methods and understanding, please refer to the official website of mathworks:
Camera projection matrix - MATLAB cameraMatrix- MathWorks 中国
Where cameraIntrinsics includes:
After getting this result, we need to write the configuration file:
according to:
camera_calibration_parsers - ROS Wiki https://wiki.ros.org/camera_calibration_parsers#File_formats Need to write .ini file or .yaml file format:
This is the INI format used by Videre cameras to store calibration parameters internally. We are using it as our preferred format for storing calibration on camera flash memory (not only Videre cameras). 下面是INI格式
example:
# Basler camera intrinsics
[image]
width
1920
height
1200
[Basler_01]
camera matrix
2780.34662 0.00000 941.549872
0.00000 2776.46900 651.074403
0.00000 0.00000 1.00000
distortion
-0.15400 1.70770 -11.77580 0.00620 -0.00010
rectification
1.00000 0.00000 0.00000
0.00000 1.00000 0.00000
0.00000 0.00000 1.00000
projection
2780.34662 0.00000 941.549872 0.00000
0.00000 2776.46900 651.074403 0.00000
0.00000 0.00000 1.00000 0.00000The YAML output is based on that used by the OpenCV calibration program.
example:
image_width: 1920
image_height: 1200
camera_name: Basler
camera_matrix:
rows: 3
cols: 3
data: [2780.34662, 0.00000, 941.549872,0.00000, 2776.46900, 651.074403, 0.00000, 0.00000 ,1.00000]
distortion_model: plumb_bob
distortion_coefficients:
rows: 1
cols: 5
data: [-0.15400, 1.70770, -11.77580, 0.00620, -0.00010]
rectification_matrix:
rows: 3
cols: 3
data: [1, 0, 0, 0, 1, 0, 0, 0, 1]
projection_matrix:
rows: 3
cols: 4
data: [2780.34662, 0.00000, 941.549872, 0.00000, 0.00000, 2776.46900, 651.074403, 0.00000, 0.00000, 0.00000, 1.00000, 0.00000]The two formats "Basler" are the name of the camera and can be adjusted according to your name; it is worth mentioning that the last item of the projection matrix can be directly added a column of 0 according to the internal parameter matrix of the camera .
Save the above text as a .ini or .yaml file. Let's import parameters for the camera (usually in the camera's configuration file) or manually import them in the code, such as directly given. (The text will be automatically given by ros tool calibration)
For Basler cameras, in config, add internal parameters as follows:
Pay attention to the format, url here is the "global path of the ini file", see:
As shown in the figure, it is calibrated:
Output image in Rviz or image_view (bottom left):
Finish. In the future, we will continue to write the actual calibration of other sensors!