Reprint link:
Robot hand-eye calibration based on OpenCv (9-point calibration method)
For beginners, the calibration of the camera is often blurred. I don’t know how to transform the machine coordinates and camera coordinates, but how are the two coordinate systems established?
We usually use Zhang's calibration method to calibrate the distortion of the camera, and use the corrected parameters to process the graphics before presenting it. Many people use this method online, and the information is more comprehensive. It will not be explained here. This article is mainly aimed at the hand-eye calibration of the manipulator. Generally speaking, the distortion of the current camera is relatively small and the accuracy is relatively accurate. Using this method for calibration can also get better results.
First, for the nine-point calibration. What we are using is the estimateRigidTransform function in OpenCv .
The function is defined as follows:
Mat estimateRigidTransform(InputArraysrc, InputArraydst, boolfullAffine) The
first two parameters can be: src=srcImage (picture Mat before transformation) dst=transImage (picture Mat after transformation)
or: src=array (before transformation Array of key points) dst=array (Array of key points after transformation) The
third parameter: 1 (full affine transformation, including: rotation, translation, scaling, shearing, reflection)
The main principle is: if we have a point before the transformation is [x,y,1], after the transformation is [x',y',1], then fullAffine is expressed as follows:
TX=Y
After expansion
If we want to find these [af] 6 variables, we need 6 equations, that is, 3 sets of points. But more than three points?
For example: 20 points. That is to use the smallest variance. 
Calibration steps
1. First, we need to prepare a calibration board. If the conditions are insufficient, nine circles can be drawn on white paper instead. 
2. The camera position and the position of the manipulator are all fixed, and the calibration needle is fixed on the manipulator. Once fixed, it cannot be moved. The position of the calibration needle must be the same height as the tool in the gripper or suction cup.
3. Place the calibration board under the camera. The location area should be the same as the working area of the manipulator, including the height must be as consistent as possible. This is the key to the accuracy of the calibration.
4. Adjust the focus of the camera, take a photo, and then identify and record the coordinates of the 9 circle centers. For how to find a circle, you can refer to my previous blog post, which is very accurate for finding a circle.
5. Move the manipulator to the center of the 9 circles in turn, and write down the coordinates of the manipulator
After completing the above five steps, we will get two point sets. One is the coordinates of the 9 circle center (points_camera), and the other is the coordinates of the manipulator (points_robot) corresponding to the 9 circle centers.
Mat warpMat;
vector<Point2f>points_camera;
vector<Point2f>points_robot;`
vector<Point2f>points_camera;
vector<Point2f>points_robot;
warpMat =estimateRigidTransform(points_camera, points_robot, true);
A = warpMat.ptr<double>(0)[0];
B = warpMat.ptr<double>(0)[1];
C = warpMat.ptr<double>(0)[2];
D = warpMat.ptr<double>(1)[0];
E = warpMat.ptr<double>(1)[1];
F = warpMat.ptr<double>(1)[2];
The 6 double type parameters obtained are the calibration parameters that we finally get in this calibration.
Then we substitute the detected image coordinates (t_px, t_py) into it, and then we can get the corresponding robot coordinates (t_rx, t_ry)
t_rx = (A * t_px) + B * t_py + C);
t_ry = (D * t_px) + E * t_py+ F);
At this point, the calibration is over, we can control the camera to take a picture for positioning, and then convert it into the robot coordinates, which means to hit which. Hope to be helpful to beginners, please indicate the source for reprinting.