Hand-eye calibration + CMake writing and sharing

It is now 2022.8.4, Tanabata, take time to update the blog and share what you have learned recently.

Hand-eye calibration is a matrix for determining the rotation-translation relationship between the camera and the end of the robotic arm. The specific formula can be found in the principles of other blogs. After understanding the principle, here is an introduction to programming, emm, and personal measurement data can reach millimeter-level accuracy. This blog only discusses the hand-eye calibration procedure, not ros.

1. Hand-eye calibration procedure

(1) Environment

My system environment is ubuntu18.04, opencv4.5.0, using Zhang Zhengyou calibration board, the camera resolution is 640*400, the depth value is 30mm+, the picture is as follows.

 (2) Ideas

Use the Zhang Zhengyou calibration method to calculate the internal and external parameter matrix of the camera (or use the camera parameters broadcast by the robot, this method is more accurate), calculate the translation and rotation matrix of each picture through the PNP method, and finally calculate the internal and external parameter matrix through the calibrateHandEye function.

(3) Code

//张正友相机标定，角点检测    
for(int i=1;i<=images_number;i++)
    {
     Mat image=imread(images_path+to_string(i)+String("/color.jpg"));
     Mat output=image.clone();
     image_size.width=image.cols;
     image_size.height=image.rows;
     if(0==findChessboardCorners(image,board_size,image_points_buf))
     {
        cout<<i<<"   can not find chessboard"<<endl;
        exit(1);
     }
     else
     {
        Mat view_gray;
        cvtColor(image,view_gray,COLOR_BGR2GRAY);
        find4QuadCornerSubpix(view_gray,image_points_buf,Size(5,5));
        image_points_seq.push_back(image_points_buf);
        drawChessboardCorners(output,board_size,image_points_buf,false);
     }
    }
    Size square_size = Size(25, 25);  
    vector<vector<Point3f>> object_points;
    Mat cameraMatrix = Mat(3, 3, CV_64FC1);
    vector<int> point_counts;
    Mat distCoeffs = Mat(1, 5, CV_64FC1); 
    vector<Mat> tvecsMat;
    vector<Mat> rvecsMat;
   for (int t = 0; t < images_number; t++)
   {
       vector<Point3f> tempPointSet;
       for (int i = 0; i < board_size.height; i++)
       {
           for (int j = 0; j < board_size.width; j++)
           {
               Point3f realPoint;
               realPoint.x = i * square_size.width;
               realPoint.y = j * square_size.height;
               realPoint.z = 0;
               tempPointSet.push_back(realPoint);
           }
       }
       object_points.push_back(tempPointSet);
   }
    calibrateCamera(object_points, image_points_seq, image_size, cameraMatrix, distCoeffs, rvecsMat, tvecsMat, 0);
//pnp
    for(int i=1;i<=images_number;i++)
    {
        Mat rvec,tvec;
        rvec=rvecsMat[i-1];
        tvec=tvecsMat[i-1];
        solvePnP(object_points[i-1],image_points_seq[i-1],cameraMatrix,distCoeffs,rvec,tvec,true);
        Rodrigues(rvec,rvec);
        R_ojb2cam.push_back(rvec);
        T_obj2cam.push_back(tvec/1000);
        Homo_target2cam.push_back(R_T2HomogeneousMatrix(rvec,tvec));
    }
//标定
calibrateHandEye(R_gribber2base,T_gribber2base,R_ojb2cam,T_obj2cam,R_cam2gripper,T_cam2gripper,CALIB_HAND_EYE_PARK);

The above code is the realization of Zhang Zhengyou's calibration method and pnp. The TR matrix of pnp uses the initial value of Zhang Zhengyou's calibration method to speed up iteration and improve accuracy.

 2. Writing of CMakeList.txt

Although the cmakelist file is automatically generated in ros, it is still necessary to understand the necessary things when faced with complexity.

(1) First write the minimum version of cmake

cmake_minimum_required（）

(2) Project name

project(calibrate_plate)

(3) The third-party library used, such as opencv

find_package(OpenCV REQUIRED)

include_directories(${OpenCV_INCLUDE_DIRS}）

(4) Generate exe and add link library

add_executable(hand_eye_board src/calibrate_plate_handeye.cpp)
target_link_libraries(hand_eye_board ${OpenCV_LIBS} )

Then execute the cmake command to compile.

Ending: I have been working recently, and I have been working overtime late, and I have not responded to many messages, so everyone should think for themselves.