UR5 robot hand-eye calibration

0. Preparation

1. Camera driver installation: Portal
2. Ur5 robot driver installation: Portal
Hardware conditions

UR5 robot	Realsense D435i
industrial computer	i7-11700F RTX 3060
system	Ubuntu 18.04

1. Install some calibration folders

mkdir -p ~/hand_eye/src
cd ~/hand_eye/src
#下载aruco_ros
git clone -b melodic-devel https://github.com/pal-robotics/aruco_ros.git
#下载easy_handeye
git clone https://github.com/IFL-CAMP/easy_handeye

#下载vision_visp / visp_hand2eye_calibration
sudo apt-get install ros-melodic-visp
git clone -b melodic-devel https://github.com/lagadic/vision_visp.git
cd ..
catkin_make --pkg visp_hand2eye_calibration
#编译
catkin_make

The above code may not be downloaded, you need to add an agent on the terminal

2. The file configuration of the eye in the hand
In the process of calibration, it is necessary to start the node of the Ur5 robotic arm, the realsense node, the aruco node, and the easy_handeye node. Most of them on the Internet write a launch file to start together, but it is easy to report an error. Here, they are written separately.
2.1 Modify the configuration file

cp ~/hand_eye/src/easy_handeye/docs/example_launch/ur5_kinect_calibration.launch   ~/hand_eye/src/easy_handeye/easy_handeye/launch


sudo gedit ~/hand_eye/src/easy_handeye/easy_handeye/launch/ur5_kinect_calibration.launch

The modified content is as follows

<launch>
    <arg name="namespace_prefix" default="ur5_kinect_handeyecalibration" />
    <arg name="robot_ip" doc="The IP address of the UR5 robot" />
//需要根据自己的实际码来 定义 marker_size（打印码的长度单位米）和marker_id（打印码的编号）
    <arg name="marker_size" doc="Size of the ArUco marker used, in meters" default = "0.1"/>
    <arg name="marker_id" doc="The ID of the ArUco marker used" default = "24"/>

//注释这一部分代码
    <!-- start the Kinect
    <include file="$(find freenect_launch)/launch/freenect.launch" >
        <arg name="depth_registration" value="true" />
   </include> -->
 //注释这一部分代码   

    <!-- start ArUco -->
    <node name="aruco_tracker" pkg="aruco_ros" type="single">
  //注意改变相机的节点
        <remap from="/camera_info" to="/camera/color/camera_info" />
        <remap from="/image" to="/camera/color/image_raw" />
        <param name="image_is_rectified" value="true"/>
        <param name="marker_size"        value="$(arg marker_size)"/>
        <param name="marker_id"          value="$(arg marker_id)"/>
 //注意改变相机的节点
        <param name="reference_frame"    value="camera_color_frame"/>
        <param name="camera_frame"       value="camera_color_frame"/>
        <param name="marker_frame"       value="camera_marker" />
    </node>
//注释机器人的代码
    <!-- start the robot
    <include file="$(find ur_bringup)/launch/ur5_bringup.launch">
        <arg name="limited" value="true" />
        <arg name="robot_ip" value="192.168.0.21" />
    </include>
    <include file="$(find ur5_moveit_config)/launch/ur5_moveit_planning_execution.launch">
        <arg name="limited" value="true" />
    </include> -->
//注释机器人的代码

    <!-- start easy_handeye -->
    <include file="$(find easy_handeye)/launch/calibrate.launch" >
        <arg name="namespace_prefix" value="$(arg namespace_prefix)" />
        //这里眼在手上选择true
        <arg name="eye_on_hand" value="true" />

        <arg name="tracking_base_frame" value="camera_link" />
        <arg name="tracking_marker_frame" value="camera_marker" />
        //robot_base_frame 的坐标系改成base
        <arg name="robot_base_frame" value="base" />
        <arg name="robot_effector_frame" value="tool0_controller" />

        <arg name="freehand_robot_movement" value="false" />
        <arg name="robot_velocity_scaling" value="0.5" />
        <arg name="robot_acceleration_scaling" value="0.2" />
    </include>

</launch>

If you copy the above code, please delete all the Chinese instructions
aruco QR code acquisition address: Portal
Be sure to use the aruco QR code, then print it out and measure the size of the QR code.

Do not print a too large Aruco code here, which will cause the robot to recognize the Aruco code less often

There will be two problems here. When executing the hand-eye calibration node, an error will be reported.

For rosdep error, try to install transform3d by command

pip install transforms3d

For AttributeError, I think it is similar to this problem, you can try this command

python -m pip install opencv-contrib-python

The original github link of the solution: Portal

2. Start the calibration program

1. Realsense camera start

roslaunch realsense2_camera rs_camera.launch

2. Start the robot

roslaunch ur_robot_driver ur5_bringup.launch limited:=true robot_ip:=192.168.125.100

Open another terminal

roslaunch ur5_moveit_config ur5_moveit_planning_execution.launch limited:=true

3. Start the calibration program

roslaunch easy_handeye hand_eye.launch

4. Open another rviz terminal, enter rqt,
click Plugins -> Visulization -> Image View in the menu bar, and select the /aruco_tracker/result topic. If the aruco code does not appear, check whether the camera node above has changed.

insert image description here

5. Adjust the angle of the robot, it is the best to be able to recognize the Aruco code in the best state.

The aruco code is best not to be too large, otherwise it will often go out of the camera's field of view, resulting in a relatively small amount of data collection.

6. Click check starting pose, as shown in the picture, if the check is successful, the interface will appear: 0/17, ready to start
insert image description here

Then click Next Pose -> Plan -> Execute. After the execution is completed, if the calibration code is within the field of view, click Take Sample as shown in the figure.
insert image description here
Repeat the cycle until 17 poses are executed, and click Compute in the window to calculate the robot's hand-eye coordinate relationship.
Click Save, the data is saved in the yaml file in the $HOME/.ros/easy_handeye directory.

insert image description here