Hand-eye calibration refers to solving the coordinate transformation relationship between the end coordinate system of the industrial robot and the camera coordinate system, or the coordinate transformation relationship between the base coordinate system of the industrial robot and the camera coordinate system. There are two cases of hand-eye calibration: the first one is that the camera (eye) is fixed at the end of the robot arm (hand), the camera is fixed relative to the end of the robot arm, and the camera moves with the robot arm. This way of hand-eye calibration is called Eye- in-hand; the second is that the camera (eye) and the robot arm (hand) are separated, the camera is fixed relative to the base of the industrial robot, and the movement of the robot arm has no effect on the camera. This method of hand-eye calibration becomes Eye- to-hand.
1. Eye-on-hand hand-eye calibration
For the Eye-in-hand calibration method, it is necessary to solve the coordinate transformation relationship between the end coordinate system of the industrial robot and the camera coordinate system. The schematic diagram of the principle of Eye-in-hand hand-eye calibration is shown in Figure 1. There are several coordinate systems, the base coordinate system (represented by base) is the base coordinate system of the robot arm, the end coordinate system (represented by end) is the end coordinate system of the robot arm, and the camera coordinate system (represented by cam) is fixed The camera's own coordinate system on the robot arm, and the calibration object coordinate system (indicated by cal) is the coordinate system where the calibration board is located. Move the robotic arm twice arbitrarily, since the calibration plate and the base of the robotic arm are not moving, so for a certain world point, its coordinate values in the base coordinate system and cal coordinate system remain unchanged, and in the end coordinate system and cam coordinate The coordinate values under the system change with the movement of the robot arm. According to this relationship, the transformation matrix between the end coordinate system and the cam coordinate system can be solved. The specific solution process is as follows.
For the solution of the hand-eye calibration matrix, that is, to solve the conversion matrix between the end coordinate system and the cam coordinate system, there are currently many methods, and the basic principles are based on the above constraints. A certain point P 0 P_0 in the cal coordinate systemP0, according to the known camera extrinsics (transformation matrix T 1 T_1T1) can be converted to point P1 in the cam coordinate system, and then can be converted to point P 2 P_2 in the end coordinate system according to the hand-eye calibration matrix (transformation matrix X ) to be obtainedP2, and then according to the known parameters of the robot itself (transformation matrix T 3 T_3T3) can be converted to point P 3 P_3 in the base coordinate systemP3. Therefore, the relationship can be obtained as follows:
T 3 XT 1 P 0 = P 3 (1) T_3XT_1P_0=P_3 \tag{1}T3XT1P0=P3( 1 )
Move the robot arm, for the same point,P 0 P_0P0and P 3 P_3P3The coordinate value remains unchanged, just P 1 P_1P1and P 2 P_2P2The coordinate value of is changed, and the above relationship becomes the following form:
T 3 ′ XT 1 ′ P 0 = P 3 (2) T_3{^\prime}XT_1^{\prime}P_0=P_3 \tag{2}T3′XT1′P0=P3( 2 ) T 3 ′ T_3^\prime
in the above formulaT3′and T 1 ′ T_1^\primeT1′The same is a known parameter for the second measurement. Combining equations (1) and (2), the following relationship can be obtained:
T 3 XT 1 = T 3 ′ XT 1 ′ (3) T_3XT_1=T_3{^\prime}XT_1^{\prime} \tag{3}T3XT1=T3′XT1′( 3 )
Transform the formula (3) to get:
T 3 ′ − 1 T 3 X = XT 1 ′ T 1 − 1 (4) T_3 ^{\prime-1}T_3 X=XT_1^{\prime} T_1^{-1} \tag{4}T3′−1T3X=XT1′T1−1( 4 )
The above formula can be regarded asAX = XB AX=XBAX=XB equation form, and the matrixA = T 3 ′ − 1 T 3 A=T_3 ^{\prime-1}T_3A=T3′−1T3和 B = T 1 ′ T 1 − 1 B=T_1^{\prime} T_1^{-1} B=T1′T1−1It's all known. Hand-eye calibration equation AX = XB AX=XBAX=Solving XB , you can get the hand-eye transformation matrixXXthe value of X.
2. Hand-eye calibration with eyes outside hands
The schematic diagram of eye-to-hand hand-eye calibration is shown in Figure 2 below. The camera is fixed outside the robot arm, and the base of the camera and the robot arm is relatively stationary. Among them, the camera coordinate system is O c O_cOc, the calibration plate coordinate system is O w O_wOw, the end coordinate system of the robot arm is O e O_eOe, the coordinate system of the robot arm base is O b O_bOb. It involves the conversion of several coordinate systems, that is, the conversion relationship from the calibration plate coordinate system to the camera coordinate system T wc T_w^cTwc, the conversion relation XX from the camera coordinate system to the base coordinate system of the robot armX , the transformation relationship from the coordinate system of the base of the robot arm to the coordinate system of the end of the robot armT be T_b^eTbe, where the transformation relationship from the camera coordinate system to the robot arm base coordinate system is XXX is the hand-eye calibration matrix that needs to be solved.
For the above conversion relationship, the calibration board is fixed at the end of the robot arm. In a certain posture, the coordinate value of the point on the calibration board in the calibration board coordinate system is P 1P_1P1，经过 T w c 、 T c b 、 T b e T_w^c、T_c^b、T_b^e Twc、Tcb、TbeAfter the conversion of the coordinate system transformation relation of , the point on the calibration board can be transferred to the coordinate value P 3 P_3 in the coordinate system at the end of the robot armP3, the conversion relationship is as follows:
T be XT wc P 1 = P 3 (1) T_b^eXT_w^cP_1=P3 \tag{1}TbeXTwcP1=P3 _( 1 )
Then the robot arm changes the pose, and another set of formulas with the same form as above can be obtained, namely:
T be ′ XT wc ′ P 1 = P 3 (2) T_b^{e\prime}XT_w^{c\ prime}P_1=P3 \tag{2}Tbe′XTwc′P1=P3 _( 2 ) T be , T be ′ T_b^e, T_b^{e\prime}
in the above formulas (1) and (2)Tbe、Tbe′It can be obtained through the pose output of the robot, T wc , T wc ′ T_w^c, T_w^{c\prime}Twc、Twc′It can be obtained through the external parameters of the monocular camera calibration. The above formula (1) (2) can be converted into the following form:
T be XT wc = T be ′ XT wc ′ (3) T_b^eXT_w^c=T_b^{e\prime}XT_w^{c\prime} \tag {3}TbeXTwc=Tbe′XTwc′( 3 )
It can be further transformed into:
T be ′ − 1 T be X = XT wc ′ T wc − 1 (4) T_b^{e\prime-1}T_b^eX=XT_w^{c\prime}T_w^ {c-1} \tag{4}Tbe′−1TbeX=XTwc′Twc−1( 4 )
Formula (4) can be understood asAX = XB AX=XBAX=The form of XB , at this time A = T be ′ − 1 T be A=T_b^{e\prime-1}T_b^eA=Tbe′−1Tbe、 B = T w c ′ T w c − 1 B=T_w^{c\prime}T_w^{c-1} B=Twc′Twc−1, of which A , BA, BBoth A and B are known numbers. By changing the end pose of the robot arm multiple times, the hand-eye calibration equationAX = XB AX=XBAX=Solving XB , you can get the hand-eye transformation matrixXXthe value of X.