[Camera calibration] The transformation relationship between the four coordinate systems
The relationship between world coordinate system, camera coordinate system, image physical coordinate system, and pixel coordinate system:
First look at how several coordinate systems are put together:
1: World coordinate system: According to the situation, it can represent any object, which is introduced by the camera at this time. The unit is m.
2: Camera coordinate system: Taking the optical center of the camera as the origin (in the pinhole model, that is, the pinhole is concerned), the z-axis coincides with the optical axis, that is, the z-axis points to the front of the camera (that is, perpendicular to the imaging plane), x The positive direction of the axis and the y axis are parallel to the object coordinate system, where f in the figure above is the focal length of the camera. Unit m
3: Image physical coordinate system (also called plane coordinate system): The position of the pixel is expressed in physical units, and the origin of the coordinate is the position of the intersection of the camera's optical axis and the image physical coordinate system. The coordinate system is o-xy on the graph. The unit is mm. The reason for the unit millimeter is that the CCD sensor inside the camera is very small at this time, such as 8mm x 6mm. But the final image photo is also in pixels, such as 640x480. This involves the transformation of the physical coordinate system of the image and the pixel coordinate system. The following pixel coordinate system will be mentioned.
4: Pixel coordinate system: in pixels, the origin of the coordinate is at the upper left corner. This is also the reason why the coordinate origin of some opencv, OpenGL and other libraries is selected in the upper left corner. Of course, it is obvious that the CCD sensor has a conversion between the unit of mm and the pixel. For example, the 8mm x 6mm on the CCD sensor is converted to a pixel size of 640x480. If dx indicates that the physical size of each pixel in the pixel coordinate system is 1/80. That is to say the relationship between millimeters and pixels It is piexl/mm.
Now that the four coordinate systems have been explained, let's talk about the transformation relationship between each coordinate system.
One: World coordinates to camera coordinate system
The coordinate system transformation between objects can represent the rotation transformation of the coordinate system plus the translation transformation, and the transformation relationship from the world coordinate system to the camera coordinate system is the same. Rotate different angles around different axes to get different rotation matrices. as follows:
Then the transformation from the world coordinate system to the camera coordinate system is as follows:
2: Camera coordinate system to image physical coordinate system
From the camera coordinate system to the image coordinate system, it belongs to the perspective projection relationship, and it is converted from 3D to 2D. It can also be seen as a modified model of the pinhole model. Satisfy the similarity theorem of triangles.
Three: Image physical coordinate system to pixel coordinate system.
At this time, unlike the previous coordinate system transformation, there is no rotation transformation at this time, but the coordinate origin position is inconsistent, and the size is inconsistent, then the expansion and translation transformations are designed.
In summary, we can see that the following relationship exists between the four coordinate systems (the matrix is multiplied to the left in turn)
The internal and external parameters of the camera can be obtained through Zhang Zhengyou's calibration. Judging from the final conversion relationship, a coordinate point in three dimensions can indeed find a corresponding pixel point in the image, but conversely, finding its corresponding point in three dimensions through a point in the image becomes a very Problem, because we don’t know the value of Zc on the left side of the equation.