Coordinate system conversion
I just stayed at the stage of being able to use it before, and I never understood the principles of calculation. Today I read through the article of the big guy, and I wrote it concisely. Thank you~~ I hereby record it for personal notes only
Thank you for posting the link~
https://blog.csdn.net/weixin_44278406/article/details/112986651
https://blog.csdn.net/guyuealian/article/details/104184551
Four different types of coordinate systems
The three-dimensional object is converted into two-dimensional coordinates on the photo, and the conversion is performed by four coordinate systems.
1. World coordinate system
The world coordinate system is a special coordinate system that establishes the frame of reference needed to describe other coordinate systems. Being able to use the world coordinate system to describe the position of other coordinate systems, but not to describe the world coordinate system in terms of a larger, external coordinate system. In a non-technical sense, the world coordinate system establishes the largest coordinate system we care about, not really the whole world.
Indicated by 
), the world coordinate system can be rotated and translated to obtain the camera coordinate system.
2. Camera coordinate system
Taking the geometric center (optical center) of the camera lens as the origin, the coordinate system satisfies the right-hand rule, using
3. Image physical coordinate system
Taking the center of the CCD image as the origin, the coordinates are represented by ( x , y ) (x, y)(x,y). The unit of the image coordinate system is generally millimeters. The origin of the coordinates is the intersection of the camera optical axis and the imaging plane (generally case, this intersection point is close to the exact center of the image)
CCD, English full name: Charge coupled Device, Chinese full name: charge coupled device, can be called a CCD image sensor. CCD is a semiconductor device that converts optical images into digital signals. The tiny photosensitive material implanted on the CCD is called a pixel (Pixel). The more pixels contained on a CCD, the higher the picture resolution it provides.
4. Image pixel coordinate system
In fact, when we refer to an image, we usually refer to the pixel coordinate system of the image. The origin of the pixel coordinate system is the upper left corner, and the unit is pixel.
Transform the origin of the image coordinate system 
into 
the coordinate system with the origin as. Reasons to use:
If you use the image coordinate system, the unit is mm, it is actually not easy to measure the specific image. If you follow the uniform pixel standard, it is easier to measure the quality of the image
If you use the image coordinate system, then there are four quadrants, so there will be problems with positive and negative numbers, but after converting to the pixel coordinate system, they are all integers. In subsequent operations and calculations, it is much simplified.
Coordinate transformation
The basic pinhole model. Mathematically, this model is the central projection from the three-dimensional space to the two-dimensional plane (image plane or focal plane), which is described by a 
projection matrix 
, K is the internal camera parameters, and [R∣t] is the external parameter ( external parameters).
world coordinates → camera coordinates (rigid transformation)
Represents the camera coordinates; 
represents the world coordinates; R represents the orthogonal unit rotation matrix, and t represents the three-dimensional translation vector. According to the rotation angle, the rotation matrix in three directions can be obtained respectively, and the rotation matrix is their product:
By the way, record the formulas of the three rotation matrices, which are often forgotten.
Rotate θ degrees around X
Rotate θ degrees around the Y axis
Rotate θ degrees around the Z axis
camera coordinates → image coordinate system (central projection)
The perspective relationship between the camera coordinate system and the image coordinate system is calculated using similar triangles.
The multiplication of matrices written in homogeneous coordinate form is
Where f represents the focal length, that is, the difference between the camera coordinate system and the image coordinate system on the Z axis. At this time, the unit of the projection point p is still mm, not pixel, which is inconvenient for subsequent calculations.
image coordinate system → pixel coordinate system (discretization)
The origin of the pixel coordinate system is the upper left corner, and the unit is pixel. Both the pixel coordinate system and the image coordinate system are on the imaging plane, but their respective origins and measurement units are different. The origin of the image coordinate system is the intersection of the camera optical axis and the imaging plane, usually the midpoint of the imaging plane or the principal point. The unit of the image coordinate system is mm, which belongs to the physical unit, while the unit of the pixel coordinate system is pixel. We usually describe a pixel with several rows and columns. So the conversion between the two is as follows: where dx and dy represent how many mm each column and each row represent, that is, 1pixel=dx mm
in
is the camera internal parameter matrix,
is the external parameter matrix. Camera calibration is to solve the parameters of these two matrices.
