1. Forward projection: from world coordinate system to pixel coordinate system
Mapping process from world 3D coordinate system (x, y, z) to image pixel coordinates (u, v)
(1) Mapping from the world coordinate system to the camera coordinate system .
The conversion of the two coordinate systems is relatively simple, that is, rotation matrix + translation matrix , and the rotation matrix is obtained by rotating around the X, Y, and Z axes.
R belongs to the rotation matrix from the world coordinate system to the camera coordinate system. The rotation matrix is R = R(z) * R(y) * R(x) , which is determined according to the rotation direction specified
when the camera external parameters are calibrated , which is 3 × 3 moments. t is the translation matrix from the origin of the world coordinate system to the origin of the camera coordinate system, which is a 3 x 1 matrix. The mapping from world coordinates to camera coordinates is:
(2) Normalized coordinate plane in the camera coordinate system
The projection of the ray onto the plane is equivalent, so it is normalized to facilitate calculation.
(3) Distortion occurs on the normalized plane
Distortion is divided into radial distortion (light refraction) and tangential distortion (installation tilt).
Simultaneously combine the radial distortion and tangential distortion formulas to obtain the distortion model distortion() as:
corresponding to the distortion parameter in the calibration file.
That is, the distortion map is:
(3) Normalize the plane coordinates to the pixel plane
1. Back projection: pixel coordinate system to world coordinate system (Z = 0) plane
Mapping process from image pixel coordinates (u, v) to world 3D coordinate system (x, y, z = 0)
(1) Pixel plane to normalized plane
(2) Reverse de-distortion on the normalized plane
Since the pixel coordinates (u, v) are obtained through internal reference mapping of the distorted point. Therefore, it is necessary to inversely undistort the distorted points into undistorted points.
Newton iteration method:
Initialization:
iterative solution:
until the error is small enough to obtain a non-distorted point.
At this point, the point on the normalized plane in the camera coordinate system is obtained.
(2) Camera coordinate system normalized plane to ground plane mapping
The mapping from one plane of the camera coordinate system to another plane of the world coordinate system can be obtained through the homography matrix H.
Suppose the camera coordinates are (xc, yc, zc) in the world coordinate system,
Rotate the normalized plane point in the camera coordinates and translate it to the world coordinate system: Assuming that the point mapped to the ground plane in the world coordinate system is (x, y, 0), then according to the similar triangle:
Get:
Similarly:
