Perspective transformation on art

perspective transformation

Perspective transformation is very important in the game, we use perspective transformation in map recognition and fine-tuning.

1. The use of fine-tuning

When the car arrives near the target board, after the camera detects the target board, the position of the picture from the car can be determined through perspective transformation. It needs to be emphasized here that the perspective transformation we use is different from that used by Four Wheels. To be precise, it is a transformation from image coordinates to world coordinates.
The perspective transformation of the four wheels is to make the image easier to judge elements, and here it is to get the exact position. Therefore, we don't need to transform the whole image at all, just transform a point. Such computation is very small.
The following is the core code of perspective transformation.

#返回透视矩阵
#XY为世界坐标，UV为相机坐标
def cal_mtx(UV:np.array,XY:np.array)->np.array:
    A = []
    B =[]
    for i in range(4):
        a = [[UV[i][0],UV[i][1],1,0,0,0,-XY[i][0]*UV[i][0],-XY[i][0]*UV[i][1]],
             [0,0,0,UV[i][0],UV[i][1],1,-XY[i][1]*UV[i][0],-XY[i][1]*UV[i][1]]]
        B+= [[XY[i][0]],
             [XY[i][1]]]
        A+=a

    A = np.array(A)
    B = np.array(B)

    x= np.solve(A,B)

    H = [[x[0][0], x[1][0], x[2][0]],
         [x[3][0], x[4][0], x[5][0]],
         [x[6][0], x[7][0], 1]]

    return np.array(H)

In fact, there are only a few short lines, essentially solving a $3\times$MatrixHH$of\; 3$$H$：
$$\left(\begin{array}{ccc}{H}_{11}& H_{12}& H_{13}\\ H_{21}& H_{22}& H_{23}\\ H_{31}& H_{32}& 1\end{array}\right)$$
There are 8 unknowns in total, so 4 points are needed to solve this matrix.
The specific derivation is not repeated here one by one, after all, it is tiring to formulate. Those who are interested can search for it.

After obtaining this matrix, turn the points on the image into secondary coordinates
$$\left(\begin{array}{c}{X}^{{}^{\prime }}\\ Y^{{}^{\prime }}\\ S\end{array}\right)=\left(\begin{array}{ccc}{H}_{11}& H_{12}& H_{13}\\ H_{21}& H_{22}& H_{23}\\ H_{31}& H_{32}& 1\end{array}\right)\left(\begin{array}{c}U\\ V\\ 1\end{array}\right)$$
Because it is a homogeneous coordinate, the real position can only be obtained after dividing by S.

A quick determination $H$ 's script (img2world.py), although not rigorous, but according to our experience, it is enough for the game.
Arrange a piece of A4 paper as follows

The red line draws the image of the A4 paper on the camera. Note that the bottom of the A4 paper should be aligned with the bottom of the camera field of view. The midline is aligned with the midline of the camera field of view. At this time, a matrix $H.$ _
For the time being, we define the left and right direction of the car as the x-axis, and the direction of the front of the car as the y-axis. At this time, you will find that the data in the x-axis direction is more accurate (you can use a tape measure to test it). But the error in the y-axis direction should be large. This is because we don't know how far the bottom of the camera field of view is from the center of the car. So here you need to add a translation constant in the y direction. Just test it by hand here.

2. Identify the map

In recognition maps, perspective transformations are particularly critical. This is related to your starting speed and recognition accuracy.
If there is no perspective transformation, it usually needs to be adjusted before the map can be recognized, and the recognition accuracy is not very accurate. However, after perspective transformation and rounding, the output is basically the point generated by the host computer, and there will be no error.

Different from fine-tuning, the perspective matrix of the recognition map is not fixed, and needs to be calculated once every frame, which is determined by the four vertices of the map frame.