"Digital Image Processing-OpenCV/Python" serial (44) Projection transformation of images

This book’s JD discount purchase link: https://item.jd.com/14098452.html
This book’s CSDN exclusive serial column: https://blog.csdn.net /youcans/category_12418787.html

Insert image description here

Chapter 6 Geometric Transformation of Images

Geometric transformation is divided into isometric transformation, similarity transformation, affine transformation and projection transformation, which refers to transforming the position, size, shape and projection of the image, and projecting the image from the original plane to a new viewing plane. The geometric transformation of OpenCV images is essentially to convert a multi-dimensional array into another multi-dimensional array through a mapping relationship.

Summary of this chapter

Introduce affine transformation and learn to use affine transformation matrices to achieve affine transformation of images.
Learn to use functions to translate, scale, rotate, flip, and skew images.
Introduce projection transformation and learn to use the projection transformation matrix to implement the projection transformation of images.
Introduce image remapping and learn to use mapping functions to implement custom transformation and dynamic transformation of images.

6.6 Projective transformation of images

Perspective Transformation is a commonly used projection transformation in OpenCV, which refers to projecting an image to a new viewing plane. The characteristic of projective transformation is that the parallel relationship and proportional relationship in the original image can be changed, but the straight lines in the image can still remain straight after the projection transformation.
Projection transformation can correct the rotation of objects in three-dimensional space, and is mainly used for image splicing and correcting image distortion caused by perspective projection.
The method of projective transformation is to determine four non-collinear points on the original image. Given the positions of these four points in the transformed image, a projective transformation is determined, and its transformation relationship is It can be described by the following 3×3 matrix.

$\begin{bmatrix} \tilde{x}\\ \tilde{y}\\ \tilde{z} \end{bmatrix} = M_P \begin{bmatrix} x\\ y\\ z \end{bmatrix} ,\hspace{1em} M_P= \begin{bmatrix} M_{11} &M_{12} &M_{13}\\ M_{21} &M_{22} &M_{23}\\ M_{31} &M_{32} &M_{33} \end{bmatrix}$

Affine transformation is performed on a two-dimensional plane, while projective transformation is performed on a three-dimensional coordinate system. Affine transformation is a 3-point transformation, and projective transformation is a 4-point transformation. Comparing the description formulas of affine transformation and projective transformation, affine transformation can be regarded as a z-axis invariant perspective transformation.

In OpenCV, the projection transformation matrix is first calculated by the function cv.getPerspectiveTransform $M_P$ , and then use the function cv.warpPerspective to transform the matrix according to the projection $M_P$ Calculate the projection transformation image.

The function cv.getPerspectiveTransform can solve to obtain the projection transformation matrix based on the corresponding position coordinates of the four non-collinear points in the image before and after transformation $M_P$ 。

function prototype

cv.getPerspectiveTransform(src, dst[,solveMethod]) → MP

Parameter Description

src: The coordinates of four non-collinear points in the original image, which is a Numpy array with a shape of (4,2).
dst: The coordinates of the corresponding four non-collinear points in the projection transformation image, which is a Numpy array with a shape of (4,2).
solveMethod: matrix decomposition method.
- DECOMP_LU: Gaussian elimination method to select the best axis, the default method.
- DECOMP_SVD: Singular value decomposition (SVD) method.
- DECOMP_EIG: Eigenvalue decomposition method, must be symmetrical to src.
- DECOMP_CHOLESKY: Cholesky LLT decomposition method.
- DECOMP_QR: Orthogonal triangular (QR) decomposition method.
- DECOMP_NORMAL: Use regular equations, used in conjunction with the previous method.
MP: Projection transformation matrix, which is a Numpy array with shape (3,3) and type np.float32.

Pay attention to the problem

(1) Although the parameters src and dst usually represent the input and output images, in the function cv.getPerspectiveTransform, they refer to the four non-collinear points in the original image and the transformed image, also known as the vertices of the quadrilateral.
(2) The parameters src and dst are Numpy arrays with a shape of (4,2), and the values are the coordinates (x, y) of the four vertices in the image.

The function cv.warpPerspective can calculate the projection transformation image through the projection transformation matrix.

function prototype

cv.warpPerspective (src, M, dsize[, dst, flags, borderMode, borderValue]) → dst

The formula for calculating the projection transformation image from the projection transformation matrix M is:

Parameter Description

src: original image, which is a Numpy array.
dst: Projection transformation output image, the type is the same as src, the image size is determined by the parameter dsize.
M: Projection transformation matrix, which is a Numpy array with shape (3,3) and type np.float32.
dsize: Output image size, format is tuple (w, h).
flags: Interpolation method and inverse transformation flags, optional.
- INTER_LINEAR: bilinear interpolation, default method.
- INTER_NEAREST: nearest neighbor interpolation.
- WARP_INVERSE_MAP: Inverse transformation flag.
borderMode: border expansion method, optional, default is cv.BORDER_CONSTANT.
borderValue: border padding value, optional, the default value is 0, indicating black padding.

Pay attention to the problem

(1) The format of the output image size dsize is (w, h), which is the opposite order of the image shape (h, w) in OpenCV.
(2) Calculate the projection transformation through the function cv.warpPerspective. The shape of the projection transformation matrix M is (3,3), and the data type must be np.float32.
(3) When flags is set to WARP_INVERSE_MAP, the inverse projection transformation matrix is first calculated from the projection transformation matrix, and then the inverse projection transformation image of the input image is calculated.

[Routine 0606] Image correction based on projection transformation

Photos taken with mobile phones or cameras usually have projection distortion. This routine implements image correction through projective transformation.

First, use the mouse to select the four vertices of the rectangle in sequence in the image, obtain the coordinates of the four vertices, and then calculate the coordinates of the four vertices after projection transformation based on the aspect ratio, and perform projection transformation to achieve image correction.

# 【0606】基于投影变换实现图像校正
import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

def onMouseAction(event, x, y, flags, param):  # 鼠标交互 (单击选点，右击完成)
    setpoint = (x, y)
    if event == cv.EVENT_LBUTTONDOWN:  # 单击
        pts.append(setpoint)  # 选中一个多边形顶点
        print("选择顶点 {}：{}".format(len(pts), setpoint))

if __name__ == '__main__':
    img = cv.imread("../images/Fig0602.png")  # 读取彩色图像(BGR)
    imgCopy = img.copy()
    height, width = img.shape[:2]

    # 鼠标交互从输入图像选择 4 个顶点
    print("单击左键选择 4 个顶点 (左上-左下-右下-右上):")
    pts = []  # 初始化 ROI 顶点坐标集合
    status = True  # 进入绘图状态
    cv.namedWindow('origin')  # 创建图像显示窗口
    cv.setMouseCallback('origin', onMouseAction, status)  # 绑定回调函数
    while True:
        if len(pts) > 0:
            cv.circle(imgCopy, pts[-1], 5, (0,0,255), -1)  # 绘制最近的一个顶点
        if len(pts) > 1:
            cv.line(imgCopy, pts[-1], pts[-2], (255, 0, 0), 2)  # 绘制最近的一段线段
        if len(pts) == 4:  # 已有 4个顶点，结束绘制
            cv.line(imgCopy, pts[0], pts[-1], (255,0,0), 2)  # 绘制最后的一段线段
            cv.imshow('origin', imgCopy)
            cv.waitKey(1000)
            break
        cv.imshow('origin', imgCopy)
        cv.waitKey(100)
    cv.destroyAllWindows()  # 释放图像窗口
    ptsSrc = np.array(pts)  # 列表转换为 (4,2)，Numpy 数组
    print(ptsSrc)

    # 计算投影变换矩阵 MP
    ptsSrc = np.float32(pts)  # 列表转换为Numpy数组，图像4个顶点坐标为 (x,y)
    x1, y1, x2, y2 = int(0.1*width), int(0.1*height), int(0.9*width), int(0.9*height)
    ptsDst = np.float32([[x1,y1], [x1,y2], [x2,y2], [x2,y1]])  # 投影变换后的 4 个顶点坐标
    MP = cv.getPerspectiveTransform(ptsSrc, ptsDst)

    # 投影变换
    dsize = (450, 400)  # 输出图像尺寸为 (w, h)
    perspect = cv.warpPerspective(img, MP, dsize, borderValue=(255,255,255))  # 投影变换
    print(img.shape, ptsSrc.shape, ptsDst.shape)

    plt.figure(figsize=(9, 3.4))
    plt.subplot(131), plt.axis('off'), plt.title("1.Original")
    plt.imshow(cv.cvtColor(img, cv.COLOR_BGR2RGB))
    plt.subplot(132), plt.axis('off'), plt.title("2.Selected vertex")
    plt.imshow(cv.cvtColor(imgCopy, cv.COLOR_BGR2RGB))
    plt.subplot(133), plt.axis('off'), plt.title("3.Perspective correction")
    plt.imshow(cv.cvtColor(perspect, cv.COLOR_BGR2RGB))
    plt.tight_layout()
    plt.show()

Program description:
(1) This routine sets a callback function and selects 4 vertices from the input image through mouse interaction. For details on how to use mouse interactive operations, see Section 4.9.
(2) The coordinates of the four vertices after projection transformation are set by the user and can be modified as needed.
(3) The running results of image correction based on projection transformation are shown in Figure 6-6. Figure 6-6(1) shows the original image, and Figure 6-6(2) shows What is shown is that the four vertices of the chessboard are selected on the original image with the mouse. Figure 6-6(3) shows the image after projection transformation. It can be seen that the tilted checkerboard photographed in perspective in the original image is corrected to a rectangle.

Insert image description here

*Figure 6-6 Image projection transformation

Copyright Statement:
youcans@xupt original work, reprints must be marked with the original link: (https://blog.csdn.net/youcans/article/details/134487182) Crated：2023-11-20
Copyright 2023 youcans, XUPT

Welcome to follow this book’s CSDN exclusive serialization column
"Digital Image Processing-OpenCV/Python" serialization: https://blog.csdn.net/youcans/category_12418787.html