[C++ OpenCV] Image transformation: connection, size, flip, rotation, affine transformation

void cv::resize(Inputarry src,
                Outputarry dst,
                Size      dsize,
                double     fx=0,
                double     fy=0,
                int interpolation = INTER_LINEAR
)

src - input image.
dst - the output image; its size is dsize (when it is non-zero) or the size calculated from src.size(), fx, and fy; dst is of the same type as src.
dsize - Output image size; if it is equal to zero, it is calculated as: dsize = Size(round(fx src.cols), round(fy src.rows)). dsize or fx and fy must be non-zero.
fx - scale factor along the horizontal axis; when it is equal to 0, it is calculated as (double)dsize.width/src.cols
fy - the scale factor along the vertical axis; when it is equal to 0, it is calculated as (double)dsize.height/src.rows
Interpolation - Interpolation method . When choosing an interpolation method, you need to weigh it based on specific needs and application scenarios. For example:
- If you simply reduce the image or resize the image to a relatively small size, nearest neighbor interpolation ( cv2.INTER_NEAREST) may be a good choice because it is fast and the effect is acceptable.
- Bilinear interpolation ( ) is a common option if you need to maintain good smoothness as the image is enlarged or reduced .cv2.INTER_LINEAR
- If you have higher requirements on image quality and can tolerate longer calculation times , bicubic interpolation ( cv2.INTER_CUBIC) or Lanczos interpolation ( cv2.INTER_LANCZOS4) may be a better choice.

interpolation algorithm

Enumerator
INTER_NEAREST	Nearest neighbor interpolation, this method selects the original pixel value closest to the target pixel as the value of the adjusted pixel. This interpolation method is fast to compute, but may introduce jagged edges and distortion. Used for simple zoom-in or zoom-out at the pixel level.
INTER_LINEAR	Bilinear interpolation, a method that uses a weighted average of four adjacent original pixels to calculate the value of an adjusted pixel. It provides smoother results than nearest neighbor interpolation, but may result in blurring of some details. It is often used in the process of enlarging or reducing the image.
INTER_CUBIC	Bicubic interpolation, this method further considers the color change trend of adjacent pixels based on bilinear interpolation to produce smoother results. It is more computationally complex than bilinear interpolation, but generally provides better results when upscaling an image.
INTER_AREA	resampling using pixel area relation. It may be a preferred method for image decimation, as it gives moire'-free results. But when the image is zoomed, it is similar to the INTER_NEAREST method. This is probably the preferred method for image extraction as it gives ripple-free results. But when the image is scaled it is similar to the INTER_NEAREST method.
INTER_LANCZOS4	Lanczos interpolation on 8×8 neighborhood, this method uses Lanczos filter for interpolation, which can obtain sharper results, but the calculation overhead is large. Suitable for large-scale enlargement of images.
INTER_LINEAR_EXACT	Bit exact bilinear interpolation Bit exact bilinear interpolation
INTER_NEAREST_EXACT	Bit exact nearest neighbor interpolation. This will produce same results as the nearest neighbor method in PIL, scikit-image or Matlab. Bit exact nearest neighbor interpolation. This will produce the same results as PIL nearest neighbor, scikit-image or Matlab.
INTER_MAX	mask for interpolation codes mask for interpolation codes
WARP_FILL_OUTLIERS	flag, fills all of the destination image pixels. If some of them correspond to outliers in the source image, they are set to zero flag, fills all of the destination image pixels. If some of them correspond to outliers in the source image, they are set to zero
WARP_INVERSE_MAP	flag, inverse transformation ，逆变换 For example, linearPolar or logPolar transforms: flag is not set: dst(ρ,ϕ)=src(x,y) flag is set: dst(x,y)=src(ρ,ϕ)

It is worth noting that size has the same effect as the two doubles. fx and fy refer to scaling the image coordinate axes (fx=2 means the x-axis is doubled), so when using the function, you only need to use one type . , the conflict between the two shall be subject to dsize

dsize =Size( round(fx *src.cols),round(fy *src.rows) )

image flip

void cv::flip(InputArray src, 
            OutputArray dst, 
            int flipCode);

Parameter src input matrix.

Parameter dst output matrix, the same size as the input matrix.

Parameter flipCode is a flag that determines how to flip the matrix; = 0 means flipping around the x-axis, positive value means flipping around the y-axis, and negative value means flipping around both axes.

Reference transpose , repeat , completeSymm

Image stitching

Longitudinal splicing

void cv::vconcat    (   const Mat *     src,
                        size_t      nsrc,
                        OutputArray     dst 
                    )   
  void cv::vconcat  (   InputArray      src1,
                        InputArray      src2,
                        OutputArray     dst 
                    )

Horizontal splicing

void cv::hconcat    (   const Mat *     src,
                        size_t      nsrc,
                        OutputArray     dst 
                    )   
void cv::hconcat    (   InputArray      src1,
                        InputArray      src2,
                        OutputArray     dst 
                    )

It is worth noting that when splicing horizontally, the height of the two images should be the same, and the width of the two images splicing vertically should be the same.

Image interpolation principle

After the image undergoes a series of processing, the size and size of its pixel matrix change, so how does the pixel value of each pixel corresponding to it change? If the transformed pixel matrix is larger or smaller than the original matrix in rows and columns, then the transformed pixel matrix may have more pixels or be compressed. How to determine the values of these pixel positions?

Case 1: Two pixels correspond to the same position after transformation

Case 2: Two adjacent pixels are no longer adjacent after transformation. How to determine the middle position pixel?

Interpolation simply means using known pixel values to infer unknown pixel values . Common interpolation methods include: nearest neighbor method, linear interpolation method, and bilinear interpolation method. This article focuses on the bilinear interpolation method, which comes from the blog post

A linear interpolation is performed once in each direction (bilinear interpolation is performed using linear interpolation first in one direction and then linear interpolation in the other direction. Although each step is linear in sample values and positions, But the interpolation is not linear overall, but quadratic at the sample locations.)

effect

Generally used to resample images and textures. Calculate a weighted average of the properties (color, transparency, etc.) of the four surrounding texels and apply it to the screen pixel. (Simply put, when I ask for a pixel value with known coordinates, I first find the coordinates of the four surrounding known pixels, which are usually the four nearest pixel values . Through two single linear interpolations, I find his pixel value as How many)

single linear interpolation

Here is the single linear interpolation

Column type

If you look carefully, the distance between x, x0, and x1 is used as a weight for weighting y0 and y1. Bilinear interpolation is essentially linear interpolation in two directions.

Bilinear interpolation formula

Insert image description here

As shown in the figure, we need to find the pixel value of point P. We already know the coordinates of Q11, Q21, Q12, Q22, and P. Also know the pixel values of Q11, Q21, Q12, and Q22. So first use single linear interpolation about X to calculate the pixel values of R1 and R2 respectively :

Insert image description here

The letters f(Q11), f(Q12), f(Q21), f(Q22), x1, x2, and x in the equation are all known. The calculated f(x,y1) and f(x ,y2) are the pixel values of R1 and R2. Then use single linear interpolation about the y direction to calculate the pixel value of point P :

The letters y1, y2, and y in the equation on the right are all known, and f(x,y1) and f(x,y2) are the R1 and R2 pixel values calculated in the previous equation.

Example of bilinear interpolation

Give a chestnut:

Insert image description here

As shown in the example on the right, the pixel at row 20.2 column 14.5 can be calculated by linearly interpolating between the values in columns 14 and 15 at rows 20 and 21. Intensity value. (This also illustrates that the nearest adjacent pixel is generally used)

Insert image description here

Visual display of bilinear interpolation

Insert image description here We can see that what is here is bilinear interpolation (Bilinear) in a plane

significance

This algorithm reduces some of the visual distortion caused by resizing images to non-integer scaling factors

Affine transformation

Affine transformation is a geometric transformation in two-dimensional space. Affine transformation is often used to perform linear transformation operations such as translation, rotation, scaling, and cross-cutting of images. It is also called three-point transformation. This transformation can maintain the flatness and parallelism of the image. Straightness means that after the image undergoes affine transformation, straight lines are still straight lines; parallelism means that after the image completes affine transformation, parallel lines are still parallel lines.

When to use:

Affine transformation has a wide range of applications in the fields of image processing and computer vision, and is often used in the following situations and purposes:

Translation and rotation correction: When an image requires translation or rotation correction, affine transformation can be used. For example, during the image shooting process, the image may be tilted or rotated due to camera posture or movement, and the image can be corrected to the horizontal or vertical direction through affine transformation.
Image scaling and cropping: By adjusting the scaling parameters in the affine transformation matrix, image scaling and cropping operations can be achieved. Zooming can be used to resize an image, while cropping can be used to extract areas of interest or change the aspect ratio of an image.
Image deformation and correction: Affine transformations can be used to perform shape transformations on images, such as miscut transformations. This is useful in special applications, such as perspective correction in document scanning, which can convert a distorted document image into a rectangular shape.
Feature alignment and matching: In computer vision, for the positioning and matching of feature points, affine transformation can be used to align one set of feature points to another set of feature points to achieve tasks such as image registration and target tracking.
Image synthesis and reconstruction: By applying affine transformations, images of different scales, orientations, or shapes can be synthesized or reconstructed. For example, in image stitching, multiple images can be aligned using affine transformations to create a panoramic image.

When the three-point coordinates before and after transformation are determined, my affine transformation matrix M is the only one that exists.

Affine transformation matrix

Matrix A is responsible for rotating pixels, and matrix B is responsible for translation.

Affine transformation principle

Program source code:

void::warpAffine( INputarry src,
                  Outputarry dst,
                  InputArry M,
                  size      dsize,
                  int  flags= INTER_LINEAR,
                  int   borderMode = BOARDER_CONSTANT,
                  CONST Scalar& borderValue = Scalar
)

Enumerator
BORDER_CONSTANT	`iiiiii\|abcdefgh\|iiiiiii` with some specified `i`	Specific value (mostly used)
BORDER_REPLICATE	`aaaaaa\|abcdefgh\|hhhhhhh`	Copy on both sides
BORDER_REFLECT	`fedcba\|abcdefgh\|hgfedcb`	flashback filler
BORDER_WRAP	`cdefgh\|abcdefgh\|abcdefg`	Positive order filling
BORDER_REFLECT_101	`gfedcb\|abcdefgh\|gfedcba`
BORDER_TRANSPARENT	`uvwxyz\|abcdefgh\|ijklmno`
BORDER_REFLECT101	same as BORDER_REFLECT_101
BORDER_DEFAULT	same as BORDER_REFLECT_101
BORDER_ISOLATED	do not look outside of ROI

BORDER_CONSTANT is used the most because it sets a specific value to 0 and the filled area is black, and we usually do not need to obtain the picture information of the filled area, so it is meaningless to study the pixel matrix of the filled area.

image rotation

There is no function that directly implements image rotation in opencv. Instead, the rotation matrix is obtained through getRotationMatrix2D() and then the affine transformation function is called to achieve image rotation.

Source code

Mat cv::getRotationMatrix2D(Point2f     center,
                            double      angle,
                            double      scale 
)

center Image rotation center position, input value, usually defined as Point2f center1(img.rows/2.0, img.cols/2.0)
angle rotation angle
scale is the scaling factor of the two axes, which can achieve image scaling during rotation.

in:

Note: This function has a return value of Mat type

After defining a Mat type matrix to accept the return value and using it as the M parameter of the affine function , image rotation can be achieved.

Get affine matrix

Mat cv::getAffineTransform(const Point2f    src[],
                            const Point2f   dst[] 
                        )

src[]: The three pixel coordinates in the original image are an array, and the data type inside is Point2f.
dst[]: The three pixel coordinates in the target image are an array, and the data type inside is Point2f.

Practical operation

1. Realize image rotation

#include<opencv2/opencv.hpp>
#include<iostream>
#include<vector>

using namespace std;
using namespace cv;


int main()
{
    Mat img = imread("E://学习//OPEN-CV学习//lena.png");
    if (img.empty())
    {
        cout << "读取失败！" << endl;
        return -1;
    }

    //实现旋转操作
    Point2f center (img.rows / 2.0, img.cols / 2.0);
    double angle = 30;
    double scale = 0.5;
    Size dsize(img.rows, img.cols);
    Mat rotation,dst;
    rotation = getRotationMatrix2D(center, angle, scale);

    warpAffine(img, dst, rotation, dsize);
    imshow("dst", dst);

        waitKey(1);
        return 0;
}

Get the rotation matrix result and the rotated result map

2. Implement affine transformation based on the three defined points and obtain the affine transformation matrix

#include<opencv2/opencv.hpp>
#include<iostream>
#include<vector>

using namespace std;
using namespace cv;


int main()
{
    Mat img = imread("E://学习//OPEN-CV学习//lena.png");
    if (img.empty())
    {
        cout << "读取失败！" << endl;
        return -1;
    }

    //实现旋转操作
    Point2f center (img.rows / 2.0, img.cols / 2.0);
    double angle = 30;
    double scale = 0.5;
    Size dsize(img.rows, img.cols);
    Mat rotation,dst;
    rotation = getRotationMatrix2D(center, angle, scale);

    warpAffine(img, dst, rotation, dsize);
    imshow("dst", dst);

    Point2f ori[3];
    Point2f res[3];
    //原图像的三个边角点
    ori[0] = Point2f (0, 0);
    ori[1] = Point2f (0, (float)(img.cols-1));
    ori[2] = Point2f ((float)(img.rows-1), (float)(img.cols - 1));
    //想仿射变换的三个点
    res[0] = Point2f((float)(img.rows*0.10), (float)(img.cols*0.20));
    res[1] = Point2f((float)(img.rows * 0.15), (float)(img.cols * 0.70));
    res[2] = Point2f((float)(img.rows * 0.65), (float)(img.cols * 0.90));
    //获取仿射变换矩阵
    Mat rotation1, dst1;
    rotation1 = getAffineTransform(ori, res);
    warpAffine(img, dst1, rotation1, dsize);
    imshow("dst1", dst1);


        waitKey(1);
        return 0;
}

Obtain the affine matrix and the result image after affine transformation.