Chapter 5 - Geometric Transformations

Geometric transformations are operations that map one image into another. OpenCV provides several mapping-related functions.

According to different OpenCV functions, the mapping relationship can be divided into scaling, flipping, affine transformation, perspective, remapping, etc.

one.zoom:

In OpenCV, use cv2.resize() to scale the image:

• dst = cv2.resize(src, dsize, [, fx [, fy [, interpolation]]])

  • dst: represents the output target image, the type of the image is the same as src, and its size is dsize (when the value is non-zero), or it can be calculated by src.size(), fx, fy.
  • src: represents the original image that needs to be scaled.
  • dsize: represents the output image size.
  • fx: Represents the zoom ratio in the horizontal direction.
  • fy: Represents the scaling ratio in the vertical direction.
  • interpolation: represents the interpolation method, as shown in the following table.

  

  

In the cv2.resize() function, the size of the target image can be determined by either "parameter dsize" or "parameters fx and fy", as follows:

1. Specify by parameter dsize:

   ​ If the value of the parameter dsize is specified, the size of the target image is determined by the parameter dsize regardless of whether the values ​​​​of the parameters fx and fy are specified.
   At this point, it should be noted that the first parameter in dsize corresponds to the width of the zoomed image (width, that is, the number of columns cols, related to the parameter fx), and the second parameter corresponds to the height of the zoomed image (height, that is, the number of rows rows , related to the parameter fy).

   When the value of the parameter dsize is specified, the zoom size in the x direction (parameter fx) is:

   • （double）dsize.width/src.cols

   At the same time, the zoom size in the y direction (parameter fy) is:

   • （double）dsize.height/src.rows

2. Specified by parameters fx and fy:
   If the value of parameter dsize is None, then the size of the target image is determined by parameters fx and fy. At this point, the size of the target image is: dsize=Size(round(fx * src.cols), round(fy * src.rows))

Interpolation :

Interpolation refers to assigning values ​​to pixels that cannot be directly obtained through mapping when performing geometric processing on images. For example, if the image is enlarged to 2 times the original size, there will inevitably be some more pixels that cannot be directly mapped. For these pixels, the interpolation method determines how to determine their values. In addition, there will be some non-integer mapping values. For example, the reverse mapping may map the pixel value in the target image to the position corresponding to the non-integer value in the original image. Of course, it is impossible in the original image. If there are such non-integer positions, that is, the pixel on the target image cannot correspond to a specific position on the original image, at this time, these pixel points should also be interpolated to complete the mapping.

Notice:

​ The function cv2.resize() can realize the zoom function of the original image. It should be noted that before the operation starts, the size and type of the target image dst before the operation have nothing to do with the final target image dst. The final size and type of the destination image dst is specified via src, dsize, fx, fy. If you want the original image to be resized to be the same size as the target image, you must specify it through the above properties.

​ When reducing the image, use the area interpolation method (INTER_AREA) to get the best results; when enlarging the image, use the cubic spline interpolation method (INTER_CUBIC) and bilinear interpolation method (INTER_LINEAR) to achieve better results . The cubic spline interpolation method is slow, and the bilinear interpolation method is relatively fast and the effect is not inferior.

example:

import cv2

img = cv2.imread('../lena.bmp')
rows, cols = img.shape[:2]
size = (int(cols * 0.9), int(rows * 0.5))
rst = cv2.resize(img, size)
print('img.shape=', img.shape)
print('rst.shape=', rst.shape)
cv2.imshow('img', img)
cv2.imshow('rst', rst)
cv2.waitKey()
cv2.destroyAllWindows()


# 输出结果
img.shape= (512, 512, 3)
rst.shape= (256, 460, 3)

two. Flip:

In OpenCV, the flipping of the image is realized by the function cv2.flip(). This function can flip the image in the horizontal direction, flip the vertical direction, and flip in both directions at the same time.

• dst = cv2.flip( src, filpCode )

  • dst: represents the target image with the same size and type as the original image.
  • src: represents the original image to be processed.
  • flipCode: represents the rotation type. The meaning of this parameter is shown in the table.

  

In this function, the relationship between the target pixel and the original pixel can be expressed as:
$ds{t}_{ij}=\{\begin{array}{l}sr{c}_{src.rows-i-1,j},filcCode=0\\ sr{c}_{i,src.cols-j-1},filcCode>0\\ sr{c}_{src.rows-i-1,src.cols-j-1},filcCode<0\end{array}$

Among them, dst is the target pixel, src is the original pixel.

example:

import cv2

img = cv2.imread('../lena.bmp')
x = cv2.flip(img, 0)
y = cv2.flip(img, 1)
xy = cv2.flip(img, -1)
cv2.imshow('img', img)
cv2.imshow('x', x)
cv2.imshow('y', y)
cv2.imshow('xy', xy)
cv2.waitKey()
cv2.destroyAllWindows()

three. Affine:

Affine transformation means that the image can be translated, rotated and other operations through a series of geometric transformations . This transformation can maintain the straightness and parallelism of the image. Straightness means that after the image undergoes affine transformation, the straight line is still a straight line; parallel line means that after the image is affine transformed, the parallel line is still parallel.

The affine function implemented in OpenCV is cv2.warpAffine(), which implements transformation through a transformation matrix (mapping matrix) M, specifically: $dst(x,y)=src(M_{11}x+M_{12}y+M_{13},M_{13}x+M_{22}y+M_{23})$

As shown in the figure, the original image O can be transformed into an affine image R through a transformation matrix M

Therefore, the affine function cv2.warpAffine() can be used to implement operations such as translation and rotation of the image:

• dst = cv2.warpAffine( src, M, dsize [, flags [, borderMode [, borderValue ]]])
  • dst: represents the output image after affine, and the type of the image is the same as that of the original image. dsize determines the actual size of the output image.
  • src: represents the original image to be affine.
  • M: represents a 2×3 transformation matrix. Using different transformation matrices, different affine transformations can be achieved.
  • dsize: represents the size of the output image.
  • flags: represents the interpolation method, the default is INTER_LINEAR. When the value is WARP_INVERSE_MAP, it means that M is an inverse transformation type, which realizes the inverse transformation from the target image dst to the original image src.
  • borderMode: represents the edge type, the default is BORDER_CONSTANT. When the value is BORDER_TRANSPARENT, it means that the values ​​in the target image are not changed, and these values ​​correspond to outliers in the original image.
  • borderValue: represents the boundary value, the default is 0.

​ Through the above analysis, we can see that the function cv2.warpAffine() is used to realize affine transformation in OpenCV, and the syntax format after ignoring its optional parameters is:

• dst=cv2.warpAffine（src ,M ,dsize）

It converts the original image src to the target image dst through the transformation matrix M:

• $dst(x,y)=src(M_{11}x+M_{12}y+M_{13},M_{13}x+M_{22}y+M_{23})$

Therefore, what form of affine transformation is performed depends entirely on the transformation matrix M . Different affine transformations realized through different transformation matrices M are introduced respectively below.

1. Panning:

Convert the original image src to the target matrix dst through the transformation matrix M:

• $dst(x,y)=src(M_{11}x+M_{12}y+M_{13},M_{13}x+M_{22}y+M_{23})$

Move the original image src 100 pixels to the right and 200 pixels to the bottom. The corresponding relationship is:

• dst(x, y) = src( x + 100, y + 200)

The above expression is incomplete, namely:

• dst(x, y) = src( 1 * x + 0 * y + 100, 0 * x + 1 * y + 200)

Therefore, the value of each element in the conversion matrix M of the opposite drink can be initially determined as:

• $M_{11}=1,M_{12}=0,M_{13}=100,M_{21}=0,M_{22}=1,M_{23}=200$

Transformation matrix M:

• $M=\left[\begin{array}{ccc}1& 0& 100\\ 0& 1& 200\end{array}\right]$

import cv2
import numpy as np

img = cv2.imread('../lena.bmp')
h, w = img.shape[:2]
x = 100
y = 200
M = np.float32([[1, 0, x], [0, 1, y]])
move = cv2.warpAffine(img, M, (w, h))
cv2.imshow('img', img)
cv2.imshow('move', move)
cv2.waitKey()
cv2.destroyAllWindows()

2. Rotate:

​ When using the function cv2.warpAffine() to rotate an image in OpenCV, the transformation matrix can be obtained through the function cv2.getRotationMatrix2D(). The syntax of this function is:

• retval = cv2.getRotationMatrix2D( center, angle, scale )
  • center: the center point of the rotation
  • angle: The angle of rotation, a positive number means counterclockwise rotation, and a negative number means clockwise rotation.
  • scale: transform scale (zoom size)

The transformation matrix M to be used can be directly generated by using the function cv2.getRotationMatrix2D() . For example, if you want to rotate 45 degrees counterclockwise with the center of the image as the origin, and reduce the target image to 0.6 times the original image, the statement when calling the function cv2.getRotationMatrix2D() to generate the transformation matrix M is:

• M = cv2.getRotationMatrix2D( ( height/2, width/2 ), 45, 0.6 )

For example: image rotation based on the above requirements

import cv2

lena = cv2.imread('../lena.bmp')
h, w = lena.shape[:2]
M = cv2.getRotationMatrix2D((h / 2, w / 2), 45, 0.6)
rotate = cv2.warpAffine(lena, M, (w, h))
cv2.imshow('lena', lena)
cv2.imshow('rotate', rotate)
cv2.waitKey()
cv2.destroyAllWindows()

3. More complex affine transformations:

As mentioned above, both translation and rotation affine transformations are relatively simple. For more complex affine transformations, OpenCV provides the function cv2.getAffineTransFORM() to generate the transformation matrix M required by the affine function cv2.warpAffine(). The syntax of this function is:

• retval = cv2.getAffineTransform(src, dst)

In the formula:

• src: represents the coordinates of three points of the input image
• dst: represents the coordinates of three points of the output image

In this function, its parameter values ​​src and dst are arrays containing three two-dimensional arrays (x, y) points. The above parameters define two parallelograms through the function cv2.getAffineTransform(). The three points in src and dst correspond to the three points in the upper left corner, upper right corner, and lower left corner of the parallelogram respectively. After the function cv2.getAffineTransform() completes the mapping of the specified point, the mapping relationship of all other points is calculated and determined according to the relationship of the specified point. The function cv2.warpAffine() takes the transformation matrix M obtained by the function cv2.getAffineTransform() as a parameter, and affines the points in src to dst.

example:

import cv2
import numpy as np

img = cv2.imread('../lena.bmp')
rows, cols, ch = img.shape
p1 = np.float32([[0, 0], [cols - 1, 0], [0, rows - 1]])
p2 = np.float32([[0, rows * 0.33], [cols * 0.85, rows * 0.25], [cols * 0.15, rows * 0.7]])
M = cv2.getAffineTransform(p1, p2)
dst = cv2.warpAffine(img, M, (cols, rows))
cv2.imshow('img', img)
cv2.imshow('dst', dst)
cv2.waitKey()
cv2.destroyAllWindows()

• First, two three-component point sets p1 and p2 are constructed, which are used to refer to the three vertices (upper left corner, upper right corner, and lower left corner) of the parallelogram in the original image and the target image, respectively.
• Then use M=cv2.getAffineTransform(p1,p2) to get the transformation matrix M.
• Next, dst=cv2.warpAffine(img,M,(cols,rows)) completes the affine from the original image to the target image.

four. Perspective:

Affine can map a rectangle to any parallelogram, and perspective transformation can map a rectangle to any quadrilateral.

The function of perspective transformation is cv2.warpPerspective(), and the specific syntax is:

• dst = cv2.warpPerspective( src, M, dsize [, flags [, borderMode [, borderValue]]])
  • dst: represents the output image after perspective processing, which has the same type as the original image. dsize determines the actual size of the output image.
  • src: Represents the image to be perspectived.
  • M: Represents a 3×3 transformation matrix.
  • dsize: represents the size of the output image.
  • flags: represents the interpolation method, the default is INTER_LINEAR. When the value is WARP_INVERSE_MAP, it means that M is an inverse transformation type, which can realize the inverse transformation from the target image dst to the original image src.
  • borderMode: represents the edge type, the default is BORDER_CONSTANT. When the value is BORDER_TRANSPARENT, it means that the values ​​in the target image are not changed, and these values ​​correspond to outliers in the original image.
  • borderValue: represents the boundary value, the default is 0.

Like the affine transformation, a function is also used to generate the transformation matrix used by the function cv2.warpPerspective(). The function is cv2.getPerspectiveTransform(), and the syntax format is:

• restval = cv2.getPerspectiveTransform( src, dst )
  • src: represents the coordinates of the four vertices of the input image
  • dst: represents the coordinates of the four vertices of the output image

It should be noted that the src parameter and dst parameter are arrays containing four points. In actual use, we can control the mapping of the four points in src to the four points in dst as needed.

example:

import cv2
import numpy as np

img = cv2.imread('../demo.bmp')
rows, cols = img.shape[:2]
pts1 = np.float32([[150, 50], [400, 50], [60, 450], [310, 450]])
pts2 = np.float32([[50, 50], [rows-50, 50], [50, cols-50], [rows-50, cols-50]])
M = cv2.getPerspectiveTransform(pts1, pts2)
dst = cv2.warpPerspective(img, M, (cols, rows))
cv2.imshow('img', img)
cv2.imshow('dst', dst)
cv2.waitKey()
cv2.destroyAllWindows()

• Specify the four vertices pts1 of the parallelogram in the original image and pts2 of the four vertices of the rectangle in the target image.
• Use M=cv2.getPerspectiveTransform(pts1,pts2) to generate the transformation matrix M.
• Next, use the statement dst=cv2.warpPerspective(img,M,(cols,rows)) to complete the conversion from parallelogram to rectangle.

five. Remapping:

The process of mapping pixels in one image to specified locations in another image is called remapping. There are many ways of remapping in OpenCV, but sometimes we want to use a custom way to complete the remapping.

​ The remapping function cv2.remap() in OpenCV provides a more convenient and freer mapping method, and its syntax format is as follows:

• dst = cv2.remap( src, map1, map2, interpolation [, borderMode [, borderValue]])

  • dst: represents the target image, which has the same size and type as src.
  • src: represents the original image.
  • map1: The parameter has two possible values:
    • Represents a map of (x,y) points.
    • Indicates the x value of a CV_16SC2, CV_32FC1, CV_32FC2 type (x, y) point.
  • map2: The parameter also has two possible values:
  • When map1 represents (x,y), the value is empty.
  • When map1 represents the x value of (x, y) point, this value is the y value of CV_16UC1, CV_32FC1 type (x, y) point.
  • Interpolation: represents the interpolation method, and the INTER_AREA method is not supported here.
  • borderMode : represents the border mode. When the value is BORDER_TRANSPARENT, it means that the pixels corresponding to the singular points (outliers) in the source image in the target image will not be modified.
  • borderValue: represents the border value, which defaults to 0.

1. Understanding of mapping parameters:

​ **Remapping obtains a new image by modifying the position of the pixels. When constructing a new image, it is necessary to determine the position of each pixel in the new image in the original image. So what the map function does is find where the new image pixels are within the original image. This process is the process of mapping the new image pixels to the original image, so it is called reverse mapping. **In the function cv2.remap(), the parameter map1 and the parameter map2 are used to illustrate the reverse mapping, map1 is for the coordinate x, and map2 is for the coordinate y.

​ It should be noted that the values ​​of map1 and map2 are both floating point numbers. Therefore, the target image can be mapped back to a non-integer value, which means that the target image can be "backmapped" to a position between two pixels in the original image (where, of course, there is no pixel value). At this time, different methods can be used to achieve interpolation, and the interpolation parameter in the function can control the interpolation method. It is precisely because the values ​​of the parameter map1 and the parameter map2 are floating-point numbers, the mapping relationship that can be realized through the function cv2.remamp() becomes more arbitrary, and different forms of mapping can be realized through custom mapping parameters.

​ It should be noted that the parameter map1 in the function cv2.remap() refers to the column number where the pixel is located, and the parameter map2 refers to the row number where the pixel is located. **For example, if we want to map a point A in the target image (mapping result image) to a pixel point B in the original image on row 0 and column 3, then we need to place the parameter map1 corresponding to point A on the corresponding position The value of is set to 3, and the value of the corresponding position of the parameter map2 is set to 0. **So, usually, we write map1 as mapx, and map2 as mapy for easy understanding.

​ Similarly, if you want to map all the pixels in the target image (mapping result image) to the pixel B on the 0th row and the 3rd column in the original image, you need to set the values ​​in the parameter map1 to 3, set The values ​​in the parameter map2 are all set to 0.

example:

• The value in the parameter map1 (mapx) used to specify the column is 3.
• The values ​​in the parameter map2 (mapy) used to specify the row are all 0.

import cv2
import numpy as np

img = np.random.randint(0, 256, size=[4, 5], dtype=np.uint8)
# rows, cols = img.shape
mapx = np.ones(img.shape, np.float32) * 3
mapy = np.ones(img.shape, np.float32) * 0
rst = cv2.remap(img, mapx, mapy, cv2.INTER_LINEAR)
print('img=\n', img)
print('mapx=\n', mapx)
print('mapy=\n', mapy)
print('rst=\n', rst)

# 输出结果
img=
 [[196  22  15  82 189]
 [228 208 215  88 113]
 [159   6 157 101 174]
 [181  95 194 206 111]]
mapx=
 [[3. 3. 3. 3. 3.]
 [3. 3. 3. 3. 3.]
 [3. 3. 3. 3. 3.]
 [3. 3. 3. 3. 3.]]
mapy=
 [[0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]]
rst=
 [[82 82 82 82 82]
 [82 82 82 82 82]
 [82 82 82 82 82]
 [82 82 82 82 82]]

2. Copy:

In order to better understand how to use the remapping function cv2.remap(), the following describes how to copy the image through this function. When mapping, the parameters are processed as follows:

• Set the value of map1 to the coordinate axis of the x-axis at the corresponding position
• Set the value of map2 as the coordinate axis of the y-axis at the corresponding position

Set the value of map1 as the coordinate axis of the x-axis on the drinking position.

example:

import cv2
import numpy as np

img = cv2.imread('../lena.bmp')
rows, cols = img.shape[:2]
mapx = np.zeros(img.shape[:2], np.float32)
mapy = np.zeros(img.shape[:2], np.float32)
for i in range(rows):
    for j in range(cols):
        mapx.itemset((i, j), j)
        mapy.itemset((i, j), i)
rst = cv2.remap(img, mapx, mapy, cv2.INTER_LINEAR)
cv2.imshow('img', img)
cv2.imshow('rst', rst)
cv2.waitKey()
cv2.destroyAllWindows()

3. Rotate around the x-axis:

If you want to rotate around the x-axis during the mapping process:

• The x-axis remains the same
• The values ​​of the y-coordinate axis are exchanged with the x-axis as the axis of symmetry.

Reflected on map1, map2:

• map1 value stays the same
• The value of map2 is adjusted to "total number of lines - 1 - current line number".

It should be noted that the subscript of the line number in OpenCV starts from 0, so there is a relationship of "current line number + symmetrical line number = total number of lines - 1" in the symmetric relationship. According to this, when flipping around the x-axis, the row number of the current row in map2 is adjusted to "total number of rows - 1 - current row number".

import cv2
import numpy as np

img = cv2.imread('../lena.bmp')
rows, cols = img.shape[:2]
mapx = np.zeros(img.shape[:2], np.float32)
mapy = np.zeros(img.shape[:2], np.float32)
for i in range(rows):
    for j in range(cols):
        mapx.itemset((i, j), j)
        mapy.itemset((i, j), rows-1-i)
rst = cv2.remap(img, mapx, mapy, cv2.INTER_LINEAR)
cv2.imshow('img', img)
cv2.imshow('rst', rst)
cv2.waitKey()
cv2.destroyAllWindows()

4. Rotate around the y-axis:

If you want to rotate around the y-axis during the mapping process:

• The y axis remains the same
• The values ​​of the x-coordinate axis are exchanged with the y-axis as the axis of symmetry.

Reflected on map1, map2:

• map2 values ​​remain unchanged
• The value of map1 is adjusted to "total column number - 1 - current column number".

It should be noted that the subscript of the row number in OpenCV starts from 0, so there is a relationship of "current column number + symmetric column number = total number of columns - 1" in the symmetric relationship. Accordingly, when flipping around the y-axis, the column number of the current column in map1 is adjusted to "total number of columns - 1 - current column number".

import cv2
import numpy as np

img = cv2.imread('../lena.bmp')
rows, cols = img.shape[:2]
mapx = np.zeros(img.shape[:2], np.float32)
mapy = np.zeros(img.shape[:2], np.float32)
for i in range(rows):
    for j in range(cols):
        mapx.itemset((i, j), cols - j -1)
        mapy.itemset((i, j), i)
rst = cv2.remap(img, mapx, mapy, cv2.INTER_LINEAR)
cv2.imshow('img', img)
cv2.imshow('rst', rst)
cv2.waitKey()
cv2.destroyAllWindows()

5. Rotate around the x and y axes:

If you want to rotate around the x and y axes during the mapping process:

• The values ​​of the x-coordinate axis are exchanged with the y-axis as the axis of symmetry.
• The values ​​of the y-coordinate axis are exchanged with the x-axis as the axis of symmetry.

Reflected on map1, map2:

• The value of map1 is adjusted to "total column number - 1 - current column number"
• The value of map2 is adjusted to "total number of lines - 1 - current line number".

import cv2
import numpy as np

img = cv2.imread('../lena.bmp')
rows, cols = img.shape[:2]
mapx = np.zeros(img.shape[:2], np.float32)
mapy = np.zeros(img.shape[:2], np.float32)
for i in range(rows):
    for j in range(cols):
        mapx.itemset((i, j), cols - j - 1)
        mapy.itemset((i, j), rows - i - 1)
rst = cv2.remap(img, mapx, mapy, cv2.INTER_LINEAR)
cv2.imshow('img', img)
cv2.imshow('rst', rst)
cv2.waitKey()
cv2.destroyAllWindows()

6. The x and y axes are interchanged:

If you want to swap the x-axis and y-axis of the image, it means that during the mapping process, for any point, you need to swap its x-axis and y-axis coordinates. Reflected on mapx and mapy:

• The value of mapx is adjusted to the line number of the line it is on.
• The value of mapy is adjusted to the column number of the column it is in.

It should be noted that if the number of rows and the number of columns are inconsistent, the above operations may have values ​​that cannot be mapped. By default, values ​​that cannot be mapped are treated as 0.

import cv2
import numpy as np

img = cv2.imread('../lena.bmp')
rows, cols = img.shape[:2]
mapx = np.zeros(img.shape[:2], np.float32)
mapy = np.zeros(img.shape[:2], np.float32)
for i in range(rows):
    for j in range(cols):
        mapx.itemset((i, j), i)
        mapy.itemset((i, j), j)
rst = cv2.remap(img, mapx, mapy, cv2.INTER_LINEAR)
cv2.imshow('img', img)
cv2.imshow('rst', rst)
cv2.waitKey()
cv2.destroyAllWindows()

7. Image scaling:

The previous mappings are all integer mappings directly, which is more convenient to handle. When dealing with more complex problems, it is necessary to perform more complex operations on row and column values.

For example: using cv2.remap() to downscale an image.

After shrinking the image, you can pin the image to a certain area around its center. For example, set its x-axis and y-axis to:

• Generate a reduced image within the x-axis (0.25 x-axis length, 0.75 x-axis length) interval of the target image; the points in the rest of the x-axis are sampled from the value of any point on the x-axis.
• Generate a reduced image within the y-axis (0.25·y-axis length, 0.75·y-axis length) interval of the target image; the points in the rest of the y-axis are sampled from the value of any point on the y-axis.

For the convenience of processing, we let the points not in the above-mentioned area take the value of (0,0) coordinate point.

import cv2
import numpy as np

img = cv2.imread('../11111.png')
print(img.shape)
rows, cols = img.shape[:2]
mapx = np.zeros(img.shape[:2], np.float32)
mapy = np.zeros(img.shape[:2], np.float32)
for i in range(rows):
    for j in range(cols):
        if 0.25 * cols < j < 0.75 * cols and 0.25 * rows < i < 0.75 * rows:
            mapx.itemset((i, j), 2 * (j - cols * 0.25) + 0.5)
            mapy.itemset((i, j), 2 * (i - rows * 0.25) + 0.5)
        else:
            mapx.itemset((i, j), 0)
            mapy.itemset((i, j), 0)
rst = cv2.remap(img, mapx, mapy, cv2.INTER_LINEAR)
cv2.imshow('img', img)
cv2.imshow('rst', rst)
cv2.waitKey()
cv2.destroyAllWindows()