Chapter 8: Morphological Operations

Morphology, that is, mathematical morphology, is a very important research direction in the process of image processing. The purpose of morphology is to extract the component information in the image, which is usually of great significance for expressing and describing the shape of the image, and is usually the most essential shape feature used in image understanding.

For example, if we handwrite an Arabic numeral, we can obtain its skeleton information through morphological operations during recognition, and only use its skeleton information in specific image operations.

Morphological processing is widely used in many fields such as visual inspection, text recognition, medical image recognition, image compression coding, etc.

Morphological operations mainly include: erosion, expansion, opening operation, closing operation, morphological gradient operation, top hat operation (top hat operation), black hat operation and other operations. Corrosion operation and dilation operation are the basis of morphological operations. Combining corrosion and dilation operations can realize different forms of operations such as opening and closing operations, morphological gradient operations, top hat operations, black hat operations, and hit and miss operations. .

(The purpose of the morphological operation is to extract the necessary component information in the image)

one. Erosion operation:

Corrosion operation is one of the most basic operations in morphology. It can eliminate the boundary points of the image, shrink the image inward along the boundary, and remove the part smaller than the specified structure element. (The structure also becomes the core).

Corrosion is used to "shrink" or "refine the foreground in the binary image, thereby achieving functions such as noise removal and element segmentation.

During the corrosion process, a structural element is usually used to scan the image to be etched pixel by pixel, and the corrosion result is determined according to the relationship between the structural element and the corroded image.

For example, the background color of the entire image in the picture is black, and the foreground object is a white circle. The small dark squares in the upper left corner of the image are structuring elements used to traverse the image. During the corrosion process, the structural element should traverse the entire image pixel by pixel, and determine the pixel value of the pixel corresponding to the central point of the structural element in the corrosion result image according to the relationship between the structural element and the corroded image.

Note : Morphological operations such as erosion operations determine the value pixel by pixel, and the point for each determination is the point corresponding to the center point of the structural element.

The following two images represent two different relationships between structuring elements and foreground colors. The pixel value of the pixel at the position corresponding to the central point of the structural element in the corrosion result image is determined according to these two different relationships.

If the structure element is completely in the foreground image, the pixel value of the pixel in the corrosion result image corresponding to the center point of the structure element is processed as the foreground color (white, the pixel value of the pixel point is 1)
If the structural element is not completely in the foreground image (it may be partially or completely absent), the pixel value of the pixel point in the corrosion result graph corresponding to the structural element center point is processed as the background color (black, the pixel value of the pixel point is 0).

The result of corrosion is that the diameter of the white circle in the foreground becomes smaller. Structural elements are also called cores.

Corrosion function:

In OpenCV, the cv2.erode() function is used to implement the erosion operation, and its syntax format is:

dst = cv2.erode(src, kernel[, anchor[, iterations[, borderType[, borderValue]]]])

dst: The output result image after erosion, which has the same type and size as the original image.
src: the original image, that is, the original image that needs to be corroded, and the number of channels of the image can be arbitrary. However, it is required that the depth of the image must be one of CV_8U, CV_16U, CV_16S, CV_32F, and CV_64F.
kernel: represents the structure type used in the corrosion operation. It can be custom generated, or generated from cv2.getStructutingElement() via the function.
anchor: Represents the position of the anchor point in the element structure. The value defaults to (-1, -1), at the center of the kernel.
iterations: The number of corrosion operation iterations, the value is 1 by default, that is, only one corrosion operation is performed.
borderType: represents the border style, generally using its default value BORDER_CONSTANT. The specific value of this item is as follows
borderValue: border value, generally using the default value. The function morphologyDefaultBorderValue() is provided in C++ to return the "magic" boundary value of erosion and expansion, which is not supported by python.

Example 1:

import cv2
import numpy as np

img = cv2.imread('../erode.bmp')
kernel = np.ones((5, 5), np.uint8)
rst = cv2.erode(img, kernel)
cv2.imshow('img', img)
cv2.imshow('rst', rst)
cv2.waitKey()
cv2.destroyAllWindows()

Example 2: Modifying the number of corrosion iterations observed

import cv2
import numpy as np

img = cv2.imread('../erode.bmp')
kernel = np.ones((5, 5), np.uint8)
rst = cv2.erode(img, kernel, iterations=5)
cv2.imshow('img', img)
cv2.imshow('rst', rst)
cv2.waitKey()
cv2.destroyAllWindows()

two. Expansion:

The dilation operation is another fundamental operation in morphology. **The expansion operation and the corrosion operation are opposite, and the expansion operation can expand the boundary of the image. **The expansion operation merges the pixels in the background that are in contact with the foreground object into the foreground object, thereby expanding the image boundary points outward. If two objects in the image are close together, the two objects may become connected during the dilation process. Dilation is very helpful to fill the gaps in the image after image segmentation.

Same as the erosion process, in the expansion process, a structural element (kernel) is used to scan the expanded image pixel by pixel, and the expansion result is determined according to the relationship between the structural element and the expanded image.

As shown in FIG:

If any point in the structural element is in the foreground image, the corresponding pixel in the dilation result image is processed as the foreground color.
If the structural element is completely outside the background image, the corresponding pixel in the dilation result image is processed as the background color.

Expansion operation:

In Opencv, the function cv2.dilate() is used to realize the expansion operation of the image, and its grammatical structure is:

dst=cv2.dilate（src,kernel[,anchor[,iterations[,borderType[,borderValue]]]]）

dst: the output result image after dilation, which has the same type and size as the original image.
src: the original image, that is, the original image that needs to be expanded, and the number of channels of the image can be arbitrary. However, it is required that the depth of the image must be one of CV_8U, CV_16U, CV_16S, CV_32F, and CV_64F.
kernel: represents the structure type used in the expansion operation. It can be custom generated, or generated from cv2.getStructutingElement() via the function.

The parameters kernel, anchor, iterations, borderType, borderValue have the same meaning as the corresponding parameters in the function cv2.erode().

Example 1:

import cv2
import numpy as np

img = np.zeros((5, 5), np.uint8)
img[2:3, 1:4] = 1
kernel = np.ones((3, 1), np.uint8)
dilation = cv2.dilate(img, kernel)
print('img=\n', img)
print('kernel=\n', kernel)
print('dilation=\n', dilation)

# 输入结果
img=
 [[0 0 0 0 0]
 [0 0 0 0 0]
 [0 1 1 1 0]
 [0 0 0 0 0]
 [0 0 0 0 0]]
kernel=
 [[1]
 [1]
 [1]]
dilation=
 [[0 0 0 0 0]
 [0 1 1 1 0]
 [0 1 1 1 0]
 [0 1 1 1 0]
 [0 0 0 0 0]]

Example 2:

import cv2
import numpy as np

img = cv2.imread('../dilate.bmp')
kernel = np.ones((3, 3), np.uint8)
rst = cv2.dilate(img, kernel, iterations=9)
cv2.imshow('img', img)
cv2.imshow('rst', rst)
cv2.waitKey()
cv2.destroyAllWindows()

three. Generic morphology functions:

Corrosion operation and expansion operation are the basis of morphological operations. Combining erosion and expansion operations can realize opening operation, closing operation (close operation), morphological gradient (Morphological Gradient) operation, top hat operation (top hat operation) , black hat calculation, hit and miss and many other different forms of calculation.

OpenCV provides the function cv2.morphologyEx() to realize the above morphological operations, and its grammatical structure is as follows:

dst=cv2.morphologyEx（src,op,kernel[,anchor[,iterations[,borderType[,borderValue]]]]]）
式中：

dst: represents the output target image after morphological processing, which has the same type and size as the original image.
src: represents the original image that needs to be morphologically operated. The number of channels of the image can be arbitrary, but the depth of the image must be one of CV_8U, CV_16U, CV_16S, CV_32F, and CV_64F.
op: represents the operation type, as shown in the following table. The operation rules of various morphological operations are obtained by combining 腐蚀and operations.膨胀
The parameters kernel, anchor, iterations, borderType, borderValue have the same meaning as the corresponding parameters in the function cv2.erode().

four. Open operation:

The operation performed by the open operation is to corrode the image first, and then dilate the corroded result. The opening operation can be used for denoising, counting, etc.

For example, in the figure below, 先腐蚀后膨胀denoising is achieved through the opening operation, where:

The left image is the original image.
The middle plot is the result of erosion on the original image.
The image on the right is the result of dilating the corroded image, that is, the result of opening the original image.

In addition, the opening operation can also be used for counting. For example, before counting the areas in the figure below, you can use the open operation to divide the different areas connected together, where:

The left image is the original image.
The middle plot is the result of erosion on the original image.
The image on the right is the result of dilating the corroded image, that is, the result of opening the original image.

The opening operation can be realized by setting the operation type parameter op in the function cv2.morphologyEx() to "cv2.MORPH_OPEN". Its grammatical structure is as follows: opening=cv2.morphologyEx(img,cv2.MORPH_OPEN,kernel)

Example:

import cv2
import numpy as np

img1 = cv2.imread('../erode.bmp')
img2 = cv2.imread('../cube.bmp')

k = np.ones((20, 20), np.uint8)
rst1 = cv2.morphologyEx(img1, cv2.MORPH_OPEN, k)
rst2 = cv2.morphologyEx(img2, cv2.MORPH_OPEN, k)

cv2.imshow('img1', img1)
cv2.imshow('img2', img2)
cv2.imshow('rst1', rst1)
cv2.imshow('rst2', rst2)
cv2.waitKey()
cv2.destroyAllWindows()

five. Closing operation:

Closing operation is 先膨胀、后腐蚀an operation that helps to close the small holes inside the foreground object, or remove the small black spots on the object, and can also connect different foreground images.

For example, the small holes inside the original image are removed by the closing operation of expansion and then erosion (closed operation of internal closure), where:

The left image is the original image.
The middle plot is the result of dilation of the original image.
The image on the right is the result of corroding the dilated image, that is, the result of closing the original image.

In addition, the closing operation can also realize the connection of the foreground image. For example, using the closing operation to connect the originally independent two parts of the foreground image together, where:

The left image is the original image.
The middle plot is the result of dilation of the original image.
The image on the right is the result of corroding the dilated image, that is, the result of closing the original image.

Example:

import cv2
import numpy as np

k = np.ones((10, 10), np.uint8)
rst1 = cv2.morphologyEx(img1, cv2.MORPH_CLOSE, k)
cv2.imshow('img1', img1)
cv2.imshow('rst1', rst1)
cv2.waitKey()
cv2.destroyAllWindows()

six. Morphological gradient operation:

Morphological gradient operation is the operation of subtracting and corroding the image by using the expansion image of the image, which can obtain the edge of the foreground image in the original image.

For example:

Example:

import cv2
import numpy as np

img = cv2.imread('../erode.bmp')

k = np.ones((5, 5), np.uint8)
rst = cv2.morphologyEx(img, cv2.MORPH_GRADIENT, k)
cv2.imshow('img', img)
cv2.imshow('rst', rst)
cv2.waitKey()
cv2.destroyAllWindows()

seven. top hat operation:

The top hat operation is the operation of 原始图像subtracting it 开运算图像. The top hat operation can obtain the noise information of the image, or get the edge information that is brighter than the edge of the original image.

For example

The left image is the original image.
The image in the middle is the image of the opening operation.
The image on the right is the top hat image obtained by subtracting the calculated image from the original image.

Example:

import cv2
import numpy as np

img1 = cv2.imread('../erode.bmp')
img2 = cv2.imread('../lena.bmp')

k = np.ones((5, 5), np.uint8)
rst1 = cv2.morphologyEx(img1, cv2.MORPH_TOPHAT, k)
rst2 = cv2.morphologyEx(img2, cv2.MORPH_TOPHAT, k)
cv2.imshow('img1', img1)
cv2.imshow('img2', img2)
cv2.imshow('rst1', rst1)
cv2.imshow('rst2', rst2)
cv2.waitKey()
cv2.destroyAllWindows()

eight. Black hat operation:

Black hat operation is the operation of 闭运算图像subtraction 原始图像. The black hat operation can obtain small holes inside the image, or small black dots in the foreground color, or obtain edge parts that are darker than the edges of the original image.

For example:

The left image is the original image.
The middle image is the closed operation image.
The image on the right is the black hat image obtained by subtracting the original image from the closed image.

Example:

import cv2
import numpy as np

img1 = cv2.imread('../noise2.bmp')
img2 = cv2.imread('../lena.bmp')

k = np.ones((10, 10), np.uint8)
rst1 = cv2.morphologyEx(img1, cv2.MORPH_BLACKHAT, k)
rst2 = cv2.morphologyEx(img2, cv2.MORPH_BLACKHAT, k)
cv2.imshow('img1', img1)
cv2.imshow('img2', img2)
cv2.imshow('rst1', rst1)
cv2.imshow('rst2', rst2)
cv2.waitKey()
cv2.destroyAllWindows()

night. kernel function:

When performing morphological operations, a specific kernel (structural element) must be used. The core can be custom generated, or can be constructed by the function cv2.getStructuringElement(). The function cv2.getStructuringElement() can construct and return a structuring element used for morphological processing. The syntax of this function is:

retval=cv2.getStructuringElement（shape,ksize[,anchor]）

This function is used to return a structuring element of the specified size and shape for morphological operations. The meanings of the parameters in the function are as follows.

shape represents the shape type, and its possible values are shown in the table.

ksize represents the size of the structure element.
anchor represents the anchor position in the structural element. The default value is (-1,-1), which is the center of the shape. Only the cross star shape is closely related to the anchor position. In other cases, the anchor position is only used for adjustment of the result of the morphological operation.

Of course, in addition to using this function, users can also construct arbitrary binary masks as structural elements used in morphological operations.

Example 1: Generate kernels of different structures using the function cv2.getStructuringElement().

import cv2

k1 = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 5))
k2 = cv2.getStructuringElement(cv2.MORPH_CROSS, (5, 5))
k3 = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))

print('k1=\n', k1)
print('k2=\n', k2)
print('k3=\n', k3)


# 输出结果
k1=
 [[1 1 1 1 1]
 [1 1 1 1 1]
 [1 1 1 1 1]
 [1 1 1 1 1]
 [1 1 1 1 1]]
k2=
 [[0 0 1 0 0]
 [0 0 1 0 0]
 [1 1 1 1 1]
 [0 0 1 0 0]
 [0 0 1 0 0]]
k3=
 [[0 0 1 0 0]
 [1 1 1 1 1]
 [1 1 1 1 1]
 [1 1 1 1 1]
 [0 0 1 0 0]]

Example 2: Morphological operations with different kernels

import cv2

img = cv2.imread('../round.bmp')

k1 = cv2.getStructuringElement(cv2.MORPH_RECT, (25, 25))
k2 = cv2.getStructuringElement(cv2.MORPH_CROSS, (25, 25))
k3 = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (25, 25))

rst1 = cv2.erode(img, k1, iterations=3)
rst2 = cv2.erode(img, k2, iterations=3)
rst3 = cv2.erode(img, k3, iterations=3)
cv2.imshow('img', img)
cv2.imshow('rst1', rst1)
cv2.imshow('rst2', rst2)
cv2.imshow('rst3', rst3)
cv2.waitKey()
cv2.destroyAllWindows()