Image processing in OpenCV - image threshold + image smoothing + morphological conversion
Table of contents
1. Image Thresholding
About image threshold mainly involves two functions: cv.threshold and cv.adaptiveThreshold (ie simple threshold and adaptive threshold)
1.1 Simple Threshold
First of all, we need to understand what a threshold is, and what can a threshold do? The simple threshold is a critical value we set. The function of this critical value is to correspond to each pixel in the image. If it is less than this critical value, it is set to 0, and if it is greater than this critical value, it is set to the maximum value. (Generally 255), after using the threshold, there will be only two color pixels left in the image: the maximum value and the minimum value, which are used more in masks, and we will talk about them in detail later
Let's talk about the simple threshold first. The function involved in the simple threshold is cv.threshold(), which needs to pass in 4 parameters. The first parameter is our image object. It should be noted that generally we need to pass in a single Channel grayscale image, the second parameter is the threshold, which is used to classify the pixels of the entire image, and the third parameter is the maximum value assigned, that is, when the pixel is greater than the threshold we set, make it equal to the one we set The maximum value is enough. The fourth parameter is a flag indicating different types. Its value can be: cv.THRESH_BINARY, cv.THRESH_BINARY_INV, cv.THRESH_TRUNC, cv.THRESH_TOZERO, cv.THRESH_TOZERO_INV
Below we use an example to show the results of using these different types of thresholds
import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt
img = cv.imread('gradient.png',0)
ret,thresh1 = cv.threshold(img,127,255,cv.THRESH_BINARY)
ret,thresh2 = cv.threshold(img,127,255,cv.THRESH_BINARY_INV)
ret,thresh3 = cv.threshold(img,127,255,cv.THRESH_TRUNC)
ret,thresh4 = cv.threshold(img,127,255,cv.THRESH_TOZERO)
ret,thresh5 = cv.threshold(img,127,255,cv.THRESH_TOZERO_INV)
titles = ['Original Image','BINARY','BINARY_INV','TRUNC','TOZERO','TOZERO_INV']
images = [img, thresh1, thresh2, thresh3, thresh4, thresh5]
for i in range(6):
plt.subplot(2,3,i+1),plt.imshow(images[i],'gray')
plt.title(titles[i])
plt.xticks([]),plt.yticks([])
plt.show()
In the process of displaying images in the above code, a very important plt.subplot() function appears. This is a drawing function provided by the matplotlib library. Its use method is relatively simple, that is, several images are displayed in the form of rows and columns. Come out, there are three parameters passed in by plt.subplot(), the first and second parameters refer to the number of rows and columns, and the third refers to the position of the displayed picture
plt also provides plt.imshow(), pli.title() and plt.x/yticks() to perfect our image display
1.2 Adaptive Threshold
In the simple threshold, we specify a threshold as the fixed threshold of the whole picture, but sometimes the lighting angle and even the angle of each part of some pictures are different. If we still use the simple threshold at this time, the effect can be imagined
As long as the thinking is not slippery, there are always more solutions than difficulties! At this time, we can use adaptive threshold to solve this problem
The adaptive threshold is related to the function cv.adaptiveThreshold(). The parameters that need to be passed in to this function are a bit complicated. We use an article to understand: image binarization-cv2.threshold(), cv2.adaptiveThreshold ( )
Let's first observe what the cv.adaptiveThredshold() function looks like when it is used
th3 = cv.adaptiveThreshold(img,255,cv.ADAPTIVE_THRESH_GAUSSIAN_C, cv.THRESH_BINARY, 11, 2)
Did you find that these parameters are too small? Let’s look at them one by one. The first parameter is a commonplace, which is our image resource src. The second parameter refers to the upper limit of the pixel value. The third parameter is interesting. It refers to It is an adaptive method, and there are two adaptive methods available to us (I don’t know if there are others, and interested friends can check it out):
- cv2.ADAPTIVE_THRESH_MEAN_C : mean within domain
- cv2.ADAPTIVE_THRESH_GAUSSIAN_C : The weighted sum of pixels in the field, the weight is a Gaussian window
Only two values can be assigned to the fourth parameter: cv2.THRESH_BINARY and cv2.THRESH_BINARY_INV. The fifth parameter Block size refers to the size of the specified area. The larger the value of BlockSize, the larger the area involved in the calculation of the threshold, and the detail outline The less it becomes, the thicker and more obvious the overall outline is. The sixth parameter is a constant C. The larger the C, the smaller the threshold calculated by the N*N neighborhood of each pixel, and the possibility that the center point is greater than this threshold The greater the sensitivity, the greater the probability of setting it to 255, and the more white pixels in the overall image, and vice versa
1.3 Binarization of Otsu
In global thresholding, we use an arbitrarily chosen value as the threshold. In contrast, Otsu's method avoids having to choose a value and automatically determines it
First of all, let's understand what a bimodal image is. As the name implies, it is an image with only two different image values, where the histogram contains only two peaks, and a good threshold should be between these two peaks to achieve the best. Image processing effect, and Otsu's method is to determine the best global threshold from the image histogram (completely different from the adaptive threshold, the adaptive threshold is to automatically determine the threshold corresponding to different regions, and the Otsu method is based on the bimodal image. determine an optimal global threshold)
We still use an example to master these methods
import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt
img = cv.imread(r"E:\image\test03.png", 0)
# 全局阈值
ret1, th1 = cv.threshold(img, 127, 255, cv.THRESH_BINARY)
# Otsu阈值
ret2, th2 = cv.threshold(img, 0, 255, cv.THRESH_BINARY + cv.THRESH_OTSU)
# 高斯滤波后再采用Otsu阈值
blur = cv.GaussianBlur(img, (5, 5), 0)
ret3, th3 = cv.threshold(blur, 0, 255, cv.THRESH_BINARY + cv.THRESH_OTSU)
# 绘制所有图像及其直方图
images = [img, 0, th1,
img, 0, th2,
blur, 0, th3]
titles = ['Original Noisy Image', 'Histogram', 'Global Thresholding (v=127)',
'Original Noisy Image', 'Histogram', "Otsu's Thresholding",
'Gaussian filtered Image', 'Histogram', "Otsu's Thresholding"]
for i in range(3):
plt.subplot(3, 3, i * 3 + 1), plt.imshow(images[i * 3], 'gray')
plt.title(titles[i * 3]), plt.xticks([]), plt.yticks([])
plt.subplot(3, 3, i * 3 + 2), plt.hist(images[i * 3].ravel(), 256)
plt.title(titles[i * 3 + 1]), plt.xticks([]), plt.yticks([])
plt.subplot(3, 3, i * 3 + 3), plt.imshow(images[i * 3 + 2], 'gray')
plt.title(titles[i * 3 + 2]), plt.xticks([]), plt.yticks([])
plt.show()
2. Image smoothing
2.1 2D convolution (image filtering)
Before learning image smoothing, we need to understand 2D convolution, that is, image filtering. We can use various low-pass filters (LPF) and high-pass filters (HPF) to filter images.
Low pass filter LPF helps to remove noise while high pass filter HPF helps to find edges in the image
OpenCV provides a cv.filter2D() function to convolve the kernel with the image. We will focus on removing the "kernel" in the following content, and what kernel to use is the key to realizing various image blurring techniques. Now we provide An example to understand the implementation process of kernel and image convolution, in this example we implement it through a 5x5 averaging filter kernel
Draw is an important image smoothing technique, we will introduce it in detail below
The essence of the code: keep this kernel on one pixel, add all 25 pixels below this pixel, take the average value, replace the center pixel with the new average value, and it will continue to do this for all pixels until processing finished image
import numpy as np
import cv2 as cv
from matplotlib import pyplot as plt
img = cv.imread('opencv_logo.png')
kernel = np.ones((5,5),np.float32)/25
dst = cv.filter2D(img,-1,kernel)
plt.subplot(121),plt.imshow(img),plt.title('Original')
plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(dst),plt.title('Averaging')
plt.xticks([]), plt.yticks([])
plt.show()
We found a strange thing in the above code: kernel, which is a 5x5 array of float32 data type, we can also understand that this is the kernel of our 5x5 average filter, this 5x5 array stores 25 pixels below the kernel pixel, passing this array to cv.filter2D() will achieve the final image smoothing
Next, let's talk about the cv.filter2D() function. It is mentioned above that it is a function used to convolve the image and the kernel, and the kernel is passed in as a parameter by ourselves. There are three must pass in cv.filter2D() Input parameters: src, ddepth and kernel. Of course, src refers to our image resource (original image), and ddepth refers to the depth of the target image. What is our convolution kernel, it is a numpy.ndarraytype of matrix, this matrix can be generated with numpy function, but in the follow-up image processing technology, we need to create some very complicated convolution kernel, at this time using numpy function seems to be Not enough, at this time we need to use OpenCV's built-in functions: getStructuringElement, getGaussianKernel, etc. to meet our needs
2.2 Image smoothing (image blurring)
Image blurring is achieved by convolving the image with a low-pass filter kernel . The low-pass filter LPF is very effective in removing noise. It removes high-frequency parts (such as noise and edges) from the actual picture. Of course it does not affect the edges. Not very friendly, the result of this operation is that the edges are blurred, OpenCV provides four types of blurring techniques
1. Average
We demonstrated above using the 5x5 average filter kernel to implement the operation, but the "average" technique also has its own function for our operation
It only gets the average value of all pixels under the kernel area and replaces the center element. This is done through the function functions **cv.blur() and cv.boxFilter()**. We need to specify the width of the kernel when doing the operation and height
The cv.boxFilter() function is used when we cannot use a standardized box filter, and the parameter normalize = False is passed in
import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt
img = cv.imread('opencv-logo-white.png')
blur = cv.blur(img,(5,5))
plt.subplot(121),plt.imshow(img),plt.title('Original')
plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(blur),plt.title('Blurred')
plt.xticks([]), plt.yticks([])
plt.show()
2. Gaussian blur
Instead of the box filter, Gaussian blur uses a Gaussian kernel for image smoothing, this is done with the function cv.GaussianBlur( ) , we should specify the width and height of the kernel, which should be positive and odd. We should also specify the standard deviation in the X and Y directions, sigmaX and sigmaY respectively
If only sigmaX is specified, sigmaY is the same as sigmaX, and if both are zero, it is calculated according to the kernel size
For some special needs we will use the cv.getGaussianKernel() function to create a Gaussian kernel
# 我们可以通过修改上面的代码实现高斯模糊
blur = cv.GaussianBlur(img,(5,5),0)
The parameter img passed in cv.GaussianBlur() in the above code is the original image, (5,5) is the size of the Gaussian kernel, 0 means that both sigmaX and sigmaY are 0, and its value is calculated according to the kernel size
3. Median fuzzy
The function cv.medianBlur() extracts the median value of all pixels under the kernel area and replaces the central element with this median value, which is very effective in removing salt and pepper noise in the image. In averaging, the central element of the kernel is the newly calculated average value , while the central element of the kernel of the median blur is the pixel value or new value in the image, but the central element is always replaced by some pixels in the median blur
The kernel size for the median blur should also be an integer technical integer
median = cv.medianBlur(img,5)
img refers to the image resource, 5 refers to the kernel size
4. Bilateral filtering
cv.bilateralFilter() is very effective at removing noise while keeping edges clean and sharp, however, this operation is slow compared to other filters
A Gaussian filter takes the neighborhood around a pixel and finds its Gaussian weighted average, the Gaussian filter is simply a function of the controls, i.e. it only considers nearby pixels when it works, regardless of whether the pixels have the same intensity or not Whether it is an edge pixel, so it is not friendly to the sharpness of the edge
But the bilateral filter improves this defect, it has two Gaussian filters inside, one is the general Gaussian filter mentioned above, and the other is a function of pixel difference, the Gaussian function of the space ensures that only the blurring of nearby elements is considered, the intensity Poor Gaussian ensures that only pixels with similar intensities to the center element are considered blurred (i.e. a display is added that if a pixel's intensity is far from the center element, it will not be blurred to keep the edges sharp)
blur = cv.bilateralFilter(img,9,75,75)
3. Morphological transformation
Here we talk about the morphological operations of OpenCV when processing images. These operations include: erosion and expansion, opening and closing operations, morphological gradients, top hat and black hat
First of all, we need to understand what morphological transformation is. Morphological transformation is a simple operation based on the shape of an image. It is usually performed on a binary image and generally requires two inputs: the original image and the structural element or kernel that determines the nature of the operation.
3.1 Erosion and dilation
Erosion : the kernel slides through the image (in 2D convolution), a pixel in the original image (whether it is 1 or 0) is only considered to be 1 if all pixels under the kernel are 1, otherwise it is eroded (become 0)
The result of erosion is that depending on the size of the kernel, all pixels near the border are discarded, so, the thickness or size of the foreground object is reduced, or just the white area in the image is reduced, it helps to remove small white noise
import cv2 as cv
import numpy as np
img = cv.imread('j.png',0)
# 创建内核
kernel = np.ones((5,5),np.uint8)
# 使用侵蚀函数处理图像
erosion = cv.erode(img,kernel,iterations = 1)
plt.subplot(121),plt.imshow(img),plt.title('Original')
plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(erosion),plt.title('Erosion')
plt.xticks([]), plt.yticks([])
plt.show()
Dilation : A pixel element is "1" if at least one pixel under the kernel is "1". So it increases white areas in the image or increases the size of foreground objects, usually in the case of noise removal dilation is followed by erosion because erosion removes white noise but also shrinks objects
dilation = cv.dilate(img,kernel,iterations = 1)
3.2 Opening and closing operations
Opening is just another name for "dilation after erosion" which is useful for removing noise, for this operation we use the function cv.morphologyEx()
The closing operation is the opposite of the opening operation. It expands first and then erodes. It is useful when closing small holes inside foreground objects or small black spots on objects.
opening = cv.morphologyEx(img, cv.MORPH_OPEN, kernel) # 开运算
opening = cv.morphologyEx(img, cv.MORPH_CLOSE, kernel) # 闭运算
We need to pay attention to the parameters here. The first parameter is the original image, and the second parameter refers to the way of changing. cv2.MORPH_OPEN performs open operation, and cv2.MORPH_CLOSE performs close operation.
3.3 Top hat and black hat
The top hat is the difference between the input image and the image opening operation, while the black hat is the difference between the input image and the image closing operation
tophat = cv.morphologyEx(img, cv.MORPH_TOPHAT, kernel) # 顶帽
blackhat = cv.morphologyEx(img, cv.MORPH_BLACKHAT, kernel) # 黑帽
Black hat is the difference between the input image and the closed operation of the image
3.4 Structural elements
With the help of Numpy, we manually created a structuring element (kernel) in the previous example, which is rectangular, but in some cases, we may need an elliptical/circular kernel, so, OpenCV provides function cv.getStructuringElement() , we only need to pass the shape and size of the kernel to get the desired kernel
(Opening and closing operations also include top hat and black hat, which we will elaborate on later)
(Note: For the content of the article, refer to the official Chinese document of OpenCV4.1)
If the article is helpful to you, remember to support it with one click and three links