Abstract: Image smoothing methods commonly used to eliminate noise include three linear filters (mean filter, box filter, Gaussian filter) and two nonlinear filters (median filter, bilateral filter). This article will explain the three linear filter methods in detail. .
This article is shared from Huawei Cloud Community " [Python from Zero to One] 55. Image Enhancement and Operation: Image Smoothing (Mean Filtering, Box Filtering, Gaussian Filtering)" , author: eastmount.
Image smoothing methods commonly used to eliminate noise include three linear filters (mean filter, box filter, Gaussian filter) and two nonlinear filters (median filter, bilateral filter). This article will explain the three linear filter methods in detail.
1. Image smoothing
Image smoothing is a simple and frequently used image processing method, which can be used to suppress, weaken or eliminate details, sudden changes, edges and noise in the image, and is most commonly used to reduce the noise on the image [1]. What is Image Noise? Noise is a factor that hinders the understanding of source information received by human sensory organs. It is an unpredictable random error that can only be recognized by the method of probability and statistics. From Figure 1, we can observe the characteristics of noise: random position, irregular size, this kind of noise is called random noise, which is a common type of noise.
Figure 2 is an example of image smoothing. The left half of the figure is the original input image containing noise, and the right half is the image after image smoothing. It is easy to observe by comparison that in the smoothed image, the noise in the object is effectively suppressed and eliminated, but the edge of the flower is blurred, which suppresses the redundant information in the image, that is, the flower The process of noise removal is called image smoothing [2].
An image is inevitably disturbed by various noise sources, so noise filtering is often the first step in image processing. The filtering effect will directly affect the subsequent processing results. Noise filtering plays a very important role in image processing. status. There are many kinds of noise filtering algorithms, which can be divided into two categories: linear filtering algorithms and nonlinear filtering algorithms in terms of design methods.
(1) Linear filtering
In image processing, when the calculation of the pixels in the neighborhood is a linear operation, such as using a window function to perform a smooth weighted sum operation, or a certain convolution operation, it can be called a linear filter. In the early research of digital signal processing and digital image processing, linear filters are the main means of noise suppression processing, such as mean filtering, box filtering, Gaussian filtering, etc.
The linear filtering algorithm has a better filtering effect on Gaussian noise, and when the signal spectrum and the noise spectrum are aliased or when the signal contains non-superimposed noise (such as noise caused by system nonlinearity or non-Gaussian noise, etc.) , the processing result of the linear filter is hardly satisfactory.
(2) Nonlinear filtering
Nonlinear filtering uses a logical relationship between the original image and the template to obtain results, such as median filtering and bilateral filtering. Nonlinear filtering technology makes up for the deficiency of linear filtering method to some extent, because it can keep the high-frequency details of the image signal well while filtering out the noise, so it is widely used. The famous scholar Tukey [3] first proposed a nonlinear filter - the median filter in 1971, which opened the prelude to the study of nonlinear filtering methods. With the development of nonlinear filtering technology up to now, improved algorithms based on median filtering emerge in an endless stream, occupying an important position in nonlinear filtering algorithms. In addition, many new nonlinear filtering algorithms have emerged one after another, such as filtering methods based on mathematical morphology, filtering methods based on fuzzy theory, filtering methods based on neural networks, etc., which provide new ideas for image filtering technology [4-5] .
Some commonly used filters will be introduced later in detail, including mean filter, box filter, Gaussian Lubo, median filter, etc., as shown in Table 23-1.
Figure 3 shows the comparison of the effects of these five filters. From the results of the filters, it can be seen that various filter algorithms have very different effects on the image, some of which have changed greatly, and some are even the same as the original image. In practical applications, appropriate filters should be selected according to the characteristics of noise, desired image and edge features, so as to maximize the advantages of image filtering.
In the process of image generation, transmission and reproduction, it is often disturbed by noise or data loss due to various reasons, which reduces the quality of the image. This requires a certain enhancement of the image to reduce the impact of these defects [6].
2. Mean filtering
Mean filtering is the simplest linear filtering algorithm, which refers to giving a template to the target pixel on the original image, which includes the surrounding adjacent pixels (8 pixels around the target pixel as the center, forming a filter template, that is, remove the target pixel itself), and then replace the original pixel value with the average value of all pixels in the template. In other words, each pixel value of the mean filter output image is the weighted average of M×M pixel values around it.
Fig. 4 shows the process of the mean value filtering process, the pixel value of the center red point is the mean value of the sum of the pixel values of the blue background area. The 5×5 matrix is called the fuzzy kernel. For the pixels in the original image, the mean value filter uses the kernel to perform mean value processing on its pixels one by one, and obtains the final rendering.
The average filtering process of the pixel value in the red area is as follows:
The mean filtering algorithm is relatively simple, the calculation speed is fast, and it has a good suppression effect on the periodic interference noise, but it cannot protect the details of the image very well. While denoising the image, it also destroys the details of the image. Thus making the image blurry.
Python calls the cv2.blur() function in OpenCV to implement mean filtering. The function prototype is shown below. The output dst image has the same size and type as the input image src.
dst = blur(src, ksize[, dst[, anchor[, borderType]]])
- src represents the input image, it can have any number of channels, but the depth should be CV_8U, CV_16U, CV_16S, CV_32F or CV_64F
- ksize indicates the size of the blur kernel, presented in the form of (width, height)
- anchor indicates the anchor point, that is, the point to be smoothed, and its default value Point (-1, -1) indicates that it is located in the center of the kernel, which can be omitted
- borderType indicates the border mode, which is used to infer a certain border mode of the pixels outside the image. The default value is BORDER_DEFAULT, which can be omitted
Common blur kernels include (3, 3) and (5, 5), as shown in formulas (2) and (3):
The Python implementation code for image mean filtering is shown below. It should be noted that the code uses a 3×3 template, and plt.rcParams is used to set the normal display of Chinese characters.
# -*- coding: utf-8 -*-
# By:Eastmount
import cv2
import numpy as np
import matplotlib.pyplot as plt
#读取图片
img = cv2.imread('lena-zs.png')
source = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
#均值滤波
result = cv2.blur(source, (3,3))
#用来正常显示中文标签
plt.rcParams['font.sans-serif']=['SimHei']
#显示图形
titles = ['原始图像', '均值滤波']
images = [source, result]
for i in range(2):
plt.subplot(1,2,i+1), plt.imshow(images[i], 'gray')
plt.title(titles[i])
plt.xticks([]),plt.yticks([])
plt.show() The output result of the "lena" image is shown in Figure 5. The left side shows the original image to be processed with noise, and the right side shows the image after the mean filtering process. The salt and pepper noise in the image has been removed.
If the noise in the image still exists, you can increase the size of the blur kernel, such as using a 5×5, 10×10, or even a 20×20 template. Figure 6 is using a 10×10 kernel, but the processed image will gradually become more blurred.
Image mean filtering is to smooth the image through the blur kernel. Since each weight value in the blur kernel is the same, it is called mean. This method eliminates the noise in the original image to a certain extent and reduces the contrast of the original image, but it also has certain defects. It reduces the noise and makes the image blurred, especially at the edges and details, and the blur kernel is more blurred. The larger the value, the more severe the blur.
3. Box filtering
Image smoothing uses convolution templates to process each pixel in the image one by one. This process can be vividly compared to filtering and sorting the pixels of the original image. The algorithm process of processing neighboring pixels one by one is called a filter. Common linear filters include mean filtering and box filtering.
Box filtering, also known as box filtering, uses convolution operations to average the pixel values of the image neighborhood, thereby eliminating noise in the image. The fuzzy kernel of the box filter is basically the same as that of the mean filter, and the difference is whether it needs to be homogenized.
Python calls the cv2.boxFilter() function in OpenCV to implement box filtering. The function prototype is as follows:
dst = boxFilter(src, depth, ksize[, dst[, anchor[, normalize[, borderType]]]])
- src represents the input image
- dst represents the output image, which has the same size and type as the input image
- Depth indicates the depth of the output image, usually set to "-1", indicating that it is consistent with the depth of the original image
- ksize indicates the size of the blur kernel, presented in the form of (width, height)
- normalize indicates whether to normalize the target image, the default value is true
- anchor indicates the anchor point, that is, the point to be smoothed, and its default value Point (-1, -1) indicates that it is located in the center of the kernel, which can be omitted
- borderType indicates the border mode, which is used to infer a certain border mode of the pixels outside the image. The default value is BORDER_DEFAULT, which can be omitted
Common fuzzy kernel ksizes include (3, 3) and (5, 5), as follows:
The parameter normalize indicates whether to normalize the target image.
- (1) When normalize is true, normalization processing needs to be performed, and box filtering becomes mean filtering. Among them, normalization is to scale the pixel values to be processed into a range for unified processing and intuitive quantification.
- (2) When normalize is false, it means non-normalized box filtering, without mean value processing, in fact, it is to find the sum of the surrounding pixels. But at this time overflow is easy to occur, the pixel value after the addition of multiple pixel values is greater than 255, and the pixel value after overflow is set to 255, that is, white.
The definition of parameter normalize is shown in formula (6).
The Python implementation code for image box filtering is shown below. A 3×3 kernel is used in the code, and normalize=0 means no image normalization processing is performed.
# -*- coding: utf-8 -*-
# By:Eastmount
import cv2
import numpy as np
import matplotlib.pyplot as plt
#读取图片
img = cv2.imread('lena-zs.png')
source = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
#方框滤波
result = cv2.boxFilter(source, -1, (3,3), normalize=0)
#用来正常显示中文标签
plt.rcParams['font.sans-serif']=['SimHei']
#显示图形
titles = ['原始图像', '方框滤波']
images = [source, result]
for i in range(2):
plt.subplot(1,2,i+1), plt.imshow(images[i], 'gray')
plt.title(titles[i])
plt.xticks([]),plt.yticks([])
plt.show()The output result of box filtering and non-normalization processing is shown in Figure 7. The processed effect image contains many white pixels, which is because the sum of image pixels overflows (more than 255). It can be seen that when non-normalized processing is performed, the resulting image contains too much white, and the damage to the source image is too great.
If a 2×2 blur kernel is set, the non-normalized box filtering effect is better, as shown in Figure 23-8. The core code is:
- cv2.boxFilter(source, -1, (2,2), normalize=0)
The following code uses a 3×3 kernel to perform normalized box filtering, and the output result is exactly the same as the 3×3 kernel mean filter.
# -*- coding: utf-8 -*-
# By:Eastmount
import cv2
import numpy as np
import matplotlib.pyplot as plt
#读取图片
img = cv2.imread('lena-zs.png')
source = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
#方框滤波
result = cv2.boxFilter(source, -1, (3,3), normalize=1)
#用来正常显示中文标签
plt.rcParams['font.sans-serif']=['SimHei']
#显示图形
titles = ['原始图像', '方框滤波']
images = [source, result]
for i in range(2):
plt.subplot(1,2,i+1), plt.imshow(images[i], 'gray')
plt.title(titles[i])
plt.xticks([]),plt.yticks([])
plt.show()The output result is shown in Figure 9:
Four. Gaussian filtering
In order to overcome the shortcomings of image blur caused by local average method, some local smoothing algorithms that preserve edge details have been proposed. Image Gaussian filtering (Gaussian smoothing) is such an algorithm. It is a linear smoothing filter that applies the idea of neighborhood averaging to smooth images. It is very effective for suppressing noise that obeys a normal distribution. It is suitable for eliminating Gaussian noise and is widely used in the noise reduction process of image processing.
The image Gaussian filter assigns different weights to the pixel values at different positions of the image. The closer the distance is, the greater the weight is, and the farther the distance is, the smaller the weight is. It is different from box filtering and mean filtering. When it averages the pixels in the neighborhood, it assigns different weights to pixels at different positions. In layman's terms, Gaussian filtering is a process of weighted averaging of the entire image, and the value of each pixel is obtained by weighted averaging of itself and other pixel values (with different weights) in the neighborhood.
The following are Gaussian filter templates for commonly used 3×3 and 5×5 kernels.
Gaussian filtering introduces the Gaussian function (normal distribution function) in mathematics. A two-dimensional Gaussian function is shown in the following formula (9), where σ is the standard deviation. In the Gaussian weighted average, the most important thing is the selection of σ, and the standard deviation represents the degree of data dispersion. If σ is small, the central area of the Gaussian distribution will be more concentrated and the smoothing effect will be worse; conversely, if σ is large, the central area of the Gaussian distribution will be More discrete, the smoothing effect is more obvious [10].
The core idea of Gaussian filtering is to discretize the Gaussian function, take the Gaussian function on the discrete point as the weight, and perform a weighted average within a certain range of neighborhoods for each pixel in the image, thereby effectively eliminating Gaussian noise. Gaussian filtering makes the pixels near the center more important, and calculates the weighted average value for the surrounding pixels, as shown in Figure 10, and the weight of the center position is up to 0.4.
OpenCV in Python mainly calls the GaussianBlur() function to achieve Gaussian smoothing. The function prototype is as follows:
dst = GaussianBlur(src, ksize, sigmaX[, dst[, sigmaY[, borderType]]])
- src indicates the input image to be processed
- dst represents the output image, which has the same size and type as the input image
- ksize represents the Gaussian filter template size, ksize.width and ksize.height can be different, but they must be positive and odd, they can also be zero, ie (0, 0)
- sigmaX represents the Gaussian kernel standard deviation of the Gaussian kernel function in the X direction
- sigmaY represents the Gaussian kernel standard deviation of the Gaussian kernel function in the Y direction. Set equal to sigmaX if sigmaY is zero, or calculated from ksize.width and ksize.height respectively if both sigmas are zero
- borderType indicates the border mode, which is used to infer a certain border mode of the pixels outside the image. The default value is BORDER_DEFAULT, which can be omitted
The following code uses a 7×7 kernel template for Gaussian filtering.
# -*- coding: utf-8 -*-
# By:Eastmount
import cv2
import numpy as np
import matplotlib.pyplot as plt
#读取图片
img = cv2.imread('lena-zs.png')
source = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
#高斯滤波
result = cv2.GaussianBlur(source, (7,7), 0)
#用来正常显示中文标签
plt.rcParams['font.sans-serif']=['SimHei']
#显示图形
titles = ['原始图像', '高斯滤波']
images = [source, result]
for i in range(2):
plt.subplot(1,2,i+1), plt.imshow(images[i], 'gray')
plt.title(titles[i])
plt.xticks([]),plt.yticks([])
plt.show()The output result is shown in Figure 11, the image to be processed is on the left, and the image after Gaussian filtering is on the right.
Figure 12 is an effect diagram of Gaussian filtering using a 15×15 Gaussian kernel template. It can be seen from the figure that the image becomes more blurred while removing noise.
In short, Gaussian filtering is one of the most effective filters, and it is very effective for suppressing noise that obeys a normal distribution.
5. Summary
This article mainly explains the image smoothing methods commonly used to eliminate noise. Common methods include three linear filters (mean filter, box filter, Gaussian filter) and two nonlinear filters (median filter, bilateral filter). This article introduces mean filtering, box filtering and Gaussian filtering. Through the comparison of principles and codes, the advantages and disadvantages of various filtering methods are described, which effectively eliminates the noise of the image and preserves the edge contour of the image.
references:
- [1] Written by Gonzales, translated by Ruan Qiuqi. Digital Image Processing (3rd Edition) [M]. Beijing: Electronic Industry Press, 2013.
- [2] zhu_hongji. [OpenCV Study Notes] Image Smoothing (Linear/Nonlinear Filter) [EB/OL]. (2018-08-11). https://blog.csdn.net/zhu_hongji/article/details /81479571.
- [3] Lu Yao. Image processing and image smoothing of matlab examples (1) [EB/OL]. (2017-07-23). https://www.cnblogs.com/luyaoblog/p/7160948.html.
- [4] Ruan Qiuqi. Digital Image Processing (3rd Edition) [M]. Beijing: Electronic Industry Press, 2008.
- [5] Shi Zhengang. Research on Image Processing Algorithm Based on Fuzzy Logic [D]. Northeastern University, 2009.
- [6] Ma Guanghao. Research on Image Smoothing Algorithm Based on Sparse High Frequency Gradient and Joint Bilateral Filtering [D]. Shandong University, 2018.
- [7] Chen Chuxia. Research on Image Filtering and Edge Detection and Enhancement Technology [D]. Hefei University of Technology, 2009.
- [8] Mao Xingyun, Leng Xuefei. Introduction to OpenCV3 Programming [M]. Beijing: Electronic Industry Press, 2015.
- [9] Eastmount. [Python image processing] 4. Mean filter, box filter, Gaussian filter and median filter for image smoothing [EB/OL]. (2018-09-02). https://blog.csdn. net/Eastmount/article/details/82216380.
- [10]Eastmount. [Digital Image Processing] Seven. MFC image enhancement: image ordinary smoothing, Gaussian smoothing, Laplacian, Sobel, Prewitt sharpening detailed explanation [EB/OL]. (2015-06-08). https://blog .csdn.net/eastmount/article/details/46378783.
Click to follow and learn about Huawei Cloud's fresh technologies for the first time~