The previous article talked about the grayscale transformation of the image, which is mainly used in single-point processing to enhance the contrast and brightness of the image. This time, we will introduce the spatial filtering of the image, which uses the pixel information in the area around the pixel to process the grayscale of the image. The main purposes of processing are as follows:

Denoising: During the imaging process, due to hardware sensors or imaging environment factors, the images obtained by the camera often have noise. In order to reduce the noise, it is necessary to design a suitable image filtering algorithm.

Feature extraction: In order to extract features, line features or edge features from images, these need to have corresponding appropriate filters.

Image enhancement: In some applications, it is necessary to enhance the details of the image, such as sharpening, to make the edges of the image sharper.

convolution filter

Before talking about the specific algorithm, let's look at one of the most basic convolution filters. Taking the Source input image as an example, there is also a 3*3 convolution kernel, and the input image is usually much larger than the size of the convolution kernel. The size of the convolution kernel is not necessarily 3*3, it can also be 5*5, 7*7 is possible. With an original image and a convolution kernel, for a 3*3 convolution kernel, it is simple to directly cover the [0,0] position and start convolving the pixel at the center [1,1] position, convolution The elements of the kernel and the elements of the image are multiplied point by point and then accumulated, then it can be expressed by the following formula.

Convolution filtering is the basis of spatial filtering. There are various filters below, such as smoothing filters, sharpening filters, and filters for extracting image edges, all of which are implemented by convolution filtering. The main difference between these filters with different functions is that their convolution kernels are different.

Convolution filtering is a local operation on the image. Usually, it can be accelerated in parallel in the calculation of a specific platform. Compared with the convolutional neural network, the parameters of the convolution filtering kernel learned here are manually designed. is ok. In the convolutional neural network, the operation process is the same, but the parameters of the convolution kernel are inferred from the training network.

mean filtering

Next, let's introduce the simplest filter-mean filter. The so-called mean filter is in the weight of the convolution kernel, all the weights are the same, as shown in the figure, the sum of all the weights is 1. On average, each item is 1/9

均值滤波器的作用是让这个图像变得更加平滑，但是需要注意的是：在通常情况下，均值滤波是各项同性的滤波器，但它在特的输入源的情况下，有可能会出现各项异性的效果。各项异性是什么意思呢？就说在这个图像的倾斜方向的结构上，或者说水平方向或者垂直方向的结构上，在应用同一个均值滤波器的时候，会经历不一样的平滑程度的处理。

以右上举例，图 (b)为输入原图，白条纹是一个单位，黑条纹是二个单位，共三个单位为一组间隔。应用一个3*3的均值滤波以后，它得到的输出是图(c)，这个是符合预期黑白混合成灰。

当输入图像变为图(d)白条纹是一个单位，黑条纹也是一个单位，也是运用3*3的均值滤波的时候，得到的输出是图(e)所示。注意到输入的黑白条纹发生了一个反转，这个明显就是在预期之外了。

有关各项异性和各项同性的介绍，可以再看看这篇文章。

高斯滤波

接着介绍高斯滤波，和均值滤波相比，高斯滤波是各项同性的，不存在特例。上图左边是一维的高斯函数，右边是二维的高斯函数。它是一个非常平滑的曲线/曲面，其中详细的高斯卷积核的权重求解可以参考之前写过的文章，这里不再详细论述。

当我们高斯滤波的时候，一个重要的参数就说它的滤波器的尺度sigma σ ，当sigma取值越小，高斯函数的输出越尖锐的一个曲线，当sigma的取值越大的时候，高斯曲线就越平坦，平滑的力度就越大。上面是一个实际的对比图，随着sigma的不断加大，它的平滑力度会越大，图像的细节已经完全看不到了。

双边滤波

那么上面介绍的高斯滤波是一个最理想的平滑滤波器，它的作用是对图像进行平滑去噪。那么有一个问题就是，高斯滤波的各项同性特性，在它平滑图像的同时会破坏图像的边缘性，即丢失了图像中边缘结构的信息。这其实是我们不希望的，譬如一幅人脸的图像，脸上有些痘痘，我们是希望用高斯滤波去掉这些痘痘，同时希望图像的人脸信息还是清晰的。那么在这种情况下需要有一个具有边缘保持特性的滤波器，双边滤波器是一个很好的选择。

双边滤波它的权重由两部分组成，一部分是依赖空间位置的权重项，第二部分是图像颜色差异的权重项。

看公式可能有点抽象，可以先参考我之前写在OpenGL实现的双边滤波的文章。

Input是一个带噪声的输入信号，同时也是一个有着明显边缘结构（出现上下层断裂），当我们用双边滤波进行操作的时候，这里有两种权重，一个空间距离相关的权重（Spatial weight）其实就是一个高斯函数；另外一个是颜色差异边缘的权重（Range weight）就是和这个中心像素点颜色差异的一个权重。

最终双边滤波的权重，由（空间和颜色差异）两种权重相乘决定的，最后我们把相乘后的权重应用到输入信号上得到Result。明显看到结果即保留了边缘结构信息，也得到平滑去噪的效果。