OpenCVForUnity (9) picture blur

---

foreword

This tutorial will introduce the use of various linear filters in OpenCV to smooth images, mainly including the following:
blur: blur filter
GaussianBlur: Gaussian blur filter
medianBlur: median filter
bilateralFilter: bilateral filter

Smoothing is one of the commonly used image processing operations, which can effectively remove the influence of noise. In this tutorial, we'll introduce the principles of smoothing and demonstrate it using noise reduction as an example. In addition, it should be noted that the following explanations are mainly derived from the classic textbooks "Richard Szeliski" and "Learning OpenCV: Algorithms and Applications" in the field of computer vision.

For smoothing, we need to apply a filter to the image. A filter is a mathematical operation that takes the weighted sum of input pixel values ​​(ie f(i+k,j+l)) to obtain an output pixel value (ie g(i,j)). The most common filter type is linear, i.e. the output pixel value is a weighted sum of the input pixel values:

These coefficients are also called kernels, and they form the heart of the filter. We can imagine the kernel as a coefficient window, and slide this window on the image for convolution calculation. In practical applications, there are many kinds of filters commonly used, we will introduce the most

1. Normalized box filter

One of them is the normalized box filter, also known as the mean filter. This is one of the simplest filters, where for each output pixel its value is the average of its neighbors (each weighted equally). Its kernel can be expressed as the following matrix:

Use the blur method to achieve

struct
Imgproc.blur(source Mat, target Mat, blur size);
Example:

//图片读取
string readPath1 = Application.dataPath + "/OpenCVForUnity/Examples/Resources/haizeiwang.png";
//放置图片
Mat blurMat = Imgcodecs.imread(Utils.getFilePath(readPath1), Imgcodecs.IMREAD_COLOR);
//模糊处理
Imgproc.blur(blurMat, blurMat,new Size(20,20));
//色彩模式转换
Imgproc.cvtColor(blurMat, blurMat, Imgproc.COLOR_BGR2RGB);
//结果放置在图片中
Texture2D blurTexture = new Texture2D(blurMat.cols(), blurMat.rows(), TextureFormat.RGBA32, false);
Utils.matToTexture2D(blurMat, blurTexture);
//展示在材质中
GameObject.Find("Cube1").GetComponent<Renderer>().material.mainTexture = blurTexture;

Original image:

After processing:

2. Gaussian filter

Gaussian filters are probably one of the most commonly used and useful filters (though not the fastest). The output image is obtained by convolving each pixel in the input image with a Gaussian kernel and summing the convolution results. It effectively smoothes images and is capable of removing various types of noise, such as Gaussian noise.
For the one-dimensional case, it can be found that the pixel located in the center has the largest weight, and the weights of neighbors decrease as the spatial distance between them and the central pixel increases.
It should be noted that the two-dimensional Gaussian kernel can be expressed as the following form:

where μ represents the mean (i.e. peak value) and σ represents the variance (for each variable x and y).
The 2D data formula can be expressed as:

Implemented using the GaussianBlur method

Structure:
Imgproc.GaussianBlur(source Mat, target Mat, blur size (positive odd), X standard deviation, Y standard deviation, pixel extrapolation);
Example:

//图片读取
string readPath1 = Application.dataPath + "/OpenCVForUnity/Examples/Resources/haizeiwang.png";
//放置图片
Mat blurMat = Imgcodecs.imread(Utils.getFilePath(readPath1), Imgcodecs.IMREAD_COLOR);
//模糊处理
Imgproc.GaussianBlur(blurMat, blurMat, new Size(33, 33), 0, 0);
//色彩模式转换
Imgproc.cvtColor(blurMat, blurMat, Imgproc.COLOR_BGR2RGB);
//结果放置在图片中
Texture2D blurTexture = new Texture2D(blurMat.cols(), blurMat.rows(), TextureFormat.RGBA32, false);
Utils.matToTexture2D(blurMat, blurTexture);
//展示在材质中
GameObject.Find("Cube1").GetComponent<Renderer>().material.mainTexture = blurTexture;

3. Median filter

A median filter iterates over each element of the signal (an image in this case) and replaces each pixel with the median of its neighbors (located in a square neighborhood around the estimated pixel).

Implemented using the medianBlur method

Structure:
Imgproc.medianBlur(source Mat, target Mat, blur size (positive odd));
Example:

//图片读取
string readPath1 = Application.dataPath + "/OpenCVForUnity/Examples/Resources/haizeiwang.png";
//放置图片
Mat blurMat = Imgcodecs.imread(Utils.getFilePath(readPath1), Imgcodecs.IMREAD_COLOR);
//模糊处理
Imgproc.medianBlur(blurMat, blurMat, 33);
//色彩模式转换
Imgproc.cvtColor(blurMat, blurMat, Imgproc.COLOR_BGR2RGB);
//结果放置在图片中
Texture2D blurTexture = new Texture2D(blurMat.cols(), blurMat.rows(), TextureFormat.RGBA32, false);
Utils.matToTexture2D(blurMat, blurTexture);
//展示在材质中
GameObject.Find("Cube1").GetComponent<Renderer>().material.mainTexture = blurTexture;

4. Bilateral filter

A bilateral filter is a filter that can remove noise while preserving edge information. It is similar to a Gaussian filter, but it considers not only the spatial distance between pixels, but also the difference between their grayscale values. Bilateral filters use two components when computing the weights of each pixel with its neighbors: one with the same spatial weights as the Gaussian filter, and one with weights based on the similarity between pixel values. Both components can be parameterized to adjust their influence on the output image.
Because the bilateral filter considers the similarity between pixel values, it can preserve edge information, unlike other smoothing filters such as Gaussian filters that blur edges. Therefore, bilateral filters work better than others in some scenarios, such as image denoising and image enhancement.
It should be noted that the bilateral filter has a relatively large amount of calculation, so it may be slower than other filters, but the calculation amount and filtering effect can be balanced by adjusting the parameters.

Implemented using the bilateralFilter method

Structure:
Imgproc.lateralFilter(source Mat, target Mat, diameter of each pixel neighborhood, affected color range in pixel neighborhood, affected color distance);
example

//图片读取
string readPath1 = Application.dataPath + "/OpenCVForUnity/Examples/Resources/haizeiwang.png";
//放置图片
Mat blurMat = Imgcodecs.imread(Utils.getFilePath(readPath1), Imgcodecs.IMREAD_COLOR);
//模糊处理
Imgproc.bilateralFilter(blurMat, blurMat, 15, 80, 80, Core.BORDER_DEFAULT);
//色彩模式转换
Imgproc.cvtColor(blurMat, blurMat, Imgproc.COLOR_BGR2RGB);
//结果放置在图片中
Texture2D blurTexture = new Texture2D(blurMat.cols(), blurMat.rows(), TextureFormat.RGBA32, false);
Utils.matToTexture2D(blurMat, blurTexture);
//展示在材质中
GameObject.Find("Cube1").GetComponent<Renderer>().material.mainTexture = blurTexture;

epilogue

Although they are all blur effects, there are still some differences when compared with the above renderings. You can consider the effects and performance. Choose to use different filters to achieve the effect you want.
So this is the end of this sharing, if you can, click like, thank you for watching.