Article directory
1 Overview
The noise in the image system comes from many aspects, including electronic components, such as thermal noise caused by resistors; shot noise and flicker noise caused by vacuum devices; particle noise and noise generated by junction transistors; channel heat generated by field effect transistors. Noise; photon quantum noise and electronic fluctuation noise of the photoelectric tube; various noises caused by the camera tube, etc. Various electronic circuits and equipment composed of these components will cause these noises to undergo different transformations to form local circuit and equipment noise. In addition, there is image noise generated by optical phenomena (such as quantum noise of X-rays in medical images).
Due to the presence of noise, our understanding of important information in images is disturbed. Usually, image noise is regarded as a multidimensional stochastic process, so probability distribution function and probability density function can be selected to describe image noise.
2. Noise model
The image denoising model can be modeled as:
Among them, y is the observed noise image, x is the true value of the image, n is the image noise, and the image denoising process is to obtain x through y.
There are mainly several types of noise modeling:
(1) Gaussian noise
Gaussian noise is the most common and important kind of noise. Many image denoising algorithms are designed to reduce Gaussian noise. The probability density The function looks like this:
(2) Rayleigh noise
The probability density function of Rayleigh noise is as follows:
The mean value and method of the probability density are respectively:
Rayleigh noise is more effective for noise modeling in images whose image histogram is approximately inclined.
(3) Gamma noise
The probability density function of gamma noise is as follows:
where the mean and variance are respectively:
Gamma noise will be more inclined than Rayleigh noise distribution
(4) Exponential noise
The probability density function of exponential noise is as follows:
The mean and variance of its probability density function are:
(5) Quantization noise
Quantization noise, also known as uniform noise, is caused by converting analog data into digital data, so it is inherent in the process of amplitude quantization, and its probability The density function is as follows:
(6) Salt and pepper noise
Salt and pepper noise, also known as impulse noise and spike noise, appears as randomly distributed black and white points on the image, and its probability density function is shown in the figure below: salt and pepper noise can be
eliminated by a median filter.
(7) Poisson noise
Poisson noise is also called shot noise. We know that the more photons emitted by the light source per second reach the CMOS, the greater the gray value of the pixel. However, because there may be some factors between the light source emission and CMOS reception, a certain photon is not received by the CMOS or a certain pixel emits too many photons within a period of time, so this leads to fluctuations in the gray value. Poisson noise is generated, and the equation is described as:
This formula describes the probability that a certain pixel receives k photons within the interval time τ.
(8) Speckle noise
Speckle noise can be seen in coherent imaging systems (such as radar, laser and acoustics, etc.), and its probability density function is as follows: Speckle noise
rarely occurs in optical imaging systems, so it will not be described here.
(9) Periodic noise
Periodic noise cannot be described by a probability density function, nor can it be eliminated in the space domain. The usual method is to eliminate it through a band-stop filter in the frequency domain.
3. Image noise reduction
According to whether the noise level is known, image denoising can be divided into blind denoising and non-blind denoising. For non-blind denoising, the noise level is considered a known parameter; for blind denoising algorithms, the noise level is unknown. Therefore, for the denoising algorithm, the noise level is a very important parameter.
(1) Mean filtering
(2) Median filtering
(3) Gaussian filtering
Gaussian filtering (Gaussian filtering) is to convolve the pixel value of each input pixel in the image neighborhood with the Gaussian kernel, and finally output the convolved pixel value, that is, the weighted average of the neighborhood pixels. Its smoothness depends on the standard deviation of Gaussian filtering, not simple smoothing.
The two-dimensional Gaussian distribution function, also known as the normal distribution, is defined as follows:
Among them, σ^2 is the variance of the Gaussian filter, which controls the smoothness of the Gaussian filter.
The filtered image of the image u0(x,y) obtained by passing the Gaussian template f(x,y) is: From the
above formula, it can be seen that this is an isotropic filter . Compared with the simple smoothing of the image, it retains The global features of the graphics have been greatly improved. However, when the Gaussian kernel performs the same processing on all pixels in the image, it cannot distinguish whether the processed pixel information is a smooth ordinary area or an edge area, and the result will degrade and blur the original edge or edge of the image. Texture information objectively weakens the visual quality of the image.
(4) Bilateral filter (NLM)
Bilateral filter (Bilateral filter) is composed of two parts: spatial proximity factor and gray similarity factor, in order to achieve the purpose of noise reduction and edge protection.
In the image, the weight of gray value j to i is:
where I_i and I_j are the gray value of the pixel; K_i is the normalization factor; σ_1 2 and σ_2 2 are the attenuation factors of the spatial distance and the gray distance, respectively. According to the form of the exponential decay function, it can be seen that the greater the difference between two gray values, the smaller the weight, and the lower the degree of noise reduction; on the contrary, the smaller the difference between the two gray values, the greater the weight , the degree of noise reduction will also increase. This shows that bilateral filtering can reduce the noise reduction effect in the edge part of the image and protect the edge information, and improve the noise reduction effect in the smooth area of the image to better filter out the noise.
In the bilateral filtering of the image, the gray value of the output image depends on the weighted combination of the gray value of the image neighborhood, which can be expressed as:
where the kernel of the definition domain (spatial neighborhood) is:
the kernel of the value domain (gray similarity) For:
the weight of bilateral filtering is composed of spatial neighborhood and gray similarity:
(5) Anisotropic filtering
Anisotropic Diffusion (Anisotropic Diffusion) is a noise reduction method using partial differential equations. The diffusion coefficient it uses is determined by the iterated gradient value instead of the initial noisy image gradient value. The most typical anisotropic diffusion method is the nonlinear anisotropic diffusion equation proposed by Perona and Malik, that is, the PM model. Its expression is: where is the diffusion function (factor), and is also the monotonous decrease of
the 
gradient function.
The classic expression given by Perona and Malik is as follows:
From formula 3.10, it can be seen that the diffusion coefficient along the gradient direction 
, in the flat area, the gradient value is low, the diffusion coefficient is large, the image is extremely smooth, and the noise is removed; while the edge area, the gradient value High, the diffusion coefficient is small, and the details are protected. When the gradient value u is greater than K, the diffusion coefficient gradually becomes negative, and the image begins to diffuse in the opposite direction, thereby enhancing edge information. The final iterative form of the Perona_Malik algorithm is as follows:
The above formula is the four gradient directions of the PM model in the south, east, and west. During the diffusion process, the PM model not only maintains the edge of the image, but also effectively removes noise, and has a strong noise reduction ability for flat areas. Therefore, there is an anisotropic diffusion noise reduction method. However, the disadvantage of the PM model is that it cannot find the precise gradient boundary between the noise and the edge, which affects the estimation of the gradient value, and finally makes the noise larger, affecting the overall noise reduction effect. In fact, traditional PM models have a staircase effect when processing images.
(6) Total variation model
The total variation model is an anisotropic model that relies on the gradient descent method to smooth the image, hoping to smooth the image as much as possible inside the image (the difference between adjacent pixels is small), and Unsmooth as much as possible around image edges (image outlines). If you want to fully understand the total variational model, you need to know concepts such as functional analysis, gradient descent method, and Euler Lagrange equation EL.
As early as 1992, Rudin, Osher and Fatemi proposed a classic total variation image denoising algorithm, the original text: Nonlinear total variation based noise removal algorithms.
Different from the previous denoising/filtering algorithms, the TV algorithm is an image restoration algorithm. It restores a clean image from a noisy image. By establishing a noise model and using an optimization algorithm to solve the module, and through the process of continuous iteration , making the restored image infinitely close to the ideal image after denoising. Very similar to deep learning, the noise model is analogous to the loss function. Through continuous training, the gap between the two is getting closer and closer. It also requires the gradient descent method to quickly obtain the optimal solution.
4 Noise level estimation
Noise level is one of the important indicators in denoising methods, and wrong noise estimation will greatly affect the effect of a denoising method. At present, most noise estimation methods are based on Gaussian white noise to estimate the noise level. These noise estimation methods can be broadly classified into three categories: filter-based methods, image patch analysis-based methods , and statistical prior-based methods.
(1) Filter-based method
The filter-based estimation method first uses a high-pass filter to process the noise image, and then performs noise estimation based on the difference between the processed image and the original image. One of the classic noise estimation methods is the mean absolute deviation. The principle of the filter-based method is simple, but it also has obvious shortcomings, especially for some texture-rich images, the generated difference images are not all noise, which greatly affects the estimation of noise.
(2) Method based on image block analysis
In the noise estimation method based on image blocks, the conventional method is to first divide the image into a certain number of equal-sized image blocks, and then select these decomposed image blocks according to certain rules, and finally according to the selected image blocks to estimate the noise level. Since the intensity variation of image blocks is mainly caused by noise, a common method for selecting image blocks is to find image blocks whose pixel value standard deviation is close to the minimum standard deviation, or to find image blocks with weak textures. In a noise estimation algorithm combining principal component analysis and image blocks proposed by Chinese scholar Liu et al. in 2013, they selected low-rank image blocks without high-frequency components through gradient and statistical data, and then used principal component analysis to Estimate the noise level. Image block-based methods are less sensitive to noise changes, but since the selection results of image blocks will fluctuate with the input image and noise level, such methods will be overestimated at low noise levels, while at high noise levels The noise level is often underestimated.
(3) Noise estimation method based on statistics
The noise estimation method based on statistics utilizes the conventional properties of natural images, such as the non-Gaussian distribution of pictures in natural scenes, and the marginal distribution of their band-pass responses is highly kurtotic. Foreign scholars Stefano et al. proposed three methods in 2004. A Noise Estimation Method Based on Training Samples and Natural Image Statistics. Zoran et al. proposed a method for statistical analysis of graph nitrogen after discrete cosine transform changes, and pointed out that the change of kurtosis value was caused by noise. After that, Chinese scholar Dong combined the scale invariance feature with the piecewise stationarity of the natural image itself, and proposed a noise estimation method in 2017. Gupta, a scholar at the University of Texas in the United States, further explored the scale-invariant properties, and proposed a noise estimation method based on natural scene statistics in 2018.
Compared with traditional methods, neural networks have more powerful data fitting capabilities. Guo et al., a scholar at Harbin Institute of Technology, used a neural network composed of 5 layers of 32-channel convolutional layers to estimate the noise level of the image, which prompted the denoising network to greatly improve the denoising performance on the real noise data set. Inspired by the above method, some authors carefully designed a multi-task noise estimation model based on neural network, which can simultaneously estimate the type of noise in the image and the corresponding noise level.
-------The relevant algorithm implementation will be updated later.
References:
[1] https://blog.csdn.net/weixin_44580210/article/details/105087523
[2] https://blog.csdn.net/weixin_45355387/article/details/118968623