阅读笔记——《FFDNet Toward a Fast and Flexible Solution for CNN based Image Denoising》

版权声明: https://blog.csdn.net/gwplovekimi/article/details/84852866

本博文属于阅读笔记,仅供本人学习理解用

论文链接:https://ieeexplore.ieee.org/abstract/document/8365806

给出代码(https://github.com/cszn/FFDNet

 

Many methods mostly learn a specific model for each noise level, and require multiple models for denoising images with different noise levels.对于基于深度学习的去噪网络,大多数都仅仅是学习网络的在某一个噪声水平下的模型,而对于不同的噪声水平,需要多个不同的去噪网络,为次本文提出了一种快速的、灵活的去噪卷积神经网络FFDNet。该网络通过下采样sub-images来加速处理的过程,并采用正交正则化方法提高泛化能力。与现有的discriminative denoisers(判别式降噪器)相比,作者提出的网络具有以下的优点:

1、可以仅仅只用一个网络就实现处理宽范围的噪声。

2、通过指定非均匀噪声水平图来去除空间变异噪声的能力(the ability to remove spatially variant noise by specifying a non-uniform noise level map)

3、速度快

 

通过合成的噪声图像和真实的噪声图像来验证所提出的FFDNet网络的性能。在本文中,假设噪声是AWGN,并且噪声水平是给定的。为了应对实际去噪的问题,一个灵活的去噪器需要有以下的特性:

1、可以只采用一个模型就实现去噪

2、有效的、高效的、容易使用的

3、可以处理空间变异(spatially variant)噪声。

When the noise level is unknown or is difficult to estimate, the denoiser should allow the user to adaptively control the tradeoff between noise reduction and details preservation.(当噪声水平未知或难以估计时,降噪器应允许用户自适应地控制降噪和细节保存之间的权衡)进一步地,the noise can be spatially variant and the denoiser should be flexible enough to handle spatially variant noise(噪声可以在空间上变化,并且降噪器应该足够灵活以处理空间变化的噪声。)

the FFDNet——the noise level map is modeled as an input  and the model parameters are invariant to noise level。 FFDNet provides
a flexible way to handle various types of noise with a single network.

the proposed FFDNet works on downsampled sub-images, which largely accelerates the training and testing speed, and enlarges the receptive field as well(这操作是为了加速用得)

 

当前去噪的方法可以分为两种:

1、model based methods——例如BM3D and WNNM  are flexible in handling denoising problems with various noise levels, but they suffer from several drawbacks. 耗时,不能直接用于去除空间变异噪声。并且需要手工制作先验图像以及nonlocal selfsimilarity

2、discriminative (辨别) learning based ones(CNN类)——learn the underlying(底层) image prior and fast inference from a training set of degraded and ground-truth image pairs.The learned model is usually tailored to a specific noise level.(只能处理特定的噪声水平)is hard to be directly deployed to images with other noise levels.all the existing discriminative learning based methods lack flexibility to deal with spatially variant noise.

 

DnCNN利用Batch Normalization和residual learning可以有效地去除均匀高斯噪声,且对一定噪声水平范围的噪声都有抑制作用。然而真实的噪声并不是均匀的高斯噪声,其是信号依赖的,各颜色通道相关的,而且是不均匀的,可能随空间位置变化的。在这种情况下,FFDNet使用噪声估计图作为输入,权衡对均布噪声的抑制和细节的保持,从而应对更加复杂的真实场景。而CBDNet进一步发挥了这种优势,其将噪声水平估计过程也用一个子网络实现,从而使得整个网络可以实现盲去噪。
 

FFDNet网络的特点:

  1. 将噪声水平估计作为网络的输入,可以应对更加复杂的噪声,如不同噪声水平噪声和空间变化噪声,而且噪声水平估计可以作为权重权衡对噪声的抑制和细节的保持。
  2. 将输入图像下采样为多张子图像作为网络输入,输出的子图像再通过上采样得到最终的输出。该操作在保持结果精度的条件下,有效地减少了网络参数,增加感受野,使得网络更有效率,更快。
  3. 使用正交矩阵初始化网络参数,从而使得网络训练更有效率。
     

FFDNet网络通过将噪声水平图(tunable noise level map)作为输入,使得去噪网络可以对噪声水平更加灵活。而为了提高去噪网络的效率,将输入图像降采样来处理。同时为了(insensitive to the bias between the input and ground truth noise levels and generate less artifacts),在卷积层中采用了orthogonal regularization

噪声估计子网络将噪声观测图像转换为估计的噪声水平图

网络结构如下图所示

 

Noise Level Map

先重温一下为啥model-based image denoising methods可以适用于不同的噪声水平,如下式子所示(data fidelity term数据保真度,regularization terms.正则化项)

m为noise level map

an implicit function(隐含功能)

这篇论文是把噪声图和 noise level map作为网络的输入,进而可以实现不同的噪声等级下,估计噪声。。。。可是问题是怎么得到 noise level map呢?应该在CBDNet有说到?

the noise level map may not be accurately estimated from the noisy observation, and mismatch between the input and real noise levels is inevitable.(噪声水平图可能无法从噪声观察中准确估计,并且输入和实际噪声水平之间的不匹配是不可避免的)If the input noise level is lower than the real noise level, the noise cannot be completely removed. Therefore, users often prefer to set a higher noise level to remove more noise. However, this may also remove too much image details together with noise.
A practical denoiser should tolerate certain mismatch of noise levels.

 

An approximation of non-uniform noise level map can then be obtained.

FFDNet exhibits similar noise level sensitivity performance to BM3D and DnCNN in balancing noise reduction and details preservation. When the ground truth noise level is unknown(当真正的 noise level 是未知时,FFDNet效果也是更好的)

关于noise level,注意看实验的E部分

 

 

参考博文:

https://blog.csdn.net/zbwgycm/article/details/82848893

https://blog.csdn.net/zbwgycm/article/details/82052003(关于CBDNet)

 

 

 

 

 

 

猜你喜欢

转载自blog.csdn.net/gwplovekimi/article/details/84852866