Convolutional Neural Network DehazeNet: An End-to-End System for Single Image Haze Removal

The core of image dehazing

Existing image dehazing technology is inseparable from a simple natural model - the Atmospheric Scattering Model . The atmospheric scattering model describes the imaging mechanism under the combined action of haze and illumination:

Sunlight forms reflected light J ( x ) on the surface of the object  , and the reflected light is scattered in the process of passing through the haze, and only part of the energy  J ( x ) t ( x ) can reach the camera. At the same time, sunlight is also scattered on the surface of the suspended particles to form atmospheric light  α  that is received by the camera. Therefore, the imaging  I ( x ) in the camera can be composed of two parts, the transmitted object brightness  J ( x ) t ( x ) and the scattered atmospheric illumination  α ( 1 - t ( x )):

Among them, t ( x ) is the medium transmission, which, as the name implies, represents the ratio that can smoothly pass through the haze to the camera . Therefore, the transmittance is inversely proportional to the distance  d ( x ) between the object and the camera, and the object farther from the camera is more affected by the haze. When the distance  d ( x ) tends to infinity, the transmittance  t ( x ) tends to zero, I ( x ) tends to  α , α = max y∈ { x|t ( x ) ≤ t0 } I ( y ). In summary, the core of dehazing is how to estimate the medium transmittance  t ( x ) more accurately.

Based on artificial features

Manual features are the foundation of traditional machine vision, which emphasizes that practice makes perfect, and relies on practice to produce true knowledge. Build a variety of features through "observation→experience→design" to meet a variety of task requirements. Image dehazing technology is also gradually developed along hand-crafted features.

(1) Dark Channel Prior [2] (Dark Channel Prior, DCP)

Speaking of the dehazing feature, the dark channel prior (DCP) has to be mentioned. The journey of the road lies in simplicity. As the best paper of CVPR 2009, DCP solves the problem of smog concentration estimation with concise and effective prior assumptions.

It is observed that a channel in the RGB color space of the clear image block is very dark (low value or even close to zero). Therefore, based on the dark channel prior, the fog concentration can be approximated by the value of the darkest channel:

(2) Maximum contrast [3] (Maximum Contrast, MC)

According to the atmospheric scattering model, haze will reduce the contrast of object imaging: ΣxΔI(x)=tΣxΔJ(x)≤ΣxΔJ(x) ‖. Therefore, based on this inference, the local contrast can be used to approximate the haze concentration. At the same time, the color and visibility of the image can also be restored by maximizing local contrast.

(3) Color Attenuation Prior [4] (Color Attenuation Prior, CAP)

The color decay prior (CAP) is a prior similar to the dark channel prior (DCP). It is observed that haze can lead to a decrease in image saturation and an increase in brightness at the same time, and the overall performance is the attenuation of color. The difference in brightness and saturation is used to estimate the haze concentration according to the color decay prior:

Based on deep intelligence

The human visual system does not rely on these explicit feature transformations to estimate fog concentration and scene depth well. DehazeNet is a specially designed deep convolutional network that uses deep learning to intelligently learn haze features and solve the difficulties and pain points of manual feature design.

(1) Feature Extraction

Feature extraction is different from traditional convolutional neural networks. DehazeNet adopts the structure of "convolution + Maxout[5]" as the first layer of the network:

And it can be proved that "convolution + Maxout" is equivalent to the traditional manual dehazing feature:

当W1是反向(Opposite)滤波器,通道的最大等价于通道的最小值,等价于暗通道先验(DCP);当W1是环形(Round)滤波器, 等价于对比度提取,等价于最大对比度(MC);当W1同时包含反向(Opposite)滤波器和全通(All-pass)滤波器,等价于RGB到HSV颜色空间转换,等价于颜色衰减先验(CAP)。

此外,从机器学习角度看,Maxout是一种样条函数,具有更强的非线性拟合能力,如下图(d)。

(2)多尺度映射(Multi-scale Mapping)与局部极值(Local Extremum)

多尺度特征会提高不同分辨率下特征提取的鲁棒性。传统去雾方法中也会采用不同尺度的滤波器(均值、中值、最小值)来增强特征在不同尺度下的鲁棒性。借鉴于GoogLeNet中的inception结构,采用3组不同尺度(3×3,5×5,7×7)的滤波器实现DehazeNet的尺度鲁棒性:

局部极值(MAX Pooling)是深度卷积神经网络的经典操作。局部极值约束了透射率的局部一致性,可以有效抑制透射率的估计噪声。此外,局部极值也对应于暗通道先验(DCP)的局部最小值和最大对比度(MC)的局部最大值。

(3)非线性回归(Non-linear Regression)

大气透射率是一个概率(0到1),不可能无穷大,也不可能无穷小。受到Sigmoid和ReLU激励函数的启发,提出双边纠正线性单元(Bilateral Rectified Linear Unit,BReLU),在双边约束的同时,保证局部的线性。

BReLU的非线性回归对应于传统去雾方法中的边缘抑制操作(如DCP和CAP)。双边约束引入先验信息缩小参数搜索空间,使得网络更加容易训练;局部线性避免Sigmoid函数梯度不一致带来的收敛困难。

DehazeNet基于手工特征,又超出传统方法,从人工到智能。因此,DehazeNet取得了更好的去雾结果,更多的对比实验和代码资源:

  • 项目主页

    http://caibolun.github.io/DehazeNet/

  • GitHub代码

    https://github.com/caibolun/DehazeNet

  • BReLU+Caffe

    https://github.com/zlinker/mycaffe

  • 其他复现

    (1)https://github.com/zlinker/DehazeNet

    (2)https://github.com/allenyangyl/dehaze

参考文献

[1] Cai B, Xu X, Jia K, et al. DehazeNet: An End-to-End System for Single Image Haze Removal [J]. IEEE Transactions on Image Processing, 2016, 25(11): 5187-5198.

[2] He K, Sun J, Tang X. Single image haze removal using dark channel prior[J]. IEEE transactions on pattern analysis and machine intelligence, 2011, 33(12): 2341-2353.

[3] Tan R T. Visibility in bad weather from a single image[C]//Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on. IEEE, 2008: 1-8.

[4] Zhu Q, Mai J, Shao L. A fast single image haze removal algorithm using color attenuation prior[J]. IEEE Transactions on Image Processing, 2015, 24(11): 3522-3533.

[5] Goodfellow I J, Warde-Farley D, Mirza M, et al. Maxout networks[J]. ICML (3), 2013, 28: 1319-1327.

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=325406905&siteId=291194637