A Fusion-based enhancing method for weakly illuminated images
approach
- \(Step\) \(1\) illumination estimation
Derived from retinex theory, an input image could be decomposed into the illumination layer and reflectance layer, as \(Eq.1\) defined. A prior assumption that RGB channels have the common illumination is adopted to \(Eq.1\).
\[S^c(x, y) = I(x, y) \odot R^c{x, y} \quad c\in{R, G. B}\tag{1}\]
\(Eq.1\) where \(S\) is input weakly illuminated image need enhancement, \(I\) and \(R\) are illumination image and reflectance image respectively. \(\odot\) means element-wise multiplication.
Due to the goal to enhance a weakly enhanced image, get the illumination layer is of vital importance. Inspired by the dark channel prior, which based on an observation that most local patches in haze-free outdoor images contain some pixels which have very low intensities in at least one color channel, initial illumination layer can be estimated by \(Eq.2\) defined as follow:
\[L(x, y) = max_{c\in{R,G, B}}S^c(x, y)\tag{2}\]
\(Eq.2\) is a transformed version of one dark channel prior theory, which is written as
\[1 - L = min_c\frac{1 - S^c}{\alpha}\quad c\in{R, G, B}\]
As the illumination is local smooth, a smoothing operator is required to adopt to refine \(L\). A simple but effective algorithm based on using a morphologically closing operator is adopted to smoothing processing, while the maximum filter which is used by the authors of one dark channel prior theory is abandoned for its time complexity. The morphologically closing operator smooths an image by fusing narrow breaks and filling gaps on the contours without over smoothing to produce nato effect.
\[I = \frac{L · P}{255}\tag{3}\]
\(Eq.3\) where P denotes the structuring element and \(·\) is the closing operation.
Guided filter is adopted to refine the estimated illumination to preserve the shape of contours.
\[I_i \larr \sum_jW_{ij}(g)I_j\tag{4}\]
\[W_{ij}(g) = \frac{1}{|\omega|^2}\sum_{k:(i, j)\in\omega_k}(1 + \frac{(g_i - \mu_k)(g_j - \mu_k)}{\sigma_k^2 - \epsilon})\tag{5}\]
\(Eq.4,5\) where \(\omega_k\), \(|\omega|\) are a window centered pixel \(k\) and pixel numbers of windows respectively. \(\mu\) and \(\sigma^2\) are the mean and variance of guided image \(g\) respectively. \(\epsilon\) is a regularization parameter. \(g\) is the \(V\) layer of HAV color space of input image \(S\).
\(Step\) \(2\) input derivation
To avoid distortion, we make original illumination estimation as the first input layer which contains the information of original structures of original estimated illumination.
\[I_1(x, y) = I(x, y)\tag{6}\]
To enhance global illumination situation, many approaches can be applied, such as gamma correction and sigmoid function. Here arc tangent transformation is adopted to this work.
\[I_2(x, y) = \frac{2}{\pi}\arctan(\lambda I(x, y))\tag{7}\]
\(Eq.7\) where \(\lambda\) is a parameter that control the degree of luminance, defined as follow.
\[\lambda = 10 + \frac{1 - I_{mean}}{I_{mean}}\tag{8}\]
As written in \(Eq.8\), a higher \(\lambda\) will be obtained when smaller \(I_{mean}\) is derived which indicated the poor level of original estimated luminance.
Since dynamic range is compressed after improving global luminance, local contrast is reduced which is designed to be enhanced by using "contrast local adaptive histogram equation"(CLAHE1) in the third input layer. CLAHE is adopted directly to original illuminance estimation, which is useful to expand local contrast between adjacent structures.
\(Step\) \(3\) Weight definition
brightness weight Considering well-exposed property on pixel-level, a pixel-level weights system is designed for fusion, which guaranty a high value on well-exposed pixels and natural-looking brightness. After statistically evaluate 2000 pics on Google image, brightness weight system is defined based on a observation on the mean of mean\standard deviation and histogram of illumination of data set are 0.5\0.25 approximately respectively.
\[W_{B, k}(x, y) = \exp{\{-\frac{(I_k(x, y) - 0.5)^2}{2(0.25)^2}\}}\tag{9}\]
chromatic contrast weight Considering color contrast is of vital importance to image quality, contrast is evaluated by combining the estimated illumination with chromatic information.
\[W_{c, k}(x, y) = I_k(x, y)(1 + cos(\alpha H(x, y) + \phi)S(x, y))\tag{10}\]
\(Eq.10\) where \(H\) and \(S\) are hue and saturation vector in HSV color space respectively, \(\alpha\) and \(\phi\) are parameters to preserve the color opponency and represents the offset angle of the color wheel respectively. In this work, parameters are set to \(\alpha = 2\) and \(\phi = 250\degree\) respectively according to2.
The impact of this weight (\(\alpha = 2\) and \(\phi = 250\degree\)) is to highlight regions containing high contrast caused by both illumination and color.
final weight
\[W_k(x, y) = W_{B, k}(x, y)W_{C, k}(x, y)\tag{11}\]
Normalization,
\[\bar{W}_k(x, y) = \frac{W_k(x, y)}{\sum_kW_k(x, y)}\tag{12}\]
\(Step\) \(4\) Multi-scale fusion
With a naive fusion method, we fusion derived input mentioned in subsection \(Step\) \(3\) by summing the multiplying results weight matrix and derived input respectively directly.
\[I_{fusion}(x, y) = \sum_k\bar{W}_k(x, y)I_k(x, y)\tag{13}\]
However, disappointing artifacts are produced in generating results processing which is mainly caused by strong transitions of the weight maps. To overcome this, multi-scale linear and non-linear filters are adopted. While the fact that non-linear filters can not avoid these artifacts is borne out by extensive experiments, a multi-scale pyramid technique is adopted to solve this problems.
The inputs are convolved by a Gaussian kernel to generate a low pass filtered versions.In our case, we decompose each derived input \(I_k\) into a Laplacian pyramid to extract image features, and each normalized weight \(\bar{W}_k\) into a Gaussian pyramid to smooth the strong transitions. This method is effective since it blends image features instead of intensities.
\[F_l(x, y) = \sum_kG_l\{\bar{W}_k(x, y)\}L_l\{I_k(x, y)\}\tag{14}\]
\(Eq.14\) where \(l\) is the number of pyramid level, \(l = 6\) is set in this work.
\[I_{final}(x, y) = \sum_lU_d(F_l(x, y))\tag{15}\]
\(Eq.15\) where \(U_d\) is the up-sampling operator with factor \(d = 2^{l - 1}\).
Finally,
\[S_{enhanced}^c(x, y) = R^c(x, y)I_final(x, y)\quad c\in\{R, G, B\}\tag{16}\]
puzzle
morphologically closing operator
chromatic filtering formula