URetinex-Net: Retinex-based Deep Unfolding Network for Low-light Image Enhancement paper reading notes

• This is a paper on dark image enhancement published by Shenzhen University, City University of Hong Kong and Nanyang Technological University at CVPR2022
• The network structure is shown in the figure:
  
• The first in the upper left corner is the initialization module, which sends the input image to four layers of 3x3 convolution to estimate the initial brightness layer and illumination layer. This module is trained separately, using dark images and bright images for training at the same time. The following loss supervision is used when dark images are input: The following loss supervision is used when bright images are input, and a
  
  smoothing loss is added. The luminance component of the bright image is considered to be smooth. of:
  
• Before introducing the second module, we need to talk about the formula of retinex. retinex is originally the following formula, to convert $I$ decomposes into$R$ and$L$ component, so that$R$ and$L$ respectively satisfies the following formulas:
  
• Using half quadratic splitting, the above optimization problem can be transformed into the following form:
  
• Half quadratic splitting uses an iterative method to optimize the four components respectively, so it is represented by the unfolding optimization module as the second module in the figure. The formula is as follows: The optimization of P and Q is very simple. Just find the gradient
  
  equal to 0. Get the minimum value expression:
  
  
• The optimization of L and R is carried out with the help of the network. The prediction network of L is relatively simple and only requires 5 layers of convolution. The network of R is displayed in the lower right corner of the network structure diagram. The network parameters of different stages are shared:
  
  
• The loss functions of the two networks are as follows, where $\hat{R}$ is the R component extracted by the corresponding bright image GT using the first-stage network,$\varphi$ represents vgg, which is perceptual loss:
  
• Then there is the third illumination adjustment module, which is to adjust the L component, but do not use gamma correction but use the network (suggested in KinD), and like KinD, it provides a factor that can artificially adjust the brightness ω \ $\omega$，将$\omega$ repeat to the same size as L, then concatenate to the back, and send it into the network together (the structure of the network is the same as that of the first initialized network, but uses 5x5 convolution). The loss function of this module is as follows, with waves The line is the output of the network, and the covered one is the brightness component of GT (also predicted using the first initialization network). During training,$\omega$ is set to L of GT divided by L of dark image (both predicted by the initialization network).
  
• The network is trained and tested on the LOL dataset, and each module is trained separately, with T=3 for the intermediate iteration module.
• As you can see from the experimental results, LOL is SOTA:
  
• The visualization effect is also good, but the speed is a bit slower, but not particularly slow:
  
• The inspiration is that I have recently seen many methods of using HQS to optimize the retinex model. This work uses the network to avoid the manual design constraints of Retinex (although the first initialization network is still used), and in some places uses SSIM loss and gradient loss to avoid A direct comparison of brightness. I feel that the article is clear and well-founded. It is indeed a good article.