Coding based on learning (6): DRNLF
The algorithm in this paper is from JVET-L0242, dense residual convolutional network based in-loop filter (DRNLF) is used in the loop filter of VTM, after DBF, before SAO and ALF, as shown in the figure below.
RDO decides whether to use DRNLF.
Network structure
The structure of DRNLF is as follows:
N represents the number of DRU (dense residual unit), and M represents the number of convolution kernels.
The algorithm in this paper is an improvement of JVET-K0391. The structure of DRU in K0391 is shown in the figure below:
There are five main improvements:
-
Removed the external (global identity skip connection) 3x3 convolutional layer to speed up training.
-
The normalized QP map and the reconstructed image are input to the DRN together, and only one model can be used to adapt to different QP situations.
-
Training in YUV space.
-
In order to reduce the computational complexity, the number of DRUs is reduced from 8 to 4, and the convolution kernel is reduced from 64 to 32.
-
The 3x3 convolutional layer is replaced with a 3x3 DSC (depth-wise separable convolutional) layer.
The above 5 improvements reduce the model parameters from 810k to 22k.
training
Use DIV2K to generate training set and validation set. The training set contains 800 images and the verification set contains 100 images. The network is trained in YUV space, so the DIV2K image needs to be converted from RGB space to YUV space. Use VTM2.0.1 to compress images with different QPs under AI (All Intra) configuration. The compressed image and the corresponding QP are used as network input. The image before compression is regarded as ground truth. Assuming that the compressed image set is {X} and the corresponding ground truth set is {Y}, the loss function is as follows:
experiment
In VTM2.0.1, AI configuration, QP is {22,27,32,37}, and the experimental results using only the CPU environment are as follows:
If you are interested, please follow the WeChat public account Video Coding
