11. Image Recovery - Repair

27 WaveFill: A Wavelet-based Generation Network for Image Inpainting

Image inpainting aims to complete missing or damaged areas of an image with realistic content. Current popular methods reconstruct results with better perceptual quality by using generative adversarial networks. But reconstruction loss and adversarial loss focus on synthesizing content of different frequencies, and simply applying them together often leads to conflict between frequencies.
This paper introduces WaveFill, based on wavelet inpainting, which decomposes the image into multiple frequency bands and explicitly fills the missing areas in each frequency band separately. WaveFill uses discrete wavelet transform (DWT) to decompose images, naturally preserving spatial information. It applies the L1 reconstruction loss to the decomposed low frequency bands and the adversarial loss to the high frequency bands, thereby effectively mitigating inter-frequency conflicts while completing the spatial domain image. In order to solve the problem of inconsistency in repairing in different frequency bands and fuse features with different statistics, a new normalization scheme is designed, which can effectively align and fuse multi-frequency features. Extensive experiments demonstrate the qualitative and quantitative superiority of WaveFill

28 Painting from Part

This paper studies image-based local rendering and inpainting, involving in-image inpainting and external inpainting. In order to make full use of the information from the local region and the information from the global domain (dataset), a new rendering method is proposed, which includes three stages: noise restart, feature repainting, local refinement, using feature level, local region level , Generative Adversarial Network's powerful representation ability to draw the entire image.
https://github.com/zhenglab/partpainting

29 High-Fidelity Pluralistic Image Completion with Transformers

Convolutional Neural Networks (CNN) have made great strides in image completion. But due to some inherent properties (e.g., local inductive priors), CNNs perform poorly in understanding global structure or supporting multivariate completion. Recently, transformers have demonstrated the ability to model long-term relationships and generate diverse results, but the computational complexity is prohibitive for processing high-resolution images.
This paper uses transformers for appearance prior reconstruction and CNN for texture supplementation

30 Image Inpainting via Conditional Texture and Structure Dual Generation

Deep generative methods have made considerable progress in image inpainting by introducing structural priors. However, due to the lack of proper interaction with image textures during structural reconstruction, current solutions cannot handle large damage cases and often suffer from distorted results.

In this paper, we propose a novel two-stream network for image inpainting, where texture synthesis with structural constraints and texture-guided structural reconstruction in a coupled manner can better utilize each other for more reasonable generation. Furthermore, to enhance global consistency, a bidirectional gated feature fusion (Bi-GFF) module is designed to exchange and combine structural and texture information, and a contextual feature aggregation (CFA) module is developed. Qualitative and quantitative experiments on CelebA, Paris StreetView and Places2 datasets demonstrate the superiority of the proposed method.

https://github.com/Xiefan-Guo/CTSDG

31 Learning High-Fidelity Face Texture Completion without Complete Face Texture

For face texture completion, supervised learning is usually performed with a multi-view imaging system or some complete textures from 3D scans. This paper attempts to accomplish face texture completion without using any full texture, training the model in an unsupervised manner with a large number of different face images (e.g., FFHQ).

We propose DSD-GAN to apply two discriminators in UV space and image space to learn structure and texture details in a complementary manner.

32 Learning a Sketch Tensor Space for Image Inpainting of Man-made Scenes

This paper studies the task of repairing man-made scenes. It's not simple because it's difficult to preserve the visual patterns of the image, such as edges, lines, and connected dots.
To this end, this paper uses the Sketch Tensor (ST) space for inpainting artificial scenes. In order to promote structure refinement, a multi-scale inpainting (MST) network is proposed. The new encoder-decoder structure: the encoder extracts lines from the input image. and edges, projecting them into ST space. From this space, the decoder learns to recover the input image. Extensive experiments verify the effectiveness.

33 Parallel Multi-Resolution Fusion Network for Image Inpainting

Traditional deep image inpainting methods are based on autoencoder architecture, where the spatial details of the image will be lost during downsampling, resulting in poor generation results. Furthermore, the deep structural information and shallow texture information of the autoencoder architecture are not well integrated. To this end, a parallel multi-resolution inpainting network with multi-resolution partial convolution is designed, where the low-resolution branch focuses on the global structure, while the high-resolution branch focuses on local texture details. Experimental results show that the method effectively fuses structural and texture information, producing more realistic results than state-of-the-art methods.

34 CR-Fill: Generative Image Inpainting with Auxiliary Contextual Reconstruction

Recent image inpainting methods use layers of attention mechanisms to motivate the generator to complete missing regions by borrowing feature blocks from known regions. Due to the lack of supervision signals for the correspondence between missing and known regions, suitable reference features may not be found, resulting in artifacts in the results. In addition, it calculates the pairwise similarity of the entire feature map during the inference process, which brings a lot of computational overhead.
To this end, joint training of auxiliary context reconstruction tasks is proposed, and the non-attentional generator can also learn the ability to borrow peripheral features to repair, making the output reasonable. The auxiliary branch can be viewed as a learnable loss function, named as contextual reconstruction (CR) loss, where the query-reference feature similarity and reference-based reconstructor are jointly optimized with the inpainting generator.
Experimental results show that the proposed inpainting model outperforms state-of-the-art models in quantitative and visual performance.
https://github.com/zengxianyu/crfill