点击上方“机器学习与生成对抗网络”，关注"星标"
获取有趣、好玩的前沿干货！
2019 cvpr：Pluralistic Image Completion
https://arxiv.xilesou.top/pdf/1903.04227.pdf
https://github.com/lyndonzheng/Pluralistic-Inpainting
对于每个输入（masked input），大多数图像补全方法只能产生一种结果（尽管有许多合理的其它可能结果）。基于学习的方法里，通常每个标签只有一个ground true（目标参照图象GT）。即便从有条件VAE采样、仍然会多样性不足。本文提出了一种多元化图像补全方法。
提出了一种具有两个平行路径的概率学习框架。一个是重建路径（网络），它只利用对应一个的GT去获取缺失区域的先验信息，并完成重建；另一个是生成路径（网络），它将条件先验耦合于重建路径所获得的分布。在对抗方式下完成训练，训练完后只用生成路径。
提出了一种新的、建模长短区域关系的注意力机制，以提高图像一致性。
在建筑、人脸（Celeba-HQ）、ImageNet等数据集上不仅取得了更好的补全效果，在多样性上也合理、令人信服。
方法
定义 为原始完整图像， 是被遮挡的（掩masked）图像，则经典的图像补全方法是去学习映射 ，它们是确定性的。
本文还定义 表示 的“反”，也就是它仅仅是由被遮挡部分构成，而本文的目标是从 采样去恢复。
损失函数：
进一步地：
分布正则化（参照VAE/CVAE）：
对于重建路径：
对于生成路径：
（这里的KL，个人感觉不应该带负号啊？？
图像外表匹配：
对于重建路径，约束重建图像和目标参考GT相似：
对于生成路径，约束生成和GT相似：
对抗损失：
在特征层面的约束和LSGAN损失：
网络结构之attention设计
在前面网络整体的图所示，decoder之前的红色模块即为本文所提出的attention设计：
部分效果
公众号近期荐读：
GAN&CV交流群，无论小白还是大佬，诚挚邀您加入！
一起讨论交流！长按备注【进群】加入：
更多分享、长按关注本公众号：
GAN图像修复/补全生成-相关阅读：
001 (2020-04-8) Attentive Normalization for Conditional Image Generation
https://arxiv.xilesou.top/pdf/2004.03828.pdf
002 (2020-03-27) GAN-based Priors for Quantifying Uncertainty
https://arxiv.xilesou.top/pdf/2003.12597.pdf
003 (2020-02-6) Automatic Segmentation and Visualization of Choroid in OCT with Knowledge Infused Deep Learning
https://arxiv.xilesou.top/pdf/2002.04712.pdf
004 (2020-02-7) Local Facial Attribute Transfer through Inpainting
https://arxiv.xilesou.top/pdf/2002.03040.pdf
005 (2020-02-5) Domain Embedded Multi-model Generative Adversarial Networks for Image-based Face Inpainting
https://arxiv.xilesou.top/pdf/2002.02909.pdf
006 (2020-02-6) Image Fine-grained Inpainting
https://arxiv.xilesou.top/pdf/2002.02609.pdf
007 (2020-02-4) Pixel-wise Conditioned Generative Adversarial Networks for Image Synthesis and Completion
https://arxiv.xilesou.top/pdf/2002.01281.pdf
008 (2020-01-11) Symmetric Skip Connection Wasserstein GAN for High-Resolution Facial Image Inpainting
https://arxiv.xilesou.top/pdf/2001.03725.pdf
009 (2019-12-31) Erase and Restore Simple Accurate and Resilient Detection of $L 2$ Adversarial Examples
https://arxiv.xilesou.top/pdf/2001.00116.pdf
010 (2019-12-18) Unsupervised Adversarial Image Inpainting
https://arxiv.xilesou.top/pdf/1912.12164.pdf
011 (2020-02-15) Image Outpainting and Harmonization using Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1912.10960.pdf
012 (2020-03-31) Image Processing Using Multi-Code GAN Prior
https://arxiv.xilesou.top/pdf/1912.07116.pdf
013 (2019-12-9) Environment reconstruction on depth images using Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1912.03992.pdf
014 (2019-12-5) Blind Inpainting of Large-scale Masks of Thin Structures with Adversarial and Reinforcement Learning
https://arxiv.xilesou.top/pdf/1912.02470.pdf
015 (2020-03-19) PiiGAN Generative Adversarial Networks for Pluralistic Image Inpainting
https://arxiv.xilesou.top/pdf/1912.01834.pdf
016 (2019-11-2) Pixel-wise Conditioning of Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1911.00689.pdf
017 (2019-10-23) Facial Expression Restoration Based on Improved Graph Convolutional Networks
https://arxiv.xilesou.top/pdf/1910.10344.pdf
018 (2019-09-27) Region-wise Generative Adversarial ImageInpainting for Large Missing Areas
https://arxiv.xilesou.top/pdf/1909.12507.pdf
019 (2019-12-23) Physics-informed semantic inpainting Application to geostatistical modeling
https://arxiv.xilesou.top/pdf/1909.09459.pdf
020 (2019-08-19) Boundless Generative Adversarial Networks for Image Extension
https://arxiv.xilesou.top/pdf/1908.07007.pdf
021 (2019-08-19) Fully Automated Image De-fencing using Conditional Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1908.06837.pdf
022 (2019-08-16) The Angel is in the Priors Improving GAN based Image and Sequence Inpainting with Better Noise and Structural Priors
https://arxiv.xilesou.top/pdf/1908.05861.pdf
023 (2019-08-14) Faster Unsupervised Semantic Inpainting A GAN Based Approach
https://arxiv.xilesou.top/pdf/1908.04968.pdf
024 (2019-10-29) Generative Modeling by Estimating Gradients of the Data Distribution
https://arxiv.xilesou.top/pdf/1907.05600.pdf
025 (2019-06-14) Single-Channel Signal Separation and Deconvolution with Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1906.07552.pdf
026 (2019-06-6) How to make a pizza Learning a compositional layer-based GAN model
https://arxiv.xilesou.top/pdf/1906.02839.pdf
027 (2019-09-28) Fashion Editing with Adversarial Parsing Learning
https://arxiv.xilesou.top/pdf/1906.00884.pdf
028 (2019-06-3) GazeCorrection Self-Guided Eye Manipulation in the wild using Self-Supervised Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1906.00805.pdf
029 (2019-05-23) Generative Imaging and Image Processing via Generative Encoder
https://arxiv.xilesou.top/pdf/1905.13300.pdf
030 (2020-01-30) Training Generative Adversarial Networks from Incomplete Observations using Factorised Discriminators
https://arxiv.xilesou.top/pdf/1905.12660.pdf
031 (2019-09-27) Invertible generative models for inverse problems mitigating representation error and dataset bias
https://arxiv.xilesou.top/pdf/1905.11672.pdf
032 (2020-03-6) PEPSI++ Fast and Lightweight Network for Image Inpainting
https://arxiv.xilesou.top/pdf/1905.09010.pdf
033 (2019-05-6) The Missing Data Encoder Cross-Channel Image Completion\\with Hide-And-Seek Adversarial Network
https://arxiv.xilesou.top/pdf/1905.01861.pdf
034 (2019-07-10) Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting
https://arxiv.xilesou.top/pdf/1904.07475.pdf
035 (2019-04-15) A deep learning framework for quality assessment and restoration in video endoscopy
https://arxiv.xilesou.top/pdf/1904.07073.pdf
036 (2019-03-28) GANs-NQM A Generative Adversarial Networks based No Reference Quality Assessment Metric for RGB-D Synthesized Views
https://arxiv.xilesou.top/pdf/1903.12088.pdf
037 (2019-11-6) Implicit Generation and Generalization in Energy-Based Models
https://arxiv.xilesou.top/pdf/1903.08689.pdf
038 (2019-03-10) A Sinogram Inpainting Method based on Generative Adversarial Network for Limited-angle Computed Tomography
https://arxiv.xilesou.top/pdf/1903.03984.pdf
039 (2019-10-23) GAN-based Projector for Faster Recovery with Convergence Guarantees in Linear Inverse Problems
https://arxiv.xilesou.top/pdf/1902.09698.pdf
040 (2019-02-10) Cross-spectral Face Completion for NIR-VIS Heterogeneous Face Recognition
https://arxiv.xilesou.top/pdf/1902.03565.pdf
041 (2019-01-25) Diversity-Sensitive Conditional Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1901.09024.pdf
042 (2019-01-9) Detecting Overfitting of Deep Generative Networks via Latent Recovery
https://arxiv.xilesou.top/pdf/1901.03396.pdf
043 (2019-01-11) EdgeConnect Generative Image Inpainting with Adversarial Edge Learning
https://arxiv.xilesou.top/pdf/1901.00212.pdf
044 (2019-07-30) Exploiting the Inherent Limitation of L0 Adversarial Examples
https://arxiv.xilesou.top/pdf/1812.09638.pdf
045 (2019-11-27) Face Completion with Semantic Knowledge and Collaborative Adversarial Learning
https://arxiv.xilesou.top/pdf/1812.03252.pdf
046 (2018-12-3) Semantic Image Inpainting Through Improved Wasserstein Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1812.01071.pdf
047 (2019-04-9) Context Encoding Chest X-rays
https://arxiv.xilesou.top/pdf/1812.00964.pdf
048 (2019-02-26) Void Filling of Digital Elevation Models with Deep Generative Models
https://arxiv.xilesou.top/pdf/1811.12693.pdf
049 (2019-01-15) SEIGAN Towards Compositional Image Generation by Simultaneously Learning to Segment Enhance and Inpaint
https://arxiv.xilesou.top/pdf/1811.07630.pdf
050 (2020-01-11) On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs
https://arxiv.xilesou.top/pdf/1811.07104.pdf
051 (2018-10-20) Improved Techniques for GAN based Facial Inpainting
https://arxiv.xilesou.top/pdf/1810.08774.pdf
052 (2018-10-15) Adversarial Inpainting of Medical Image Modalities
https://arxiv.xilesou.top/pdf/1810.06621.pdf
053 (2019-02-15) Empty Cities Image Inpainting for a Dynamic-Object-Invariant Space
https://arxiv.xilesou.top/pdf/1809.10239.pdf
054 (2018-09-6) Dense Pose Transfer
https://arxiv.xilesou.top/pdf/1809.01995.pdf
055 (2018-08-29) Chest X-ray Inpainting with Deep Generative Models
https://arxiv.xilesou.top/pdf/1809.01471.pdf
056 (2018-08-25) Painting Outside the Box Image Outpainting with GANs
https://arxiv.xilesou.top/pdf/1808.08483.pdf
057 (2019-07-18) Cross-view image synthesis using geometry-guided conditional GANs
https://arxiv.xilesou.top/pdf/1808.05469.pdf
058 (2018-08-6) X-GANs Image Reconstruction Made Easy for Extreme Cases
https://arxiv.xilesou.top/pdf/1808.04432.pdf
059 (2018-11-2) Latent Convolutional Models
https://arxiv.xilesou.top/pdf/1806.06284.pdf
060 (2018-06-13) Self-Supervised Feature Learning by Learning to Spot Artifacts
https://arxiv.xilesou.top/pdf/1806.05024.pdf
061 (2018-07-17) AVID Adversarial Visual Irregularity Detection
https://arxiv.xilesou.top/pdf/1805.09521.pdf
062 (2018-05-21) Unsupervised Deep Context Prediction for Background Foreground Separation
https://arxiv.xilesou.top/pdf/1805.07903.pdf
063 (2018-06-4) An Unsupervised Approach to Solving Inverse Problems using Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1805.07281.pdf
064 (2019-07-20) Generative Steganography by Sampling
https://arxiv.xilesou.top/pdf/1804.10531.pdf
065 (2018-04-25) Generative Model for Heterogeneous Inference
https://arxiv.xilesou.top/pdf/1804.09858.pdf
066 (2018-05-14) The Reincarnation of Grille Cipher A Generative Approach
https://arxiv.xilesou.top/pdf/1804.06514.pdf
067 (2018-04-3) Correlated discrete data generation using adversarial training
https://arxiv.xilesou.top/pdf/1804.00925.pdf
068 (2018-03-27) Structural inpainting
https://arxiv.xilesou.top/pdf/1803.10348.pdf
069 (2018-05-10) Digital Cardan Grille A Modern Approach for Information Hiding
https://arxiv.xilesou.top/pdf/1803.09219.pdf
070 (2018-03-27) Image Inpainting using Block-wise Procedural Training with Annealed Adversarial Counterpart
https://arxiv.xilesou.top/pdf/1803.08943.pdf
071 (2018-03-20) Patch-Based Image Inpainting with Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1803.07422.pdf
072 (2018-07-5) Generating Realistic Geology Conditioned on Physical Measurements with Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1802.03065.pdf
073 (2018-08-30) Deep Structured Energy-Based Image Inpainting
https://arxiv.xilesou.top/pdf/1801.07939.pdf
074 (2017-12-20) Context-Aware Semantic Inpainting
https://arxiv.xilesou.top/pdf/1712.07778.pdf
075 (2017-12-11) GibbsNet Iterative Adversarial Inference for Deep Graphical Models
https://arxiv.xilesou.top/pdf/1712.04120.pdf
076 (2018-08-6) Discriminative Region Proposal Adversarial Networks for High-Quality Image-to-Image Translation
https://arxiv.xilesou.top/pdf/1711.09554.pdf
077 (2017-11-26) Semantically Consistent Image Completion with Fine-grained Details
https://arxiv.xilesou.top/pdf/1711.09345.pdf
078 (2017-11-17) Improving Consistency and Correctness of Sequence Inpainting using Semantically Guided Generative Adversarial Network
https://arxiv.xilesou.top/pdf/1711.06106.pdf
079 (2019-04-2) Perceptual Adversarial Networks for Image-to-Image Transformation
https://arxiv.xilesou.top/pdf/1706.09138.pdf
080 (2017-12-6) r-BTN Cross-domain Face Composite and Synthesis from Limited Facial Patches
https://arxiv.xilesou.top/pdf/1706.00556.pdf
081 (2017-09-1) Adversarial Inverse Graphics Networks Learning 2D-to-3D Lifting and Image-to-Image Translation from Unpaired Supervision
https://arxiv.xilesou.top/pdf/1705.11166.pdf
082 (2017-10-12) CVAE-GAN Fine-Grained Image Generation through Asymmetric Training
https://arxiv.xilesou.top/pdf/1703.10155.pdf
083 (2016-11-21) Context Encoders Feature Learning by Inpainting
https://arxiv.xilesou.top/pdf/1604.07379.pdf