Author: Zen and the Art of Computer Programming
1 Introduction
After the great success of GANs (Generative Adversarial Networks) in recent years, many researchers began to explore some optimization techniques in GAN training, such as using gradient clipping, adding dropout and other methods to improve the quality of generated samples and suppress model overfitting. Recently, at ICML2019, researchers proposed a new technology Random Residual Block (RRB) to improve the training of GAN, which can effectively solve the problem of GAN generated image quality degradation in some cases.
This article first introduces GAN and its related technologies, and then discusses how to use RRB to improve the training process of GAN. Finally, we will show the experimental results on the CIFAR-10 dataset, proving that RRB can significantly improve the quality of GAN-generated images while avoiding overfitting.
2. Overview of related work
In this section, we first briefly review the basics of deep learning-based generative models, including GAN, VAE, InfoGAN, etc. Next, we describe how to apply these models to real-world scenarios.
2.1 Introduction to generative models
The generative model is a method based on statistical learning, which generates new data instances or evaluates the quality of the generative model by learning the probability distribution mapped from the latent space to the data space. In general, generative models can be divided into two categories:
- Supervised Learning: This type of method takes the label information of the given data as input and learns a mapping function F so that the output samples conform to the characteristics of the real data distribution. Supervised learning methods include VAE, GAN, InfoGAN, etc.
- Unsupervised Learning: This type of method does not require label information, and finds the distribution of input data through self-learning. Typical unsupervised learning methods include EM algorithm, hidden Markov model, Deep Boltzm