GAN principles (GAN CHANG notes)

We usually do Gerneration with GAN.

For a picture x, its usually a popular distributed in high-dimensional space. In the figure below, the blue area is located just a great point probability in our database in.

So what is the main task of image generation is it?

To define the distribution of original data , in order to give a set $\Theta$ as a probabilistic model parameters $P_{G}(x,\Theta )$ , the model can be obtained by a learning generator. We hope, and that is to find the appropriate parameters $\theta$ , so that the original data set of samples x probability G defined by the distribution of $P_{G}(x,\theta )$ probability appears as large as possible.

In order to find such a parameter, we can start database m samples were randomly selected, and the probability of reaching each sample is generated based on the calculation $P_{G}(x^{\theta };\theta )$ , then these m multiply the probability corresponding to the sample, to obtain likelihood corresponding function.

Further specified likelihood function, we find maximum likelihood estimation parameters obtained $\theta ^{*}$ , but also happens so that the KL divergence generated image and the real image is minimized.

Our generator is on the nature of a network, the network defines a probability distribution $P_{G}(x)$ , we hope is that $P_{G}(x)$ with $P_{data}(x)$ as close as possible. The question is how to get $P_{G}(x)$ it?

We put aside his previous question regardless. We turned to look discriminator.

Although we do not know $P_{G}(x)$ and $P_{data}(x)$ specific form, but we can sample the real picture and generate images.

For a given generator, we want to optimize the loss function in the following figure, the cross-entropy loss function is very similar to the form of the two-class problems.

We found that when the real picture and generate a picture with a small divergence, discriminator is difficult to discern the true picture and generate images. However, when both have a large divergence, discriminator is relatively easily be able to identify them.