Generative Adversarial Network (GAN) Principle Derivation and Network Construction Ideas

0 Preface

Imagine this scenario: you are the owner of a studio, and your studio is mainly used to produce fakes of famous paintings; while the real famous paintings are created by predecessors and stored in the collection room. Your counterfeit paintings will be appraised by connoisseurs together with the real ones, and your ultimate goal is to become a master craftsman who can reconcile the real ones. The road to the goal is naturally very bumpy, the first thing you have to do is to confuse the real with the fake. In fact, it is easier to confuse the fake with the real. After all, you can say that the fake is the real one if you deceive a young boy, but it is very difficult for authoritative appraisal experts to be blind. And since your goal is to become a master craftsman, then naturally you will not just satisfy and deceive Xiaobai. So you find a person who is determined to become an appraisal expert, and you ask him to identify your fake paintings and real paintings, and he will tell you the results of his appraisal, so that you can know whether your paintings have been authenticated, so that you can be more accurate. Improve your skills well; on the contrary, you will also tell him whether you have identified your fake paintings, so that his identification skills will continue to improve.

In the above example, the master craftsman imitates famous paintings to create fakes, that is, the generator captures the potential distribution of real data samples and generates new data samples; the appraisal expert judges whether the painting sent to him is genuine or fake, that is, the discriminator judges whether the input is real data Or generated samples.

1 Introduction

Generative Adversarial Nets (GAN) was proposed by Ian J. Goodfellow in 2014. The basic idea of ​​GAN comes from the two-person zero-sum game of game theory, which consists of a generator and a discriminator, and is trained by confrontational learning. The goal is to estimate the underlying distribution of data samples and generate new data samples. The optimization process of GAN is a minimax game problem. The optimization goal is to achieve Nash equilibrium, so that the generator can estimate the distribution of data samples. GAN is being widely studied in the fields of image and visual computing, speech and language processing, information security, chess games, etc., and has great application prospects.

2 Basic principles

The core idea of ​​GAN comes from the Nash equilibrium of game theory. It sets the two parties participating in the game as a generator (Generator) and a discriminator (Discriminator). The purpose of the generator is to learn the real data distribution as much as possible, and the purpose of the discriminator is to correctly determine whether the input data comes from real data. It still comes from the generator; in order to win the game, the two game participants need to continuously optimize and improve their own generation ability and discrimination ability respectively. This learning optimization process is to find a Nash equilibrium between the two.

The calculation process and structure of GAN are shown in the above figure. Any differentiable function can be used to represent the generator and discriminator of GAN. Therefore, we can use the differentiable function $D$ and$G$ to represent the discriminator and the generator respectively, and their inputs are the real data distribution$x$ and the random variable$z$ 。$G$ will be the random variable$z$ (eg Gaussian distribution, denoted as$p_z$) maps to $G(z)$，$G\left(z\right)$ obeys a distribution pdata p_{data}as close as possible to the real data$p_{data}$The probability distribution, this probability distribution is usually recorded as $p_g$。

For discriminator $D$ , returns 1 if the input is from real data; if the input is$G\left(z\right)$ , then mark it as 0. Here$The\; goal\; of\; D$ is to realize the two-category discrimination of data sources:

• True - derived from real data $distribution\; of\; x$ ;
• Pseudo - Fake data from the generator $G(z)$。

And $The\; goal\; of\; G$ is to make the fake data G ( z ) G(z)generated by itself$G\left(z\right)$ inDDExpress$D$$D\left(G\right(z))$ and real data$x$ inDDExpress$D$$D\left(x\right)$ agrees.

$D$ and$The\; process\; of\; G$ fighting against each other and iterative optimization makes the performance of the two continuously improve. When the final$When\; D\text{'}$ s discriminative ability is improved to a certain extent (becoming an appraisal expert), and the data source cannot be correctly identified, it can be considered that the generator$G$ has learned the distribution of real data (becoming a master of the craft).

3 Objective function

According to the content of Section 2, the goal of the discriminator is, if the input $x$ from$p_{data}$, then $D\left(x\right)$ should be as large as possible; if input$x$ from$p_g$ ，则 $1-D\left(G\right(z))$ should be as large as possible. In order to make the objective function easier to express, take the logarithm of the two, that is,$\log D(x)$ 与 $\log (1-D(G(z))).Independent$ :
$$\underset{D}{\max }{E}_{x\sim p_{data}}[\log D(x)]+E_{z\sim p_z}[\log (1-D\left(G\right(z)))]$$
Here the expectation is$\frac{1}{N}{\sum }_{i=1}^N\log D(x_i)$。

The goal of the generator is to generate $G\left(z\right)$ is as far as possible identified by the discriminator$D$ is recognized as real data, that is,$D\left(G\right(z))$ is as large as possible, or$1-D\left(G\right(z))$ ​is as small $as\; possible$Similarly, taking the logarithm, the mathematical formula is:
$$\underset{G}{\min }{E}_{z\sim p_z}[\log (1-D\left(G\right(z)))]$$
According to the objective functions of the two,$D$ and$GBefore$ there was a relationship that what he gained must be what he lost, and the two constituted a zero-sum game.

Remember the example mentioned in Section 0, we have to fool not only the little boy, but also the appraisal experts. corresponds to $D$ and$G$ is on,$The\; fake\; samples\; generated\; by\; G$ can fool the top-notch$D.$ _ Therefore, in the idea of ​​GAN,$D$ is just a tool, our ultimate goal is still$G$ , so we get the overall goal of GAN optimization, which is a minimax problem, described as follows:
$$\underset{G}{\min }\underset{D}{\max }{E}_{x\sim p_{data}}[\log D(x)]+E_{z\sim p_z}[\log (1-D(G(z)))]$$

4 Existence of Global Optimal Solution

In the previous section, we logically constructed the overall objective function of GAN, and $p_{data}=p_{g}Should$ reach the global optimal solution (ie$Ggenerated$ fake samples are indistinguishable from real samples).

For the learning process of GAN, we need to train the model $Dto$ maximize the discriminant data from real data$xor$ pseudo-data distribution$G\left(z\right)Accuracy$ rate, at the same time, we need to train the model$Gto$ minimize$\log (1-D(G(z)))$ ​. Alternate optimization can be used here: first fix the generator$G,$ optimize the discriminator$D,$ such that$DThe$ discriminant accuracy is maximized; then the fixed discriminator$D,$ optimized generator$G,$ such that$The\; discriminative\; accuracy\; of\; Dis$ minimized. If and only if$p_{data}=p_{g}to$ reach the global optimal solution.

In actual training, when the parameters are updated in the same round, generally for $Dparameter$ update$k$ ​against$The\; parameters\; of\; Gare$ updated once.

Next we prove that pdata = pg p_{data} = p_g in the case of the global optimal solution$p_{data}=p_g$Whether it is established or not. Since $z\sim p_z$ 时 G ( z ) G(z) The result of$G$$($$z$$)$$x\sim p_g$when $x$ is equivalent, so we can write
$$V(D,G)=E_{x\sim p_{data}}[\log D(x)]+E_{x\sim p_g}[\log (1-D(x))]$$
First, in the given generator$In\; the\; case\; of\; G$ , we only consider optimizing the discriminator$D$ ，即 ${\max }_{D}V(D,G)$，而
$$\begin{array}{rl}V(D,G)& =E_{x\sim p_{data}}[\log D(x)]+E_{x\sim p_g}[\log (1-D(x))]\\ & =\int p_{data}\cdot \log Ddx+\int p_g\log (1-D)dx\\ & =\int [p_{data}\cdot \log D+p_g\log (1-D)]dx\end{array}$$

The above equation uses the basic knowledge of probability theory, a brief review:

Let continuous random variable XXThe probability density of$X$$f\left(x\right)$ , if the integral${\int }_{-\infty }^{+\infty }xf\left(x\right)dx$ is absolutely convergent, then this integral is called a random variableXXThe mathematical expectationormeanof$X$ , denoted as$E(X)$ ，即
$$E\left(x\right)={\int }_{-\infty }^{+\infty }xf\left(x\right)dx$$
assume continuous random variableXXThe probability density of$X$$f\left(x\right)$ , if the integral${\int }_{-\infty }^{+\infty }g\left(x\right)f\left(x\right)dx$ is absolutely convergent, then this integral is called a random variable$Y=$The mathematical expectationormeanof$g$$($$X$$)$ , that is,
$$E\left(AND\right)=E[g(X)]={\int }_{-\infty }^{+\infty }g(x)f(x)dx$$

Since only $D,$ so consider making$V(D,G)$ to$D$ 求偏导
$$\begin{array}{rl}\frac{\partial }{\partial D}V(D,G)& =\frac{\partial }{\partial D}\int [p_{data}\cdot \log D+p_g\log (1-D)]dx\\ & =\int \frac{\partial }{\partial D}[p_{data}\cdot \log D+p_g\log (1-D\left)\right]dx\\ & =\int [p_{data}\cdot \frac{1}{D}+p_g\frac{-1}{1-D}]dx\end{array}$$
What we want is ${\max }_{D}V(D,G)$ , so let the derivative result of the above formula be equal to 0, namely:
$$\int [p_{data}\cdot \frac{1}{D}+p_g\frac{-1}{1-D}]dx=0$$
can be obtained at this time:
$$D_G^{\ast }=\frac{p_{data}}{p_{data}+p_g}$$
This is the optimal solution of the discriminator.

Again, $D\left(x\right)$ means input$x$ comes from real samples ($x\sim p_{data}$)The probability. $G\left(z\right)$ means input a$z\sim p_z$, output a $x\sim p_g$。

将 $D_G^{\ast }=\frac{p_{data}}{p_{data}+p_g}$Substituted into the total objective function, there are:
$$\begin{array}{rl}\underset{G}{\min }\underset{D}{\max }V(D,G)& =\underset{G}{\min }V(D_G^{\ast },G)\\ & =\underset{G}{\min }{E}_{x\sim p_{data}}\left[\log D_G^{\ast }\right]+E_{x\sim p_g}[\log (1-D_G^{\ast })]\\ & =\underset{G}{\min }{E}_{x\sim p_{data}}\left[\log \frac{p_{data}}{p_{data}+p_g}\right]+E_{x\sim p_g}\left[\log \right(1-\frac{p_{data}}{p_{data}+p_g}\left)\right]\\ & =\underset{G}{\min }{E}_{x\sim p_{data}}\left[\log \frac{p_{data}}{p_{data}+p_g}\right]+E_{x\sim p_g}\left[\log \frac{p_g}{p_{data}+p_g}\right]\\ & =\underset{G}{\min }{E}_{x\sim p_{data}}\left[\log \right(\frac{p_{data}}{(p_{data}+p_g)/2}\cdot \frac{1}{2}\left)\right]+E_{x\sim p_g}\left[\log \right(\frac{p_g}{(p_{data}+p_g)/2}\cdot \frac{1}{2}\left)\right]\\ & =\underset{G}{\min }{E}_{x\sim p_{data}}\left[\log \right(\frac{p_{data}}{(p_{data}+p_g)/2}\left)\right]+E_{x\sim p_{data}}\left[\log \right(\frac{p_g}{(p_{data}+p_g)/2}\left)\right]-\log 2-\log 2\\ & =\underset{G}{\min }WL(p_{data}||\frac{p_{data}+p_g}{2})+WL(p_g||\frac{p_{data}+p_g}{2})-\log 4\\ & \ge -\log 4\end{array}$$

The above formula uses KL divergence, that is, relative entropy (value range is $[0,+\infty )$）：

Let $P(x),Q\left(x\right)$ is the random variable$Two\; probability\; distributions\; on\; X$ , in the case of discrete and continuous random variables, the relative entropy is defined as:
$$KL(P||Q)=\sum P(x)\log \frac{P(x)}{Q(x)}$$

$$KL(P||Q)=\int P\left(x\right)\log \frac{P\left(x\right)}{Q\left(x\right)}dx$$

The concept, principle and derivation of the specific KL divergence are being written and written (still create a folder).

Because in the above formula as the denominator $p_{data}+p_{g}is$ the addition of two probability distributions, the value range is$[0,2]$ ​, the probability distribution as $a$That is$\log \frac{p_g}{p_{data}+p_g}$ 重构成 $\log (\frac{p_{data}}{(p_{data}+p_g)/2}\cdot \frac{1}{2})$ 。

显然，当 $p_{data}=\frac{p_{data}+p_g}{2}=p_g$When , the equality sign holds. So $p_g^{\ast }=p_{data}$, at this time $D_G^{\ast }=\frac{1}{2}$. This shows that when the total objective function of GAN reaches the global optimal solution, the generator $G$ can obey the input$p_z$The random variable zz distributed$z$ maps to a subject$p_g$The random variable $G\left(z\right)$ , and at this time$p_g=p_{data}$. And the discriminator $D$ can no longer tell whether the input input is a real sample or a fake sample (regardless of the input$x$ comes from the real sample or$The\; pseudo\; samples\; generated\; by\; G$ can only output the probability of$\frac{1}{2}$）。

5 Network construction and training

Generally speaking, the discriminator $D$ and generator$Gare$ two independent neural networks. Depending on the actual use, you can choose to use multi-layer perceptron (MLP), convolutional neural network (CNN), Seq2Seq or some other neural networks.

The discriminator is generally a simple classification network, and the output value is in $[0,1],$ indicating the probability that the input comes from a real sample, greater than 0.5 means that the discriminator thinks it comes from a real sample, otherwise it thinks it is a fake sample generated by the generator.

The generator is more complicated. Its input is a noise, and this noise usually makes it obey the Gaussian distribution (others are also fine, there is no hard requirement, and the effect is good in actual application), and then the generator produces a sample similar to the real one. fake samples. For example, if we want to use GAN to generate an image, the generator inputs one or more random noises, which are mapped by the neural network and converted into a two-dimensional image.

For the training process of the network, it has been described at the beginning of Section 4, and some more will be added here.

During each iterative training process of GAN, the stochastic gradient descent method (SGD) is used to update the parameters (parameters are generally expressed as $i_D$和$i_G$). The overall idea of ​​training is: after randomly initializing the global parameters, for each round of iterative training, first train the discriminator so that it outputs $\ge 0.5$ , for pseudo samples, output$\le 0.5$ , and then fix the discriminator; then train the generator so that the samples generated by it can be ≥ 0.5 \ge 0.5as much as possible after being output by the discriminator$\ge 0.5$。

Here for SGD and $The\; loss\; function\; of\; D$ (generally the cross-entropy loss function) belongs to the basic content of the neural network and will not be repeated here.