Generative Modeling by Estimating Gradients of the Data Distribution reading notes

Overview

The paper proposes a generative model and uses it for image generation tasks.
The paper first introduces the traditional score-based generative modeling method, then analyzes the existing problems of traditional score-based generative modeling, and finally proposes an algorithm to solve the problem, noise conditional score network.

Introduction to traditional score-based generative modeling

Assume that the data in the data set obeys $p_{data}\left(x\right)$ distribution.
The goal of generative modeling is to learn a generative model to generate obey$p_{data}A\; new\; sample\; from\; the\; (x)$ distribution.
Define score function as the probability density function$p\left(x\right)$ value∇${\nabla }_{\mathbf{x}}\log p\left(x\right)$ .
Define score network as a parameterθ \thetaNeural network s of$\theta$$s_{}$, which attempts to approximate the score function.
Score-based generative modeling generates new samples that conform to the distribution by learning the score function and adding Langevin dynamics sampling. The steps are shown in the figure below:

score matching

Using the score matching algorithm, we can directly train a score network $s_{}\left(x\right)$ to estimate${\nabla }_{\mathbf{x}}\log p_{data}\left(x\right)$ without training the model to estimate$p_{data}\left(x\right)$ . The advantage is that the normalization constant in the probability density function can be avoided. For details,see the introduction to the score matching algorithm.
The optimization objectives of the score matching algorithm are as follows:
$$\frac{1}{2}{\mathbb{E}}_{p_{data}}[\Vert {\mathbf{s}}_{}(\mathbf{x})-{\nabla }_x\log p_{data}(\mathbf{x}){\Vert }_2^2]$$ The above formula needs to calculate${\nabla }_{\mathbf{x}}\log p_{data}\left(x\right)$ , this is a non-parametric estimation problem and is not easy to calculate. Fortunately, the above formula is equivalent to$${\mathbb{E}}_{p_{data}}[\text{tr}({\nabla }_x{\mathbf{s}}_{}(\mathbf{x}))+\frac{1}{2}\Vert {\mathbf{s}}_{}(\mathbf{x}){\Vert }_2^2]$$ Minimize the above formula to find$s_{}\left(x\right)$ . In reality, the expectation can be replaced by the average of the sample.
However, high-dimensional data calculation$\text{tr}({\nabla }_{\mathbf{x}}{\mathbf{s}}_{}(x))$ is very complex. Denoising score matching and Sliced ​​score matching are two commonly used improvement methods for high-dimensional big data.

Langevin dynamics

Langevin dynamics is a method that only requires score function ${\nabla }_x\log p\left(x\right)$ can be obtained from the probability density function$p_{data}The\; sampling\; method\; in\; (x)$ is a Markov chain Monte Carlo (MCMC) method.
Given an initial distribution${\tilde{\mathbf{x}}}_0\sim \pi \left(x\right)$ , and fixed step size$ϵ>0$ , the Langevin method repeats the following steps cyclically:
$${\tilde{x}}_t={\tilde{x}}_{t-1}+\frac{}{2}{\nabla }_{\mathbf{x}}\log p({\tilde{x}}_{t-1})+\sqrt{ϵ}{\mathbf{z}}_t$$Expand $z_t\sim \mathcal{N}(0,I)$。ϵ$ϵ\to 0$，$T\to \infty$时，${\tilde{\mathbf{x}}}_T$The distribution is $p_{data}(\mathbf{x})$。

Problems with traditional score-based generative modeling

Problems with the manifold hypothesis

The manifold hypothesis states that real-world data tend to be concentrated on low-dimensional manifolds embedded in high-dimensional space (also called environment space).

The manifold hypothesis states that data in the real world tend to concentrate on low dimensional manifolds embedded in a high dimensional space (a.k.a., the ambient space).

Under the manifold assumption, score-based generative models have two problems:

1. ${\nabla }_{\mathbf{x}}\log p_{data}\left(x\right)$ is not defined on low-dimensional manifolds.
2. The score estimator is consistent only when the data distribution is the entire space.

Problems with low-density areas

1. In low-density areas of data, there are not enough data samples to accurately learn the score function.
2. When two peaks (modes) of a data distribution are separated by a low-density region, Langevin dynamics will not be able to correctly recover the relative weights of the two peaks in a reasonable time and may not converge to the true distribution. For example, suppose $p_{data}(\mathbf{x})=\pi p_1\left(x\right)+(1-\pi )p_2\left(x\right)$ , and$p_1$and $p_2$There are no intersecting support sets. After derivation, the weight $\pi$ will not affect the score function.

Noise Conditional Score Network

In order to solve the above problems, the author improved the traditional score-based generative modeling.
The author proposes to 1) use various noise levels to perturb the data; 2) use a conditional score network to simultaneously estimate the scores corresponding to all noise levels.
After the conditional score network training is completed, when using Langevin dynamics to generate samples, the score corresponding to high noise is initially used, and then the noise is gradually reduced. This helps smoothly transfer the benefits of high noise to low noise. The data with low noise interference is almost indistinguishable from the original data.

The principle is as shown below. By adding noise, the data can fill the low data density area to improve the accuracy of the estimated score.
Larger noise can obviously cover more low-density areas to get better score estimates, but it will overly corrupt the data and change the original distribution significantly. On the other hand, smaller noise will result in less corruption of the original data distribution, but will not cover low-density areas as much as we would like. Therefore, the author proposed to use multi-scale noise interference.

Noise Conditional Score Networks

$\{p_i{\}}_{i=1}^L$is a series of noise levels that satisfy the condition $\frac{p_1}{p_2}=\cdots =\frac{p_{L-1}}{p_L}>1$ ,$q_{}(\mathbf{x})\triangleq \int p_{data}(\mathbf{t})\mathcal{N}(\mathbf{x}|\mathbf{t},p^{2}I)dt$is the data distribution after noise disturbance. we have to learnoneNoise conditional score network $s_{}(\mathbf{x},\sigma )$ to estimate the fraction of noisy data, that is,$s_{}(\mathbf{x},s)\approx {\nabla }_{\mathbf{x}}\log q_{}\left(x\right)$ . Note that the score network here is a conditional score network, and the input is compared to the traditional$s_{}\left(x\right)$ has one more$\sigma$ .
The author considers the problem of image generation, so$s_{}(x,$The author chose U-Net for the structure of$\sigma$$)\; .$

For the training of the noise conditional score network, the noise distribution chosen by the author is
$q_{}(\tilde{\mathbf{x}}|\mathbf{x})=\mathcal{N}(\tilde{x}|\mathbf{x},p^{2I})$.$\_$
For a given noise$\sigma$ ,one of the equations:
$$l(i,s)=\frac{1}{2}{\mathbb{E}}_{p_{data}}{E}_{\tilde{\mathbf{x}}\sim N(x,p^{2I}}[\Vert s_{}(\tilde{x},s)+\frac{\tilde{x}-\mathbf{x}}{p^2}{\Vert }_2^2]$$ Let us solve the problem of a specific inequality:
$$L(i;\{p_i{\}}_{i=1}^L)\triangleq \frac{1}{L}\sum _{i=1}^{L}l(p_i\left)l\right(i,p_i)$$ amongλ$l\left(p_i\right)$ is the weight.

annealed Langevin dynamics

In the noise conditional fractional network $s_{}(\mathbf{x};After\; the\; training\; of\; \sigma )$ is completed, the author proposes the annealed Langevin dynamics algorithm to generate samples, as shown in Algorithm 1. Use the score corresponding to high noise first, and then gradually reduce the noise.

reference

Yang Song’s blog《Generative Modeling by Estimating Gradients of the Data Distribution》
NIPS 2019《Generative Modeling by Estimating Gradients of the Data Distribution》