DDPM模型——公式推导

论文传送门：Denoising Diffusion Probabilistic Models
代码实现：DDPM模型——pytorch实现
推荐视频：54、Probabilistic Diffusion Model概率扩散模型理论与完整PyTorch代码详细解读

需要的数学基础：

联合概率(Joint probability)：
$$P(A,B,C)=P(C\mid B,A)P(B,A)=P(C\mid B,A)P(B\mid A)P(A)$$
条件概率(Conditional probability)：
$$P(B,C\mid A)=P(B|A)P(C|A,B)$$
马尔可夫链(Markov Chain)：
$$p\left(X_{t+1}|X_t,\dots ,X_1\right)=p\left(X_{t+1}|X_t\right)$$
贝叶斯公式(Bayes Rule)：
$$P\left(A_i\mid B\right)=\frac{P\left(B\mid A_i\right)P\left(A_i\right)}{{\sum }_{j}P\left(B\mid A_j\right)P\left(A_j\right)}$$
正态分布(Normal distribution)$X\sim N\left(\mu ,{\sigma }^2\right)$的概率密度函数：
$$f(x)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}$$
两个正态分布$X\sim N\left({\mu }_X,{\sigma }_X^2\right)$和$Y\sim N\left({\mu }_Y,{\sigma }_Y^2\right)$的叠加：
$$U=X+Y\sim N\left({\mu }_X+{\mu }_Y,{\sigma }_X^2+{\sigma }_Y^2\right)$$
两个正态分布$p,q$的KL散度(Kullback-Leibler divergence)：
$$KL(p,q)=\log \frac{{\sigma }_q}{{\sigma }_p}+\frac{{\sigma }_p^2+{\left({\mu }_p-{\mu }_q\right)}^2}{2{\sigma }_q^2}-\frac{1}{2}$$
重参数技巧(Reparameterrization)：
$$若X\sim N\left(\mu ,{\sigma }^2\right),Y=\frac{X-\mu }{\sigma }\sim N(0,1)$$
从正态分布$X$中采样$z$，等价于从标准正态分布$Y$中采样$z^{\prime }$，$z=\mu +\sigma \times z^{\prime }$
一元二次式的配方：
$$a{x}^2+bx=a{\left(x+\frac{b}{2a}\right)}^2+C$$

概念：

$t$：时刻(加噪次数)
$T$：总时长(总加噪次数)
$\mathbf{x}$：图像
${\mathbf{x}}_0$：初始时刻图像
${\mathbf{x}}_t$：$Time\; t$ image
${\mathbf{x}}_T$: Termination moment image
$x_0$ ~ $q(x_0)$，$q\left(x_0\right)$ : Real image distribution
$p_{}(x_0):=\int p_{}(x_{0:T})d{x}_{1:T}$, $p_{}\left(x_0\right)$ : Generate image distribution
$\theta$ : (network) parameter
$b_t$: The variance
$\beta$ : noise variance sequence, length T, in$(0,1)$ Monotonically increasing within the interval

Reverse process：

The mathematical expression of the inverse diffusion process:
$$p_{}\left({\mathbf{x}}_{0:T}\right):=p\left(x_T\right)\prod _{t=1}^T{p}_{}\left(x_{t-1}\mid {\mathbf{x}}_t\right)$$ The joint probability distribution p θ ( x 0 : T ) p_{\theta}\left(\mathbf{x}_{0: T}\right) of
all time images$p_{}\left(x_{0:T}\right)$，整个过程是马尔科夫链。其中，
$$p\left({\mathbf{x}}_T\right)=\mathcal{N}\left({\mathbf{x}}_T;0,\mathbf{I}\right)$$
$p\left({\mathbf{x}}_T\right)$是标准正态分布，${\mathbf{x}}_T$为采样值，与网络参数无关。
$t$时刻去噪的数学表达：
$$p_{\theta }\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right):=\mathcal{N}\left({\mathbf{x}}_{t-1};{\mathbf{\mu }}_{\theta }\left({\mathbf{x}}_t,t\right),{\mathbf{\Sigma }}_{\theta }\left({\mathbf{x}}_t,t\right)\right)$$
${\mathbf{x}}_{t-1}$服从均值为${\mathbf{\mu }}_{\theta }\left({\mathbf{x}}_t,t\right)$，方差为${\mathbf{\Sigma }}_{\theta }\left({\mathbf{x}}_t,t\right)$的正态分布，作者在原文中将方差${\mathbf{\Sigma }}_{\theta }\left({\mathbf{x}}_t,t\right)$设为${\sigma }_t^2={\tilde{\beta }}_t=\frac{1-{\bar{\alpha }}_{t-1}}{1-{\bar{\alpha }}_t}{\beta }_t$(经实验，${\sigma }_t^2={\beta }_t$和${\sigma }_t^2={\tilde{\beta }}_t$的结果相似)，与模型参数无关(${\tilde{\beta }}_t$在后续计算中会提到)。

Forward process：

扩散过程的数学表达：
$$q\left({\mathbf{x}}_{1:T}\mid {\mathbf{x}}_0\right):=\prod _{t=1}^{T}q\left({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1}\right)$$
给定初始图像${\mathbf{x}}_0$，全部时刻($t>0$)的联合概率分布，整个过程是马尔科夫链。
$t$时刻加噪的数学表达：
$$q\left({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1}\right):=\mathcal{N}\left({\mathbf{x}}_t;\sqrt{1-{\beta }_t}{\mathbf{x}}_{t-1},{\beta }_t\mathbf{I}\right)$$
${\mathbf{x}}_t$服从均值为$\sqrt{1-{\beta }_t}{\mathbf{x}}_{t-1}$，方差为${\beta }_t$的正态分布。
使用重参数技巧，任意时刻的图像${\mathbf{x}}_t$可以由初始时刻图像${\mathbf{x}}_0$和噪声方差序列$\beta$ to determine, to simplify the expression, define$a_t:=1-b_t$，${\bar{a}}_t:={\prod }_{s=1}^t{a}_s$,but:
$$\begin{array}{rl}{\mathbf{x}}_t& =\sqrt{a_t}{x}_{t-1}+\sqrt{1-a_t}{ϵ}_{t-1}\\ & =\sqrt{a_t}(\sqrt{a_{t-1}}{x}_{t-2}+\sqrt{1-a_{t-1}}{ϵ}_{t-2})+\sqrt{1-a_t}{ϵ}_{t-1}\\ & =\sqrt{a_t{a}_{t-1}}{x}_{t-2}+\sqrt{a_t}\sqrt{1-a_{t-1}}{ϵ}_{t-2}+\sqrt{1-a_t}{ϵ}_{t-1}\\ & =\sqrt{a_t{a}_{t-1}}{x}_{t-2}+\sqrt{a_t-a_t{a}_{t-1}+1-a_t}{\overline{ϵ}}_{t-2}\\ & =\sqrt{a_t{a}_{t-1}}{x}_{t-2}+\sqrt{1-a_t{a}_{t-1}}{\overline{ϵ}}_{t-2}\\ & =\dots \\ & =\sqrt{{\bar{a}}_t}{x}_0+\sqrt{1-{\bar{a}}_t}\end{array}$$
The above formula can be rewritten, using $x_t$and $ϵ$ to express$x_0$:
$$x_0=\frac{1}{\sqrt{{\bar{a}}_t}}\left({\mathbf{x}}_t-\sqrt{1-{\bar{a}}_t}ϵ\right)$$
(the derivation process uses the superposition formula of two normal distributions)
where,${ϵ}_i$ ~ $\mathcal{N}\left(0,\mathbf{I}\right)$

Loss：

An upper bound on the negative log-likelihood:
$$\begin{array}{rl}{\mathbb{E}}_q\left[-\log p_{}\left({\mathbf{x}}_0\right)\right]& \le E_q\left[-\log p_{}\left(x_0\right)\right]+D_{\mathrm{KL}}\left(q\left(x_{1:T}\mid {\mathbf{x}}_0\right)\Vert p\left(x_{1:T}|x_0\right)\right)\\ & =E_q\left[-\log p_{}\left(x_0\right)\right]+E_q\left[\log \frac{q\left(x_{1:T}|x_0\right)}{p_{}\left(x_{0:T}\right)/p_{}\left(x_0\right)}\right]\\ & =E_q\left[-\log p_{}\left(x_0\right)\right]+E_q\left[-\log \frac{p_{}\left(x_{0:T}\right)}{q\left(x_{1:T}|x_0\right)}+\log p_{}\left(x_0\right)\right]\\ & =E_q\left[-\log \frac{p_{}\left(x_{0:T}\right)}{q\left(x_{1:T}|x_0\right)}\right]\end{array}$$
定义损失函数L：
$$E_q\left[-\log p_{}\left(x_0\right)\right]\le E_q\left[-\log \frac{p_{}\left(x_{0:T}\right)}{q\left(x_{1:T}|x_0\right)}\right]:=$$
Further derivation of$$L\; L:$$
$$\begin{array}{rl}L& ={\mathbb{E}}_q\left[-\log \frac{p_{}\left(x_{0:T}\right)}{q\left({\mathbf{x}}_{1:T}\mid {\mathbf{x}}_0\right)}\right]\\ & =E_q\left[-\log p\left(x_T\right)-\sum _{t\ge 1}\log \frac{p_{}\left(x_{t-1}|x_t\right)}{q\left(x_t|x_{t-1}\right)}\right]\\ & =E_q\left[-\log p\left(x_T\right)-\sum _{t>1}\log \frac{p_{}\left(x_{t-1}|x_t\right)}{q\left(x_t|x_{t-1}\right)}-\log \frac{p_{}\left(x_0|x_1\right)}{q\left(x_1|x_0\right)}\right]\\ & =E_q\left[-\log p\left(x_T\right)-\sum _{t>1}\log \frac{p_{}\left(x_{t-1}|x_t\right)}{q\left(x_{t-1}|x_t,x_0\right)}\cdot \frac{q\left(x_{t-1}|x_0\right)}{q\left(x_t|x_0\right)}-\log \frac{p_{}\left(x_0|x_1\right)}{q\left(x_1|x_0\right)}\right]\\ & =E_q\left[-\log \frac{p\left(x_T\right)}{q\left(x_T|x_0\right)}-\sum _{t>1}\log \frac{p_{}\left(x_{t-1}|x_t\right)}{q\left(x_{t-1}|x_t,x_0\right)}-\log p_{}\left(x_0|x_1\right)\right]\\ & =E_q\left[D_{\mathrm{KL}}\left(q\left(x_T|x_0\right)\Vert p\left(x_T\right)\right)+\sum _{t>1}{D}_{KL}\left(q\left(x_{t-1}|x_t,x_0\right)\Vert p_{}\left(x_{t-1}|x_t\right)\right)-\log p_{}\left(x_0|x_1\right)\right]\end{array}$$
with $L_T$,$L_{t-1}$, $L_0$Definition:
$$L={\mathbb{E}}_q[\underset{L_T}{\underbrace{D_{\mathrm{KL}}\left(q\left({\mathbf{x}}_T\mid {\mathbf{x}}_0\right)\Vert p\left(x_T\right)\right)}}+\sum _{t>1}\underset{L_{t-1}}{\underbrace{D_{\mathrm{KL}}\left(q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,x_0\right)\Vert p_{}\left(x_{t-1}|x_t\right)\right)}}\underset{L_0}{\underbrace{-\log p_{}\left(x_0|x_1\right)}}]$$
first item$L_T$and network parameters $\theta$ is irrelevant and can be ignored.
For the third item$L_0$进行分析：
$$\begin{array}{rl}{p}_{}\left({\mathbf{x}}_0\mid {\mathbf{x}}_1\right)& =\prod _{i=1}^D{\int }_{{\delta }_{-}\left(x_0^i\right)}^{{\delta }_{+}\left(x_0^i\right)}\mathcal{N}\left(x;{\mu }_{\theta }^i\left({\mathbf{x}}_1,1\right),{\sigma }_1^2\right)dx\\ {\delta }_{+}(x)& =\{\begin{array}{ll}\infty & \text{ if }x=1\\ x+\frac{1}{255}& \text{ if }x<1\end{array}{\delta }_{-}(x)=\{\begin{array}{ll}-\infty & \text{ if }x=-1\\ x-\frac{1}{255}& \text{ if }x>-1\end{array}\end{array}$$
相当于从连续空间向离散空间的变化，即将连续高斯分布转化为离散高斯分布，以对应输入的图片数据。
对第二项$L_{t-1}$进行分析：
KL散度的第一项$q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0\right)$，设
$$q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0\right)=\mathcal{N}\left({\mathbf{x}}_{t-1};{\tilde{\mathbf{\mu }}}_t\left({\mathbf{x}}_t,{\mathbf{x}}_0\right),{\tilde{\beta }}_t\mathbf{I}\right)$$
使用贝叶斯公式和配方，计算$q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0\right)$：
$$\begin{array}{rl}q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,x_0\right)& =q\left(x_t|x_{t-1},x_0\right)\frac{q\left(x_{t-1}|x_0\right)}{q\left(x_t|x_0\right)}\\ & \propto \exp \left(-\frac{1}{2}\left(\frac{{\left(x_t-\sqrt{a_t}{x}_{t-1}\right)}^2}{b_t}+\frac{{\left(x_{t-1}-\sqrt{{\bar{a}}_{t-1}}{x}_0\right)}^2}{1-{\bar{a}}_{t-1}}-\frac{{\left(x_t-\sqrt{{\bar{a}}_t}{x}_0\right)}^2}{1-{\bar{a}}_t}\right)\right)\\ & =\exp \left(-\frac{1}{2}\left(\left(\frac{a_t}{b_t}+\frac{1}{1-{\bar{a}}_{t-1}}\right)x_{t-1}^2-\left(\frac{2\sqrt{a_t}}{b_t}{x}_t+\frac{2\sqrt{a_{t-1}}}{1-{\bar{a}}_{t-1}}{x}_0\right)x_{t-1}+C\left(x_t,x_0\right)\right)\right)\end{array}$$
Get the variance ${\tilde{b}}_t$：
$$\begin{array}{rl}{\tilde{b}}_t& =1/\left(\frac{a_t}{b_t}+\frac{1}{1-{\bar{a}}_{t-1}}\right)\\ & =\frac{1-{\bar{a}}_{t-1}}{a_t+b_t-{\bar{a}}_t}\cdot b_t\\ & =\frac{1-{\bar{a}}_{t-1}}{1-{\bar{a}}_t}\cdot b_t\end{array}$$
Get the mean value ${\tilde{m}}_t\left({\mathbf{x}}_t,{\mathbf{x}}_0\right)$：
$$\begin{array}{rl}{\tilde{\mathbf{\mu }}}_t\left({\mathbf{x}}_t,{\mathbf{x}}_0\right)& =\left(\frac{\sqrt{{\alpha }_t}}{{\beta }_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}}}{1-{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0\right)/\left(\frac{{\alpha }_t}{{\beta }_t}+\frac{1}{1-{\bar{\alpha }}_{t-1}}\right)\\ & =\frac{\sqrt{{\alpha }_t}\left(1-{\bar{\alpha }}_{t-1}\right)}{{\alpha }_t+{\beta }_t-{\bar{\alpha }}_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}}{\beta }_t}{{\alpha }_t+{\beta }_t-{\bar{\alpha }}_t}{\mathbf{x}}_0\\ & =\frac{\sqrt{{\alpha }_t}\left(1-{\bar{\alpha }}_{t-1}\right)}{1-{\bar{\alpha }}_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}}{\beta }_t}{1-{\bar{\alpha }}_t}{\mathbf{x}}_0\end{array}$$
使用${\mathbf{x}}_t$和$ϵ$来表达${\mathbf{x}}_0$：
$$\begin{array}{rl}{\tilde{\mathbf{\mu }}}_t& =\frac{\sqrt{{\alpha }_t}\left(1-{\bar{\alpha }}_{t-1}\right)}{1-{\bar{\alpha }}_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}}{\beta }_t}{1-{\bar{\alpha }}_t}\frac{1}{\sqrt{{\bar{\alpha }}_t}}\left({\mathbf{x}}_t-\sqrt{1-{\bar{\alpha }}_t}ϵ\right)\\ & =\frac{1}{\sqrt{{\alpha }_t}}({\mathbf{x}}_t-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}ϵ\end{array}$$
The second term of KL divergence $p_{}\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right)$ , has been defined in the process of inverse diffusion, the author puts the variance$S_{}\left(x_t,t\right)$ Set$p_t^2={\tilde{b}}_t$p θ ( xt − 1 ∣ xt ) : = N ( xt − 1 ; μ θ ( xt , t ) , Σ θ ( xt , t ) ) =
$$\begin{array}{rl}{p}_{}\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right)& :=\mathcal{N}\left(x_{t-1};m_{}\left(x_t,t\right),S_{}\left(x_t,t\right)\right)\\ & =N\left(x_{t-1};m_{}\left(x_t,t\right),p_t^2\right)\end{array}$$
Using the KL divergence calculation formula of two normal distributions (with the same variance), it can be calculated as follows:
$$L_{t-1}={\mathbb{E}}_q\left[\frac{1}{2p_t^2}{\Vert {\tilde{m}}_t\left({\mathbf{x}}_t,{\mathbf{x}}_0\right)-m_{}\left(x_t,t\right)\Vert }^2\right]+C$$
uses$x_0$and $ϵ$ to express$x_t$, $ϵ$ ~$\mathcal{N}\left(0,I\right)$ :
$${\mathbf{x}}_t({\mathbf{x}}_0,)\_=\sqrt{{\bar{a}}_t}{x}_0+\sqrt{1-{\bar{a}}_t}ϵ$$
Then,$L_{t-1}-C$ can be expressed as:
$$\begin{array}{rl}{L}_{t-1}-C& ={\mathbb{E}}_{{\mathbf{x}}_0,}\left[\frac{1}{2p_t^2}{\Vert {\tilde{m}}_t({\mathbf{x}}_t(x_0,)\_,\frac{1}{\sqrt{{\bar{a}}_t}}\left(x_t(x_0,)\_-\sqrt{1-{\bar{a}}_t})\right)\_-m_{}\left(x_t(x_0,)\_,t\right)\Vert }^2\right]\\ & =E_{x_0,}\left[\frac{1}{2p_t^2}{\Vert \frac{1}{\sqrt{a_t}}(x_t(x_0,)\_-\frac{b_t}{\sqrt{1-{\bar{a}}_t}})\_-m_{}\left(x_t(x_0,)\_,t\right)\Vert }^2\right]\end{array}$$
The above formula shows that, given $x_t$In the case of , the network output $m_{}\left(x_t(x_0,)\_,t\right)$ field$\frac{1}{\sqrt{a_t}}\left({\mathbf{x}}_t(x_0,)\_-\frac{b_t}{\sqrt{1-{\bar{a}}_t}}ϵ\right)$ .
But the author did not build the network in this way, but chose to use$m_{}\left(x_t,t\right)$进行参数化处理：
$$m_{}\left({\mathbf{x}}_t,t\right)={\tilde{m}}_t\left(x_t,\frac{1}{\sqrt{{\bar{a}}_t}}\left(x_t-\sqrt{1-{\bar{a}}_t}{ϵ}_{}\left(x_t\right)\right)\right)=\frac{1}{\sqrt{a_t}}\left(x_t-\frac{b_t}{\sqrt{1-{\bar{a}}_t}}{ϵ}_{}\left(x_t,t\right)\right)$$
It can be seen that the network passes input${\mathbf{x}}_t$and $t$ , the actual output is${ϵ}_{}$(i.e. prediction noise), while $x_t$And it can be given by $x_0$To express, the final $L_{t-1}-$Define$C$
$${\mathbb{E}}_{{\mathbf{x}}_0,}\left[\frac{b_t^2}{2p_t^2{a}_t\left(1-{\bar{a}}_t\right)}{\Vert ϵ-{ϵ}_{}\left(\sqrt{{\bar{a}}_t}{x}_0+\sqrt{1-{\bar{a}}_t}ϵ,t\right)\Vert }^2\right]$$
Objectively, we can calculate the quantity and the loss ratio:
$$L_{\text{simple }}\left(i\right):={\mathbb{E}}_{t,{\mathbf{x}}_0,}\left[{\Vert ϵ-{ϵ}_{}\left(\sqrt{{\bar{a}}_t}{\mathbf{x}}_0+\sqrt{1-{\bar{a}}_t}ϵ,t\right)\Vert }^2\right]$$