DDPM and DDIM

Explanation at station b

The advantages and disadvantages of diffusion and gan in the factory just hang out
Diffusion Probabilistic Models (DDPM) and Diffusion Implicit Models (DDIM).

• DDIM is an acceleration work for DDPM. In fact, DDPM can also be generated by T step during training and sampling N steps during prediction, but the quality of step generation is higher than that of DDIM.

• DDIM observes the loss function of DDPM, $x_{t}0$ can be pushed directly to$x_t$，
  

• The loss function only sums $x_t$It is related: In fact, not only the tth step is optimized, but all the previous ones get $x_t$The process, including the added noise, is implicitly optimized; optimize the model of a non-Markovian process (non-chain conduction)

• After getting the pre-trained DDPM, you can choose a part of the time nodes to perform the reverse process, and the middle (t-1) steps are actually optimized; therefore, you can choose the time steps in the subset of <t during training. Optimization;
  • The DDIM model itself defines a forward process. In the reasoning process, it is not the noise predict of DDPM (predicting the noise at time t), but the data predict (directly predicting $x_0$), and then $x_t$together get $x_{t-1}$

Differences in the sampling process:

• DDIM is not a Markov chain, all time step variance = 0, the randomness of Gaussian noise is removed, and it becomes a deterministic result,
• DDPM is a Markov chain, and the variance is random each time.
  DDIM can be regarded as an ordinary differential equation in the continuous time dimension. By analogy, Dr. Song Yang proposed a noise addition and denoising framework based on stochastic differential equations. The reverse stochastic differential equation has a corresponding ordinary differential equation, and they share the same marginal distribution, so the problem can be solved by solving the ODE equation, and DDIM corresponds to this process.