AI Briefing[NeurIPS 2021] Part 1: Generative Models - Diffusion Model Review
Key point
This piece analyzes the principles of Diffusion Models, a core component of generative models, and the differences among various generative models.
Details
A Generative Model is a model that estimates the distribution of data to generate new data, and it is broadly distinguished from Discriminative Models. The performance of a generative model is judged by how well it predicts the Likelihood of the data and by the quality of the generated data.
The characteristics of major generative models are as follows.
- Auto-regressive models (ARMs): Compute Likelihood as a product of conditional probabilities, but sampling is slow.
- Variational Autoencoders (VAEs): Enable fast sampling by leveraging latent variables, but Likelihood is difficult to handle.
- GANs: Generate high-quality samples through competition between a generator and a discriminator, but training is unstable.
- Diffusion Models: Estimate the distribution through a Diffusion Process that adds noise to data and a Reverse Diffusion Process that reverses this.
Recently, Diffusion Models have been actively studied, showing superior Likelihood performance compared to AR models and better sample quality than GANs. This review covers in detail the mathematical structure and principles of Diffusion models, which draw their ideas from thermodynamics.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.