Learning the Integral of Diffusion Models

Diffusion sampling is the process of integrating, over multiple steps, the path from noise to data. This post reorganizes the relationship with diffusion, focusing on flow maps, which directly predict that integral.

First, sampling is divided into stochastic and deterministic. In the deterministic approach, the same starting point always leads to the same endpoint, creating a bijection between noise and data, and the denoiser plays a memoryless, myopic role, producing the next direction based only on the current position and time. Because of this, the path must be unique, and different paths must never cross.

Next, flow maps are explained as a way to directly predict one point on the path from another point. Being more flexible than simple tangent prediction, this can lead to faster sampling, reward-based learning, and steerability.

In the latter half, based on the taxonomy of Boffi et al., the post organizes the following in order:

• how to define consistency
• whether to use backprop or not
• how to train from-scratch
• what the actual applications and alternative strategies are