Simplifying, Stabilizing, and Scaling Continuous-Time Consistency Models

Existing Diffusion models show excellent generative performance, but they require dozens to hundreds of steps during the sampling process, which limits their real-time applications.

To address this, OpenAI proposes a new approach called sCM (continuous-time consistency models). sCM simplifies the theoretical formulation to stabilize the training of continuous-time consistency models and enable scaling to large datasets.

sCM demonstrates the following performance:

• Generates samples of quality comparable to existing diffusion models using only 2 steps of sampling
• Achieves approximately 50x improvement in real-time speed compared to existing methods
• Successfully trains a model with 1.5 billion (1.5B) parameters at ImageNet 512x512 resolution
• Takes only 0.11 seconds to generate a single sample on a single A100 GPU without any special optimization

sCM uses a distillation approach that distills knowledge from a pretrained diffusion model (Teacher model). Notably, sCM's performance scales proportionally as the teacher model's size increases, and even with just 2-step sampling, it maintains high quality with a FID score difference of less than 10% compared to teacher models that require hundreds of steps.