Explorative Modeling: Learning from the Best of K Guesses

When a generative model has multiple correct answers, predicting a single value directly causes it to converge to the average of the possible outcomes. This is why images come out as blurry averages, and why text generation produces only impoverished outputs like "the."

Existing autoregressive models and diffusion models address this problem by splitting generation into multiple steps, simplifying the correct answer at each step. Explorative Modeling adds exploration as a third pretraining axis for generative models on top of this.

Specifically, the model generates K candidates at once, and is trained based on the candidate closest to the correct answer. This allows it to explore diverse possible outcomes instead of a single averaged prediction, and it can be combined with existing generative models or used as an end-to-end generative model.

The key results reported by the paper are as follows.

• Across image, video, and language tasks, increasing the amount of exploration monotonically improved the performance of existing models.
• The performance gains scaled with data size ranged from 7% to 36%, and gains scaled with parameter size ranged from 13% to 23%.
• Explorative Models achieved 6.2x higher sample efficiency, 4.1x higher FLOP efficiency, and 47% improved parameter efficiency.
• Leveraging exploration also enables scaling generalization of the model and end-to-end scaling of existing models.
• As an end-to-end generative model, it matched the performance of diffusion models on control tasks while using up to 256x less inference compute.