Researchers demonstrate data-free self-improvement in LLMs using on-policy self-training

Details

A new framework reformulates on-policy self-distillation as a self-training process driven entirely by self-generated question-answer pairs and privileged reasoning demonstrations.
The model synthesizes its own curriculum of queries and reasoning traces using only coarse, high-level topic descriptions, eliminating the need for ground-truth pairs, human-curated problem sets, or stronger external teachers.
Self-distillation on fully synthetic privileged information allows the model to improve across multiple domains while retaining its capabilities on domains it is not self-trained on.
The approach demonstrates generality across text and multimodal domains, achieving consistent self-improvement on standard language benchmarks and Visual Question Answering.
The findings suggest that foundation models possess rich latent capabilities that can be amplified without external task supervision or curated problem distributions.

Key point