End-to-End Tokenizer Training for Autoregressive Image Generation

This research, accepted as an ICML 2026 Spotlight, presents a method for end-to-end training of the visual tokenizer, a key bottleneck in autoregressive image generation. Instead of the conventional decoupled two-stage training, it jointly optimizes reconstruction and generation, so that signals from generation results are directly reflected in tokenizer training.

The authors refined a 1D semantic tokenizer to better suit autoregressive models, while also exploring how to leverage a vision foundation model to enhance the representational power of the 1D tokenizer. The focus was not merely on a tokenizer that compresses tokens well, but on learning representations that actually improve generation quality.

• Applied end-to-end optimization instead of conventional decoupled training.
• Used a structure that directly feeds generation results back into the tokenizer.
• Achieved FID 1.48 without guidance on ImageNet 256x256 generation.