Linum AI Proposes Pyramid-JiT: A VAE-Free Pixel-Space Text-to-Image Model

Linum AI has published Pyramid-JiT (P-JiT) as a research artifact, a new text-to-image architecture that operates in pixel space without a Variational Autoencoder (VAE). The model addresses scalability limitations in previous encoder-decoder designs by adopting a decoder-only structure similar to Large Language Models, utilizing a DiT trunk that predicts target images at multiple resolutions.

Performance and Efficiency

P-JiT demonstrates significant improvements in training efficiency and image quality metrics compared to its predecessors:

• FD-DINOv2 Score: P-JiT achieved 27.9, significantly outperforming JiT-DDT (39.1) and approaching the real-vs-real floor of 25.4. Linum v2 (latent diffusion) scored 79.9.
• Training Efficiency: P-JiT reached Linum v2's final FD-DINOv2 score using 11.3x fewer samples and 4.3x fewer GPU-hours (at 4x the pixel count).
• Token Reduction: Despite using a more aggressive token reduction (256 tokens for 512x512 images) compared to Linum v2, P-JiT generates higher-quality images with more natural details in eyes, wrinkles, and fur.

Architecture and Methodology

The core innovation lies in replacing the traditional encoder with K-Readouts, which place output heads at various depths of the DiT trunk to predict images at different resolutions (e.g., 128x128, 256x256, 512x512). This multi-scale supervision in pixel space eliminates the need for a VAE.

• Gated Residuals: The team experimented with Gated Residuals inspired by recent LLM research but found that the baseline architecture using AdaLN + tanh + post-norm provided stronger bounds for detail learning. Consequently, P-JiT retains the baseline normalization strategy.
• Timestep Conditioning: To handle flow matching, the model uses Scale-based timestep conditioning rather than Shift-based, which proved more effective for controlling structure and detail.
• Text Encoder: Uses frozen Qwen3.5-4B hidden states projected into the DiT space.

Prompt Following Benchmarks

While P-JiT excels in distribution matching (FD-DINOv2), it currently lags behind Linum v2 in prompt-following tasks, particularly in counting and color accuracy, indicating it is still undertrained in these specific areas.

• GenEval: Linum v2 (0.855) > P-JiT (0.817) > JiT-DDT (0.808).
• Counting Task: Linum v2 scores 0.80, while P-JiT scores 0.56.
• Color Task: Linum v2 scores 0.95, while P-JiT scores 0.90.

The code and weights are released under the Apache 2.0 license as research artifacts. Linum AI plans to continue training P-JiT and will release updated checkpoints en route to Linum v3.