AI Briefing109M JEPA Model Demonstrates Superior Alignment with Vision Models Compared to 72B LLM
·2026.11.04 15:30
Key point
Theoretical proofs and experiments confirm that a 109M JEPA model aligns better with vision models than a 72B LLM.
Details
- Proved the theorem that JEPA, trained on positive pairs of two observations of the same scene, learns Platonic representations
- The 109M text JEPA recorded 0.184 on the WIT-1024 benchmark, surpassing Qwen2.5-72B (0.154) and Llama-3-70B (0.151)
- Reached DINOv2 alignment levels with 21.6M samples, confirming the use of 42 times fewer samples compared to next-token prediction
- A captioner joined by a single rotation achieved CIDEr 0.668 on COCO transfer, confirming performance higher than CLIP zero-shot
Core Mechanism
- Good view JEPA minimizes distractors with a scene description ratio of 0.804, while bad views increase alignment error at 0.655
- When positive pair distributions are shared, the optimal encoders for all modalities are represented as rotations of the world embedding
- Next-token prediction has a structure disadvantageous to vision alignment, acting as a log-partition function collapse prevention term
Experiments and Limitations
- Trained on text from 940 million web images, with results verified by averaging over 3 seeds
- The LLM comparison measures alignment, not generation or reasoning capabilities
- Proven under the theoretical conditions of finite world and injective rendering assumptions, with actual experiments using LeJEPA
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.