AI BriefingBalestriero and LeCun Prove Platonic Representation Hypothesis Using JEPAs
·2026.11.01 22:24
Key point
A new theorem demonstrates that Joint-Embedding Predictive Architectures (JEPAs) achieve cross-modal alignment with 42 times fewer training samples than next-token prediction.
Details
- Proves the Platonic Representation Theorem, establishing that JEPAs trained on different modalities converge to the same representation up to one rotation when using positive pairs of good views.
- Identifies next-token prediction as a JEPA with bad views, explaining why language models require massive scale to align with vision models.
- A 109M-parameter text JEPA aligns better with DINOv2 and CLIP image encoders than language models up to 72B parameters.
- Achieves 64% top-5 accuracy on ImageNet-100 zero-shot, surpassing language models up to 8B parameters which max out at 56%.
Theoretical Framework
- Positive pairs defined as two observations of one scene (good views) are the key lever for building platonic representations.
- Misalignment is bounded when views are not good views, providing a mathematical guarantee for representation convergence.
Empirical Validation
- Scaling law of alignment predicts unseen runs within seed noise across datasets ranging from 145k captions to 940M web images.
- One rotation of separately trained encoders creates an image captioner achieving a CIDEr score of .838 on COCO.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
StreamVibe-VL Improves Proactive Video Interaction by 10.7% Over Baselines
Claude Code skill converts arXiv papers to translated books preserving LaTeX structureResearchers demonstrate data-free self-improvement in LLMs using on-policy self-training