Open Source JEPA World Model DVD-JEPA Released

DVD-JEPA is an open source project implementing the JEPA (Joint-Embedding Predictive Architecture) architecture, which predicts latent representations instead of pixel-level predictions. It was designed to address the problem of getting bogged down in unnecessary detail that occurs with pixel prediction.

Key features and performance are as follows:

• Reproducibility: It targets a DVD logo moving within a 16x16 box, and is trained using a context encoder, an EMA target encoder, and a Latent Predictor.
• Representation learning: Without any separate labels, it can recover the logo's exact coordinates (y, x) in a 32-dimensional latent space within a 0.73px margin of error.
• Visual prediction and anomaly detection: When combined with a decoder, it can successfully render future frames including wall reflections, and the prediction error can be used as a monitoring tool to detect anomalies.
• Lightweight: The trained MLP model has been reimplemented in about 40 lines of JavaScript, enabling client-side execution even in a browser environment.

This project is a demonstration that most simply and clearly illustrates the core architecture of I-JEPA, V-JEPA, and V-JEPA 2.