RSI-Anything: Coupled Policy-World Model Updates via World Models

PhAI Labs and researchers from major universities proposed the RSI-Anything framework, which selects execution candidates using predictions from a World Model in environments with high external feedback costs, and jointly updates the policy and world model. This research addresses the 'coupled-update' problem, where the reliability of world model predictions decreases when the policy changes, by setting the policy-world model pair as the unit of improvement.

Core Mechanism and Validation

• JEPA-Anything-based World Model: Applies Orthogonal Prediction Factorization (OPF) to predict latent states and candidate actions.
• Selective Update: The policy is updated via feedback, while the world model is updated only when predictions are unreliable and adaptation data is sufficient.
• Validation Process: Existing models are replaced only if they pass component checks and pair compatibility checks.

Key Experimental Results

• Code Repair (CodeContests): Hidden test success rates improved from 26.0% (random) to 26.3% (static verifier), 29.2% (fixed world model), and 31.2% (world model adaptation).
• GUI Manipulation (MiniWoB++): On the development set, applying execution rules that follow policy ranking (alpha=0) raised the success rate to 99%. However, compared to the rule-based baseline (100%), this result is primarily diagnostic. In 'world-based control,' where candidates are chosen by world model values, the success rate rose from 49% to 98%, with the world model update effect averaging +53.23%p.
• Lab Planning (LabHorizon): The Final Score for the 4B policy model was 46.01%, exceeding the existing baseline (45.32%).

Limitations and Caveats

• The 99% success rate in GUI tasks is the result of execution rules following policy ranking; since the rule-based baseline achieved 100% success, this task is more about observing the improvement process of learned agents rather than demonstrating direct contributions from the world model.
• Reaction optimization experiments were offline evaluations reusing measured yield tables, demonstrating prediction-based selection efficiency rather than actual experimental outcomes.
• Immunology applications are pending experiments, and reproducibility issues were noted, such as deployed baseline checkpoints failing some checks during code repair reproduction.