Apple Unveils REFACTOR-VLA: Developing Reusable Robot Motion Skills via Unsupervised Learning

Apple researchers have unveiled the REFACTOR-VLA system to overcome the limitations of existing monolithic VLA models. This system learns reusable skills through a 'wake/sleep' architecture, enhancing the ability to perform long-horizon, multi-step tasks.

Core Technology and Operating Principles

REFACTOR-VLA consists of two main phases. In the Sleep phase, it clusters motor program segments using the Behavior Equivalence Kernel (BEK) based on a learned latent world model (Mφ). In the Wake phase, it generates typed lambda terms using a vocabulary inspired by the Hindley-Milner type system, feeding them into a library-conditioned rectified-flow action decoder to produce actual motions. To be recognized as a skill, it must pass both the Minimum Description Length (MDL) criterion and the reward preservation gate. Training proceeds through a three-stage schedule: world model warm-up, policy optimization, and skill discovery.

LIBERO Benchmark Results and Implications

In evaluations across the entire LIBERO benchmark suite, two key findings emerged. First, increasing the world model size from 188 million to 430 million parameters led to performance degradation in all 4 of the 4 suites. This demonstrates that simply scaling up the world model does not always yield performance improvements.

Second, changes in the learning objective significantly impacted skill clustering quality. Adding an auxiliary supervised contrastive loss (InfoNCE loss) in Phase A greatly improved skill clustering quality in Phase C. Based on n=3 multi-seeding, the Normalized Mutual Information (NMI) was measured at 0.462 for the Object suite, 0.867 for the Spatial suite, 0.915 for the Goal suite, and 0.754 for the LIBERO-10 suite.