AI BriefingMeta Releases InteractiveWorld Simulator and InteractiveBench for Wearable AI Assistants
·2026.10.28 09:00
Key point
The open-source repository includes a closed-loop simulator with 44 environments and a 120-task benchmark that evaluates models based on final world state rather than text.
Details
- Meta (facebookresearch) published the InteractiveWorld repository on GitHub, providing a verifiable human-centric environment for building and testing interactive multimodal large language model (MLLM) assistants.
- The system comprises two layers: the InteractiveWorld simulator, which creates a closed-loop 3D environment where an assistant sees a first-person view and a human agent reacts to commands, and InteractiveBench, a benchmark of 120 fixed multi-turn tasks across six categories.
- Evaluation is grounded in world state verification rather than text judgment; the benchmark uses Gemini 3.1 Pro as the fixed human model and requires the final world state to satisfy all goals to pass.
- The simulator operates on Linux with NVIDIA GPUs (tested on A100 80 GB), utilizing Isaac Sim 5.1 and OmniGibson 3.9.0, and supports 36 human action families controlled via short verb-object commands.
- Users can run evaluations with local open-source models (via vLLM or Ollama) or hosted APIs (OpenAI, Anthropic, Gemini), with tasks sharded across GPUs in a round-robin fashion.
- Custom tasks are defined in TaskPack JSON files, which must be kept outside the repository to maintain release tree integrity, and the project is licensed under CC BY-NC 4.0.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
OpenAI releases GPT-6.1 Terra with 50% lower API costs and improved computer use
gpu-unlock Linux module doubles FP/BF16 instruction rate on Nvidia 3000 series GPUs