From Where Things Are to What They're For: A Spatial-Functional Intelligence Benchmark for Multimodal LLMs

SFI-Bench is a video benchmark built from diverse first-person indoor video scans, comprising over 1,700 questions that jointly test the spatial and functional intelligence of multimodal LLMs. It targets the ability to understand real environments by connecting perception, memory, and reasoning.

There are two evaluation axes.

• Structured Spatial Reasoning: the ability to understand complex indoor layouts and weave together multiple scenes into a coherent spatial representation
• Functional Reasoning: the ability to infer the purpose of objects and their context-specific utility

Questions are composed of tasks such as conditional counting, multi-hop relational reasoning, functional pairing, and knowledge-grounded troubleshooting. In experiments, current MLLMs consistently struggled to combine spatial memory with functional knowledge and external knowledge together, and this benchmark serves as a reference point for moving toward multimodal agents that are more grounded in real environments.