Beyond Visual CoT: Internalized Visual Thinking for Active Video Reasoning

Multimodal Large Language Models (LLMs) utilize Chain of Visual Thought (Visual CoT) to understand spatial, temporal, and physical environments. However, the approach of generating intermediate reasoning images incurs significant computational overhead during inference, which is particularly critical for active video reasoning.

Apple researchers introduced the Internalized Visual Thinking (IVT) framework, which performs visual thinking during the training phase and generates answers directly during the inference phase. IVT leverages unlabeled video data to simultaneously optimize text prediction and next-embedding prediction. This trains the model to predict both the latent representations of future frames and the final text answer for partially observed videos.

During inference, IVT generates answers directly without synthesizing or re-encoding future frames. This maintains performance equal to or better than Visual CoT while reducing end-to-end latency by more than 5x. The results suggest that explicit pixel-space generation is not essential for active video reasoning, and internalizing predictive world modeling during training can yield multimodal reasoners with both accuracy and efficiency.