AI BriefingApple Introduces Selection-Based Structured Reasoning for Efficient Multimodal Search Agents
Key point
SSR reduces per-turn reasoning latency by over 90% and total inference latency by 28–54% on 2B and 4B models.
Details
Multimodal agents often generate lengthy, freeform reasoning that offers little guidance for small models while increasing inference costs. To address this, Apple researchers introduced Selection-based Structured Reasoning (SSR), a framework that reformulates reasoning as a selection task rather than open-ended generation.
How SSR Works
SSR represents recurring high-level reasoning as pre-specified, reusable natural-language candidates. At each turn, the model selects the best candidate based on likelihoods given the current context, eliminating the need for an auxiliary task head. This approach enables parallel scoring, where teacher-forced prefilling computes token likelihoods concurrently within and across candidates using a shared context KV cache.
Performance Results
Evaluated on seven multimodal search benchmarks using 2B and 4B models, SSR demonstrated significant efficiency gains without sacrificing task performance. The framework achieved an average success rate competitive with leading search agents of the same scale while delivering substantial speed improvements:
- Per-turn reasoning latency: Reduced by over 90%
- Total per-question model inference latency: Reduced by 28–54%
These results held across multiple reinforcement learning objectives and supervised fine-tuning methods.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.