Microsoft Research Researchers Discuss 'Surprising Failures' in Multi-Turn AI Conversations

In a conversation, Jennifer Neville, Partner Research Manager at Microsoft, and Principal Applied Scientist Chad Atalla explore the limitations of current AI systems when tested beyond traditional benchmarks. Neville's team, the AI Interaction and Learning team, focuses on pushing the boundaries of AI behavior in realistic work environments. A key finding is that while models perform well on single-turn benchmarks, their performance degrades significantly in multi-turn scenarios where users clarify requirements over time. This 'surprising failure' occurs because models lose track of context and intent as conversations progress.

To address this, the team is developing reinforcement learning methods to improve multi-turn behavior. They also recommend practical user strategies, such as resetting chats and providing fully specified single-turn prompts when confusion arises. The research relies on analyzing consumer logs from Microsoft products in a privacy-preserving manner, extracting high-level patterns of success and failure without accessing raw user data directly.

Beyond chat, the discussion covers long-horizon workflows, such as iterative document editing, where agents struggle to maintain state and track user intent. Neville notes that tasks easy for humans are not necessarily easy for AI, particularly for Transformer architectures. Current solutions involve using agentic harnesses and backend inference to mitigate these limitations. Neville advises users to treat AI as a productivity tool requiring human oversight and verification, rather than expecting 100% accuracy, and to provide detailed feedback to help improve future models.