Further Clarification on AI Delegation and Long-Horizon Task Reliability

The recent paper LLMs Corrupt Your Documents When You Delegate presents the DELEGATE-52 benchmark, which examines how much meaning is preserved when AI modifies documents, spreadsheets, code, and structured files across multiple steps. Microsoft Research re-emphasized that this work is not meant to judge overall model capability, task success, or user outcomes, but is a stress test diagnosing information preservation in long-horizon delegation workflows.

The evaluation used chained tasks that repeat transformations and inverse transformations, along with domain-specific semantic parsing. Tallying only semantic changes—not formatting changes—frontier models accumulated rare but significant errors, with artifact fidelity dropping by roughly 19-34% after 20 iterations.

• Python workflows were relatively robust, with an average degradation of less than 1%.
• Even in a setup where a simplified agentic harness was equipped with Python execution and file operation tools, the degradation did not completely disappear.
• However, these figures come from experiments with limited human verification and do not represent production systems as a whole, which include verification loops, orchestration, and domain-specific tools.

The message is clear: short benchmark performance alone cannot guarantee the reliability of long-horizon delegated execution, but this does not mean AI systems are useless in practice. Microsoft Research views future verification loops, memory systems, retrieval, and orchestration as key to reducing this failure mode.