AI BriefingLLM source-boundary failure
Key point
Explicitly stating source boundaries was needed for LLMs to distinguish text usable as grounds for an answer.
Details
Context Is Not Control is a working paper addressing the problem of distinguishing whether text within the context window is admissible evidence for a task, even if it's present. The replication artifacts were also released alongside it.
In long-context, RAG, and tool-using workflows, user instructions, retrieved docs, stale docs, suspended docs, quoted claims, answer choices, and injected/adversarial text all get mixed together. The key issue here is separating the fact that 'text exists' from the fact that it is 'admissible as grounds for an answer.'
The key findings are as follows.
- Simply adding an
INSUFFICIENTanswer option was not enough. - Explicitly specifying source admissibility / source boundaries in the task frame produced stronger improvements.
Claude was tested as one of several frontier/API models and open-weight models, and the results were not limited to Anthropic models alone. This research highlights a narrower failure mode than hallucination overall, namely that a document being visible is different from that document being permissible to use as grounds.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.