Zero-Mem: Zero-Token Memory Operations for LLM Agents

Zero-Mem proposes zero-token memory operations that avoid LLM calls and input/output tokens, except for the final question-answering stage. While existing agent memory required additional LLM calls for generating intermediate records and coordinating retrieval, Zero-Mem utilizes the original conversation records directly as evidence.

Memory is structured in two ways.

• Entity-Context Graph: Exposes connections between entities across multiple interactions.
• Temporal Hierarchy: Preserves the locality of conversations and session states.

When a query arrives, the importance of both perspectives is adjusted, information is retrieved from both sides, and relevant relationships or surrounding context are reconstructed by following the structure. Subsequently, a deterministic correction process removes conflicting evidence and links the answer to the retrieved original records. In practice, only the leader (reader) LLM is called to generate the final answer.

In long-term memory and long-context question-answering benchmarks, Zero-Mem demonstrated competitive performance. When using the same final QA leader and context budget, it reduced memory operation time costs by 57.6% compared to the fastest baseline. Ablation studies supported the contributions of the two memory perspectives and the query-specific adjustment method.