DLite: History-free speculative decoding framework for agentic AI

·2026.10.20 22:50

Key point

The framework reuses target model hidden states to avoid maintaining historical KV caches, outperforming DSpark by up to 70%.

DLite is a history-free speculative decoding framework designed for agentic AI workloads with long, evolving contexts.
Existing draft models suffer up to 80% degradation in acceptance length under agentic workloads, while maintaining historical KV cache creates substantial system overhead.
DLite reuses contextual hidden states from the target model and uses a lightweight sequential head to restore intra-block causal dependencies without explicit historical context.
Experiments on general-purpose, long-context, and agentic workloads show DLite achieves 2-6 acceptance length and outperforms the state-of-the-art method DSpark by up to 70%.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.