'Cache-to-Cache' Framework Revealed for Direct KV-cache Communication Between LLMs Without Text

Existing multi-LLM systems require converting internal representations into text tokens for inter-model communication, leading to semantic information loss and generation latency. To address this, the Cache-to-Cache (C2C) paradigm was proposed to enable direct semantic communication between LLMs without text generation.

C2C uses neural networks to project and fuse the KV-cache of the source model into the KV-cache of the target model to transmit semantic information. A learnable gating mechanism selectively utilizes target layers where cache communication is beneficial, and preliminary experiments demonstrated that KV-caches are effective as a communication medium between models.

In terms of performance, average accuracy improved by 6.4~14.2% compared to individual models, showing a performance advantage of approximately 3.1~5.4% over existing text-based communication methods. Additionally, it achieved an average 2.5x speedup in latency. This research was accepted to ICLR'26, and the code is available on GitHub.