AI BriefingPACE: Post-hoc Four-Bit State Quantizer for Linear-Attention LLMs
·2026.10.23 14:35
Key point
PACE uses periodic rounding and channel-aligned scaling to maintain accuracy at four bits, matching 8-bit performance on models up to 35B parameters.
Details
- Introduces PACE, a post-hoc four-bit state quantizer for linear-attention language models that reduces memory usage while preserving accuracy.
- Rounds recurrent states only once every few generated tokens instead of after every token, storing intermediate tokens in a small buffer to reduce accumulated error.
- Shares quantization scales within columns to handle large entries in gated-delta states and keeps heads with the longest rounding error retention at eight bits.
- On four hybrid models ranging from 4B to 35B parameters, PACE with mixed four- and eight-bit heads is competitive with eight-bit DAMP at about two-thirds of its memory.
- PACE with four-bit heads throughout scores higher than Q-Mamba at equal memory, and the periodic rounding approach preserves the decoding speedup of the buffer.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Hybrid Reasoning Agents achieve 35-55% token reduction and up to 2.6x faster inferenceResearchers Introduce Context Language Models for Native Context Management