AI BriefingNVIDIA Introduces AutoCompress: Hardware-Guided LLM Compression Framework
·2026.11.10 09:00
Key point
The framework compresses Nemotron 3.5 Super from 120B to 74B parameters while retaining 95% of its mean benchmark score.
Details
- AutoCompress is a hardware-guided compression framework that incorporates latency, throughput, and memory constraints directly into differentiable architecture search.
- It outperforms the Minitron pruning framework by 6.57, 2.35, and 0.57 percentage points on average accuracy across eight benchmarks on RTX 5070, RTX 5080, and H100 GPUs, respectively.
- The system compresses Nemotron 3.5 Super from 120B to 74B parameters for single-B200 BF16 serving at 128K context, retaining 95% of the original model's mean benchmark score.
- Unlike existing workflows that target parameter counts, AutoCompress jointly optimizes global and per-layer structure to produce heterogeneous architectures guided by serving-engine cost models.
- Evaluation covers long-context local coding, batched datacenter serving, and latency-sensitive in-game interaction workloads.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.