Ternary Nemotron 30B-A3B GGUF Enables 368K Context on 12 GB GPUs

·2026.10.26 16:49

Key point

The 1.58-bit MoE model achieves 172 tok/s decode and 7000+ tok/s prefill speeds in both Stock llama.cpp and the PrismML fork.

30B-A3B MoE model quantized to ternary (1.58-bit) format
Supports 368K context window on 12 GB GPU hardware
Delivers 172 tok/s decode and 7000+ tok/s prefill performance
Compatible with Stock llama.cpp (b11555+, b11796) and PrismML fork / Bonsai (b10793)

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.