AI Briefing
KOSign in

ISTA Releases GSQ-RCO Quantized GGUFs for DeepSeek-V4.1-Flash-1017

·2026.10.24 17:05

Key point

ISTA DASLab releases GSQ-RCO quantized GGUFs for the 763.2B parameter DeepSeek-V4.1-Flash-1017, achieving 100% benchmark retention at 2.00 bpw on a 12GB GPU.

Details

  • ISTA DASLab released three GGUF quantizations (IQ1_M, IQ2_XXS, IQ2_XS) for DeepSeek-V4.1-Flash-1017 using GSQ and RCO methods.
  • At 2.00 bpw (IQ2_XXS), the model achieves 100.0% of the base model's task average score, with identical results on AIME25, GPQA-Diamond, and LiveCodeBench v6.
  • All builds run on a single 12 GB NVIDIA GPU using the Strata inference engine, as llama.cpp does not yet support the model.
  • The release includes a shared 110.6 GB Engram file (IQ4_NL) containing the model's 196.6B parameter lookup tables, which is memory-mapped on disk.
  • Quantization uses GSQ for scalar quantization and RCO for exact budget enforcement via gradient descent, with per-tensor allocations provided for 1008 tensors.
quantizationdeepseekggufllm-inferenceistagsqrco
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

RedditBase Compute releases BaseQuant V1 for Qwen3.8-27B with lower KL divergenceRedditQUASAR 4-bit QAT checkpoints for Qwen3.5, Gemma 4 and Muse Glimmer: closer to BF16 than Google’s and NVIDIA’s 4-bit releasesDeepSeekDeepSeek Releases Search and Fetch APIs... $0.75 per 1,000 RequestsPyTorchKRDeepInfer Offline Release: Decode Wide-EP Configuration and Operational Metrics SharedTLDR AIAnt Group and vLLM Community Open-Source Humming for High-Performance Quantized InferenceTLDR AIllm-d 1.0 releases production-ready LLM inference stack with multi-tenant QoS and stable API