AI Briefing#nvfp4
The latest AI and developer news about #nvfp4, with the original source and a short summary.
The latest AI and developer news about #nvfp4, with the original source and a short summary.
Unsloth updated Qwen3.8-27B GGUFs with Dynamic V3.1 for 5-10% accuracy gains and introduced new 1-bit quantization types for the 2.4T parameter model.
By optimizing the memory hierarchy across GPUs, CPUs, and storage, NVIDIA achieves a 30x increase in agentic throughput per dollar with only a 3% rise in HBM capacity.
The technique uses lossless GPU compression and Blackwell Decompression Engines to sync weights faster than intra-cluster P2P RDMA.
New NVFP4 and GGUF checkpoints use quantization-aware distillation to achieve lower KL divergence than reference 4-bit models from NVIDIA, Google, and community quantizers.
This article introduces vLLM optimization strategies to maximize Gemma 4's SLO-compliant throughput through NVFP4 precision and the separation of interactive and batch pools.
The NInfer fork supports a 555k token context window on the RTX 5090 via NVFP4 KV cache and YARN extension.
NVIDIA has released the NVFP4 quantized version of DeepSeek-V4-Pro-0813, supporting a MoE architecture that activates 400B parameters out of a total of 1.65T.
A checkpoint quantizing Qwen3.8-27B to NVFP4 using the QUASAR QAT algorithm has been released, maintaining BF16-level performance.
CKA-QAD improves inference and coding performance by preserving internal representations during NVFP4 quantization-aware distillation.
poolside has released a new Laguna-S 2.1 NVFP4 checkpoint that replaces the previous version.
Poolside updated the FP8 and NVFP4 checkpoints for Laguna S 2.1.
Unsloth has released NVFP4 quantized models for Qwen3.6 that boost inference speed by up to 2.5x on Blackwell GPUs.
How to maximize performance by optimizing GLM-5.2 with NVFP4 and TP=4 settings in an NVIDIA B200 environment
NVIDIA has released a quantized version of Alibaba's Qwen3.6-35B model using the NVFP4 data type.
llama.cpp now supports NVIDIA FP4 quantization and Multi-Token Prediction (MTP) features.
NVIDIA released LongLive 2.0, an NVFP4 and parallelization-based system.
NVIDIA released NVFP4 versions of Kimi-K2.6 and Kimi-K2.5.
On a Mac Mini M4 Pro 64GB, the 14B dense model hit 16.2 t/s, while Qwen3.6 35B NVFP4 reached 69.8 t/s.
NVIDIA released an NVFP4 quantized checkpoint of the Gemma 4 26B model.
Native NVFP4 in llama.cpp b8967 boosted prefill speed by up to 68%, but generation speed stayed nearly the same.
llama.cpp b8967 adds native NVFP4 support for Blackwell.
llama.cpp has merged initial native NVFP4 MMQ support for SM120, and related GGUF conversions have also been released.
Different FP4 inference support has been added to llama.cpp and ik_llama.cpp.