Mismatch between GGUF filenames and actual bit counts

It has been confirmed that llama.cpp's quantization tools generate files where the filename and the actual saved bits per weight (bpw) do not match under certain conditions. An audit of 443 GGUF files across 25 repositories revealed that 64 files had quantization types listed in their filenames that differed from the actual data.

Cause and Mechanism

k-quant and i-quant require the first tensor dimension to be divisible by 256. If this condition is not met, llama-quantize substitutes the requested low-bit type with a compatible 32-block type (e.g., IQ4_NL or Q4_0). As a result, the low-bit model requested by the user is actually saved at approximately 4.5 bpw, but the filename, model card, and metadata still display the originally requested type (e.g., IQ2_XXS).

Impact and Cases

This issue has existed since PR #3747 in 2023. While warnings are output in the quantization logs, users downloading the completed GGUF files cannot see them. A representative case is the Nemotron-3.5-Lightning model, where n_embd is 2688 and expert widths are 1856 and 3712, which are not multiples of 256. Consequently, all IQ2 stages were actually saved as identical 4.58 bpw files. This is due to the tool's behavior itself, not an error by the uploader.