BeeLlama.cpp v0.4.0 Released

BeeLlama.cpp v0.4.0, a fork project of llama.cpp, has been updated. This version removes the existing DFlash and TurboQuant features and focuses on core KV Cache-related functionality based on the latest llama.cpp codebase.

The main updates are as follows:

• KVarN (Variance-normalized KV-cache quantization): Resolves the performance degradation and VRAM spike issues of the existing implementation, providing higher precision than existing quantization methods at the same bit width.
• Improved KV cache precision: Optimized precision through KV precision tail technology.
• Support for various quantization types: Supports additional standard KV cache types such as q2_0-q3_1, q6_0, q6_1.

However, in the case of SWA (Sliding Window Attention) architectures (such as Gemma), it is still in a preparatory stage before being ready for production environments due to VRAM usage-related issues.