LFM2.5-24B-A2B-GGUF: QAD Quantized Model Running 24B Parameters on 4GB VRAM

This is a 24B parameter model based on a hybrid architecture developed by Liquid AI. It is optimized for edge devices and on-device deployment, designed to maintain high quality and fast inference speeds even with limited hardware resources.

In particular, it provides checkpoints that apply Quantization-Aware Distillation techniques to recover Q4_0 quantization loss. It is compatible with major local inference engines such as llama.cpp, vLLM, and Ollama, and can integrate with various agent frameworks via an OpenAI-compatible API.

It supports 10 languages and has built-in capabilities for handling tool calls and reasoning processes. It is suitable for developers who need to leverage large language model performance in local environments without cloud dependency, as well as for builders of edge AI applications.