What Does GGUF Contain Beyond Weights, and What's Still Missing?

GGUF is a single-file format used by llama.cpp, making it far more convenient to manage and use compared to Hugging Face's safetensors approach, which requires multiple JSON files, or Ollama's approach with its complex layer structure.

Chat Templates The Jinja2 scripts that define the conversational format for chat models are stored under the tokenizer.chat_template key in GGUF metadata. This is essential not just for simple conversations but also for handling tool calling, reasoning blocks, and multimedia message encoding.

Currently, various libraries such as Hugging Face, llama.cpp, and minijinja each use different Jinja2 implementations, which can cause performance differences depending on the implementation.

Special Tokens These include special tokens used to control the model's generation process.

• EOS (End of Sequence): A token that causes the model to stop generating.
• BOS (Beginning of Sequence): A token that signals the start of the input sequence.

These tokens carry meaning beyond simple text and play an important role in helping the model understand context and control its behavior.