PickOrnith-1.5-9B-GGUF: 9B Multimodal LLM, Run Locally in GGUF Format

Ornith-1.5-9B is a 9B-scale LLM that supports both text generation and image understanding. Provided in GGUF format, it can be run immediately in various local inference environments such as llama.cpp, Ollama, and vLLM. Released under the MIT license, it has few restrictions on commercial use.

The model provides GGUF files at various quantization levels, including BF16, Q4_K_M, Q5_K_M, Q6_K, and Q8_0. It also includes mmproj files for image processing, allowing immediate use for multimodal inference without additional configuration. In particular, the Q4_K_M version has low memory usage, making it easy to run even on low-spec GPUs or CPU environments.

The chat template has built-in tool calling and reasoning capabilities, making it suitable for developing agent-based applications. It is designed to specify function call formats in the system prompt and include the thought process in assistant responses. Unlike previous methods, it handles tool calls using a standardized XML tag format without complex parsing processes.

It is useful for developers building local LLMs or in environments that prioritize privacy. It serves as an option when seeking reasonable performance while reducing dependence on cloud APIs and ensuring data sovereignty. It is worth considering as an alternative in scenarios requiring small models with multimodal capabilities.