Single-File C Engine for Gemma Multimodal Inference Released

A single-file pure C inference engine named 'gemma.c' has been released for Google's Gemma model family (1, 2, 3). Without dependencies on external libraries such as llama.cpp or ggml, it manually implements all core components including the transformer architecture, hybrid sliding window attention, RoPE, int8 quantization, BPE tokenizer, and SigLIP vision encoder.

Key Technical Features

• Multimodal Support: Includes a SigLIP vision tower to process not only text but also image inputs, and provides a Pan & Scan feature for high-resolution images.
• Performance Optimization: Directly written GEMM/GEMV kernels support fp32, fp16, bf16, and W8A8 (8-bit weights, 8-bit activations) quantization. It uses hybrid attention with FlashAttention-style causal tiling.
• Easy Build and Deployment: Single-command builds are possible with gcc, clang, msvc, etc., and it supports weight loading via mmap and KV cache streaming generation.

Usage and Limitations

It provides a Python script to convert Hugging Face checkpoints, and the --text-only option allows skipping the vision tower to save memory. Currently, weight data types are selected at compile time, but improvements for runtime dispatch are planned. It is useful for developers who want to understand the internal structure of modern transformers without complex dependencies, or users looking to build lightweight inference environments.