Google Unveils Gemma 3n for On-Device Use

Google's on-device-specialized multimodal model Gemma 3n has been officially released through major open-source libraries such as transformers, llama.cpp, ollama, and MLX. This model is a native multimodal model that supports not only text but also image, audio, and video inputs.

The newly released model comes in two sizes: E2B (actual 5B) and E4B (actual 8B). As the name 'Effective(E)' suggests, it is designed to run with far less VRAM (2GB and 3GB, respectively) than the actual parameter count would require, thanks to memory efficiency techniques.

Key technical features:

• MatFormer architecture: A nested transformer design that allows layers to be selectively extracted for use according to hardware specifications and memory budget.
• Per-Layer Embeddings (PLE): Offloads embeddings to the CPU, dramatically reducing accelerator memory usage.
• Multimodal encoders: Equipped with a 300M-parameter MobileNet-V5 vision encoder and a USM-based audio encoder.
• KV Cache sharing: Accelerates long-context processing for audio and video, achieving 2x faster prefill speed compared to the previous generation.

Performance and support:

• The E4B model is the first model under 10B to break the 1300-point LMArena score.
• It supports 140 languages for text and 35 languages for multimodal interaction.