OVH Releases Olala mini 7A1B Open-Source Model with 160 Tokens Per Second on 8GB GPU

·2026.11.17 19:14

Key point

The 7B parameter model uses 256 experts per layer and hybrid attention to achieve 50 tokens per second on standard CPUs.

Details

  • OVH announces Olala mini 7A1B, a 7-billion-parameter model with 1 billion active parameters per layer across 256 experts.
  • The model utilizes hybrid attention combining Differential-TPA and Mamba-3 in BF16 format to reduce memory usage.
  • Performance benchmarks show 160 tokens per second on an RTX A4500 GPU and 50 tokens per second on a standard processor without a GPU.
  • The model requires as little as 8 GB of GPU memory and supports a 32,000-token context window.
  • Olala mini is released as open source on Hugging Face under the Apache 2.0 license.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.