Apriel-H1: 2.1x Faster Inference Efficiency

ServiceNow AI has released the Apriel-H1 model family, which maximizes efficiency without retraining an existing 15B reasoning model from scratch. This model adopts a Mamba hybrid architecture, improving throughput by up to 2.1x while minimizing degradation in reasoning performance.

The key technical insight lies in the type of data used during distillation. Instead of using general pretraining data, one must use high-quality reasoning traces extracted from the teacher model's SFT (Supervised Fine-Tuning) dataset. This is to effectively transfer the complex multi-step reasoning patterns and long-range dependencies of the attention mechanism into Mamba's linear recurrent structure.

The main methodology and features are as follows:

• Data Matching: Distillation data must be selected based on the capability one wants to preserve (reasoning ability), not the capability one is trying to build.
• Reverse KL Divergence: Reverse KL divergence was used instead of Forward KL to increase training efficiency, allowing the student model to follow the teacher model's high confidence.
• Performance: MATH500 and MTBench performance actually improved slightly, and while there were minor declines on some benchmarks such as GSM8k and GPQA, overall reasoning quality remained stable.