Soofi S 30B-A3B Model Released

Soofi S 30B-A3B is a sovereign open-source foundation model optimized for German and English. To address the high deployment cost problem of existing transformer models, this model adopts a hybrid architecture combining Mamba-2 and MoE (Mixture-of-Experts).

Key features are as follows:

• High Efficiency: Out of a total of 31.6B parameters, only about 3.2B are activated during inference, delivering 30B-class performance at the cost level of a 3B model.
• Hybrid Design: Of the 52 layers, 23 are Mamba-2 sequence mixing layers, 23 are MoE layers, and 6 are GQA (Grouped-Query Attention) layers, maximizing long-context processing capability.
• Performance Advantage: At a 40K context, it records 8-9x higher decoding throughput compared to Dense 14B-24B models, showing top-tier open model performance on English and German evaluations.
• Genuine Openness: By releasing the full set of outputs including not only the weights but also the training data mixture and recipe, it enhances the model's reproducibility and verifiability.