German AI Consortium Unveils Soofi S, a Top-Performing Open-Source 30B Model on Benchmarks

Soofi S 30B Model Overview

A German research consortium led by the German AI Association (KI Bundesverband) has released the open-source language model Soofi S. The model uses a Mixture-of-Experts (MoE) architecture that activates only 3.2B of its total 31.6B parameters per token. This means the actual compute cost is on the order of a 3B model, while performance is far superior.

Architecture and Performance

The model adopts a hybrid design combining Mamba-2 with standard attention layers. Only 6 of the 52 layers maintain a KV cache, preserving processing speed even at long context lengths. In actual benchmarks, it achieved more than 8x the throughput of comparable 14-24B models at a 40,000-token context. It maintains near-constant throughput up to 256,000 tokens, while standard transformers see performance drop sharply.

German-Focused Training Strategy

The model was trained on a total of 27 trillion tokens across 3 stages. In stage 1, 20 trillion tokens of web, code, and math data were used, starting with a German-language share of 7.2%. In stage 2 (6 trillion tokens), the German share was expanded to 15.3%, focusing on high-quality sources. A combination of German newspapers (the Genios corpus with 193 million articles), HPLT, and the Commons corpus was used. This strategy improved German-language performance by 15.1 points over the baseline model (Nemotron).

Benchmark Results

Soofi S achieved the best performance in every category among fully open-source models:

• English overall: 70.1 (surpassing OLMo 3 32B and Apertus 70B)
• German overall: 79.1 (ahead of all comparison models)
• Code (HumanEval): 73.8%, MBPP: 70.2%, MBPP-DE: 84.2%
• German regional knowledge (INCLUDE-DE): 61.2 points (tied with Qwen3.5)

There are also weaknesses. On German competition math (Minerva MATH-DE), it scored 56 points, trailing Qwen3.5 (76.5 points). The 3.2B active-parameter constraint also limits performance on fact-based retrieval tasks.

Overtraining Controversy and Rebuttal

Following the release, critics pointed out that Soofi S was "severely overtrained" by Chinchilla scaling law standards. Fitting 27 trillion tokens to 30B parameters results in a token-to-parameter ratio reaching hundreds to one.

Michael Fromm, the project's technical lead, pushed back: "There is new research showing that existing scaling laws don't apply to MoE architectures. In MoE models, if individual experts see the same document multiple times, repeated data isn't as harmful as it is in dense models." Nvidia also has precedent, having trained its own model on up to 25 trillion tokens.

Sovereign Infrastructure and Transparency

Training was carried out from March to May at Deutsche Telekom's Industrial AI Cloud in Munich using 512 Nvidia B200 GPUs (a total of 253,000 GPU hours). The facility runs on 100% renewable energy, is cooled via the Elsbach waterway, and releases waste heat to the surrounding Tucherpark.

Major German institutions participated, including Fraunhofer IAIS/IIS, DFKI, TU Darmstadt, the University of Würzburg, the L3S Research Center, Berlin University of Applied Sciences, Ellamind, and Merantix Momentum. Funding was provided by the German Federal Ministry for Economic Affairs and the European IPCEI-CIS program.

Fully Open Release

Soofi S meets the Open Source AI Definition 1.0. The team released the model weights, intermediate checkpoints, the full training and evaluation code, and a data inventory that documents the token count and contribution of each data source in detail. Excluded sources were also recorded. 99% of the training data is independently reproducible. However, due to the commercially licensed Genios corpus (1.3%), it does not meet the stricter European Open Data Definition.