Alibaba Previews Qwen4 Architecture: Only 6B of 125B Parameters Active

The Alibaba Qwen team released Qwen3.8-Flash-Next, a preview of the architecture to be applied to Qwen4. This model has a total of 125B parameters, but only 6B are actually activated per token. This represents a reduction to approximately one-third of the active compute usage compared to the previous model, Qwen3.7-Plus (397B parameters, 17B active).

The team's primary focus is not performance but inference cost. The goal is to reduce the impact of architectural choices on inference costs in scenarios where agent tasks and very long contexts become standard workloads.

There are four main technical changes. These include a new sparse attention scheme that operates on micro-blocks instead of the previous individual token selection method, a gated residual mechanism that controls information transfer between layers, and a training recipe that completely eliminates batch size warmup.

A notable change is the combination of 51B parameters into separate embeddings instead of adding experts. These embeddings are indexed by 2-3 character fragments, resulting in lower computational costs and easier offloading on memory-constrained accelerators. This design is interpreted as reflecting the reality of Chinese research labs facing export controls that make purchasing top-tier chips difficult.

However, the released performance metrics are based on Alibaba's own benchmarks, and independent verification has not been conducted. The model card explicitly states that the 'Humanity's Last Exam' score was graded by GPT-4o.

Regarding licensing, the applicability of Europe's AI Act is a point of contention. While the weights were released under Hugging Face's qwen-community license, reports emerged that Alibaba intends to charge large-scale commercial users. The AI Act stipulates that open-source exemptions do not apply to components that are paid for or monetized, which could create legal uncertainty for European companies using Qwen-based models.