Qwen3.8-Flash-Next: A New Architecture for Ultimate Cost Efficiency

The Qwen team has released Qwen3.8-Flash-Next, a multimodal MoE model that serves as an early preview of the Qwen4 architecture. As an evolution of the hybrid Gated DeltaNet + Gated Attention design applied to the existing Qwen3.5–3.8 series, it was released early to allow the community to review the architecture before the official launch of Qwen4.

This model has been systematically upgraded in four aspects: attention, residuals, embeddings, and optimization.

• Attention: Applied a hybrid architecture of GDN and Qwen Sparse Attention (QSA) to significantly reduce the cost of processing long sequences.
• Residuals: Expanded the residual stream into 4 branches via Gated Residual (GR) and controlled information flow with dynamic gates to enhance training stability.
• Embeddings: Utilizes N-gram Embedding to expand model capacity without additional computational burden, supporting host memory offloading and asynchronous prefetching.
• Optimization: Refined the Muon optimizer and refitted scaling laws tailored to the new architecture.

It features a main model with 125B parameters and 51B N-gram embeddings, with only 6B parameters activated per token. While training costs are reduced to approximately 1/9 compared to Qwen3.7-Plus, performance in coding and office tasks is superior. It supports a base 262,144 token context, expandable up to 1,000,000 tokens via YaRN.

In benchmark results, it scored 62.5 on SWE-bench Pro and 91.7 on GPQA Diamond, outperforming competing models. It is expected to be offered on QwenCloud at a price of 0.16 USD per input token and 0.47 USD per output token.