Qwen3: Thinking Deeper and Acting Faster

Qwen3 is the latest large language model in the Qwen family, and its core feature is a hybrid structure that offers both thinking mode and non-thinking mode together. It is designed to reason step by step for complex problems while responding quickly to simple questions, allowing the reasoning budget to be adjusted flexibly.

The largest model, Qwen3-235B-A22B, is introduced as achieving performance competitive with top-tier models such as DeepSeek-R1, o1, o3-mini, Grok-3, and Gemini-2.5-Pro on code, math, and general capability benchmarks. It's explained that Qwen3-30B-A3B, also released alongside it, outperforms QwQ-32B, which uses 10x more active parameters, and that Qwen3-4B matches the level of Qwen2.5-72B-Instruct.

The released models consist of two MoE models and six dense models.

• MoE: Qwen3-235B-A22B (235B total, 22B active), Qwen3-30B-A3B (30B total, 3B active)
• Dense: Qwen3-32B, 14B, 8B, 4B, 1.7B, 0.6B
• License: Apache 2.0
• Context length: 32K for small dense models, 128K for most others

On the multilingual front, it supports 119 languages and dialects. This covers a wide range of language regions including English, Chinese, Arabic, Korean, and Japanese, emphasizing scalability for international services and multilingual applications.

Agentic capabilities have also been strengthened. It states that the model has been optimized for coding and agentic tasks, and that MCP support has also been enhanced. SGLang and vLLM are recommended for deployment, while Ollama, LMStudio, MLX, llama.cpp, and KTransformers are suggested for local execution.

The pretraining scale has expanded to nearly double, from 18T tokens for Qwen2.5 to approximately 36T tokens. The data included not only web content but also PDF-like documents, with text extracted using Qwen2.5-VL and then quality-enhanced using Qwen2.5, while math and code data were synthesized using Qwen2.5-Math and Qwen2.5-Coder.

Pretraining proceeded in three stages.

• S1: Trained on 30T+ tokens with 4K context, establishing basic language ability and common sense
• S2: Additional training on 5T tokens with increased emphasis on STEM, coding, and reasoning
• Final stage: 32K context extension using high-quality long-context data

Post-training is organized as a four-stage pipeline. It proceeds in the order of long CoT cold start, reasoning-based RL, thinking mode fusion, and general RL, refining a wide range of capabilities from math, coding, logic, and STEM problems to instruction following, format following, and agent capabilities.

The development example shows how to load the model with Hugging Face transformers and switch modes using enable_thinking=True/False. It further explains that inserting /think and /no_think into the prompt or system message allows flexible switching of thinking mode even during multi-turn conversations.