Qwen2.5-Math: A World-Class Open Source Math-Specialized LLM

Qwen has released the Qwen2.5-Math series, an upgrade of the existing Qwen2-Math. This series consists of Base models (1.5B/7B/72B), Instruct models, and a math-specific Reward Model (72B).

Unlike the previous model, which only used CoT (Chain-of-Thought) for English math problems, Qwen2.5-Math supports both CoT and TIR (Tool-integrated Reasoning) in both English and Chinese. In particular, TIR greatly improves precise calculation and symbolic manipulation ability for complex computation or algorithmic reasoning.

To improve performance, the following technical improvements were applied.

• Initialized from the Qwen2.5 base model to strengthen language understanding and code generation ability
• Built the Qwen Math Corpus v2 (utilizing over 1T tokens)
• Training optimization via Rejection Sampling and GRPO (Group Relative Policy Optimization)

As a result, it achieved dramatic performance improvements over the previous model on major math benchmarks such as GSM8K, MATH, and Gaokao.