Open Reproduction of DeepSeek-R1

Hugging Face is running the Open-R1 project to help anyone reproduce and extend the core technology of DeepSeek-R1. This project aims to provide the key components that make up R1's training pipeline as open source.

The main components are as follows:

• src/open_r1: Includes scripts for training models using GRPO (Group Relative Policy Optimization), SFT (Supervised Fine-Tuning), and synthetic data generation using Distilabel.
• Makefile: Provides simple commands for running each stage.

The project is currently planned in 3 stages total, with Step 1 completed:

• Step 1 (Complete): Released Mixture-of-Thoughts, a high-quality reasoning dataset distilled from R1 (350,000 verified traces), and provides a recipe for training the OpenR1-Distill-7B model using it.
• Step 2 (In Progress): Plans to build large-scale datasets in math, reasoning, and coding to reproduce a pure RL (reinforcement learning) pipeline like R1-Zero.
• Step 3 (Planned): Plans to demonstrate a multi-stage training process leading from a base model to RL tuning.