Scaffold CoT Dataset Released

The Scaffold CoT dataset has been released to address the issue where free-form Chain-of-Thought (CoT) degrades the accuracy and structure of responses in small language models with 5B parameters or fewer. This dataset consists of approximately 4 million examples and 3 billion tokens, designed to enable small models to reason more concisely and reliably.

Structured Reasoning Framework Applied Without exception, all examples follow a fixed Scaffold structure composed of three stages: Interaction, Execution, and Answer. By maintaining a consistent format, the model avoids wasting resources on determining its own reasoning approach, allowing it to focus on actual reasoning and answer generation.

Improved Training Efficiency and Verifiability The length of each example is capped at 2048 tokens, facilitating training on consumer-grade hardware. Additionally, it includes metadata for various topics and sub-domains, enabling specialized training for specific tasks. Thanks to the fixed format, structural errors in model outputs can be mechanically verified using Regular Expressions (Regex).