Microsoft Releases ThinkingBox Training Stack for RL Post-Training of Agents Using Database State Rewards

• Microsoft released the ThinkingBox training stack, a framework that converts executable database state checks into binary reinforcement learning rewards for post-training agents.
• Qwen3.8-27B fine-tuned with GRPO for 50 steps on 24 H100s improved average success (pass@1) from 51.70% to 60.78% and increased tasks completed correctly in all 20 attempts (Observed 20/20) from 7.50% to 21.10%.
• The training process sharpened model repeatability, as indicated by the significant rise in Observed 20/20 metrics.
• Cost per dependable task (20/20 pass) dropped from $24.36 to $8.00.

Technical Methodology

• Task selection uses offline policy-relative difficulty to filter the training pool, ensuring groups contain mixed success/failure outcomes to generate valid advantage signals for GRPO.
• Reward signals are primarily deterministic terminal-state checks on database state and side effects, with a minority of narrow rubric requirements handled by an LLM judge.
• Zero-variance groups are replaced using a DAPO-style method, and backend timeouts are refilled rather than scored as failures to maintain training stability.

Evaluation and Availability

• Evaluation on ThinkingBox-Bench (507 tasks) is out-of-distribution, as the benchmark domains and databases were excluded from the RL training pool.
• Code is available on GitHub under the MIT license, and benchmark data is released under the CDLA-Permissive-2.0 license.
• The project is a collaboration between the Microsoft Copilot Studio team, Toloka, and researchers from the University of Pittsburgh, Northwestern University, Columbia University, and UC Irvine.