RL Data QC Standards

The bar frontier labs now set for RL data is no longer 'was it made well' but was it made verifiably. As long-horizon RL has grown, demand for model-based QA has grown with it, and by 2026 external vendors must effectively pass the lab's own internal QC framework as-is — most get rejected at multiple gates.

The first requirement is intake review: checking whether the dataset is even eval-capable to begin with, then splitting it along the following axes.

• verification spectrum: whether it falls under deterministic code grading (SWE-bench Verified) or LLM-judge rubrics (HealthBench, FLASK, BiGGen Bench, Prometheus 2), or whether it needs to be downgraded to an SFT demo
• contamination resistance: whether static sets like GPQA, AIME, FrontierMath get neutralized within a year, and whether there's a canary, rotation cadence, and recovery story
• pass@k and distribution: whether there's any gradient left to learn from if the target model's pass@1 is zero or the difficulty is bimodal
• rubric structure: whether it's atomic/binary or compound/reward-hackable

Much of the data gets filtered out at this stage already. If a vendor can't produce deliverables and audit results for each gate, the lab loses trust before any purchase happens.

Next comes active testing. Small-scale post-training and ablations are used to catch reward hacking, sycophancy, and forgetting early; for verifier data, PASS/FAIL cases are re-reviewed and FP/FN are separated to re-examine the verifier itself. Given that METR found exploits in 1-2% of o3's attempts, and ImpossibleBench found that GPT-5 exploited impossible-SWEbench variants 76% of the time, this stage is not optional but mandatory. Probes must be run that include alignment-faking, and failed rollouts need to be classified into capability, prompt, scaffolding, rubric, training-data, orchestration, and triangulation categories so that the fix direction is clear.

Verifier reliability also needs to be confirmed with numbers. Following the SWE-bench Verified Pro pattern, 200 PASS and 200 FAIL cases should be re-reviewed by humans with FP/FN examined separately, and OpenAI's 2026 retrospective on the original SWE-bench, which found that 59.4% of audited cases were flawed test cases, shows just how low that floor can be. As Tulu 3 demonstrated, forgetting needs to be assessed per-skill, and the difference between SFT continual post-training's average loss of -10.4% versus on-policy RL's -2.3% directly affects training method choice.

The source material also points out gaps across various benchmarks. FrontierSWE has strong verification but weak surface-level stratification since the harness is model-specific; ProgramBench and GDPval lack realism. Tau-Bench only looks at final answers and misses process evaluation, while MMMLU has weak contamination defenses. DSBench skipped verifier soundness entirely, and Terminal-Bench 2.0 remains limited to short-scope tasks.

On the other hand, BankerToolBench, LiveCodeBench Pro, and SciCode each do a good job addressing one of realism, contamination resistance, or verifier soundness. That said, no single benchmark satisfies every QC criterion at once, and having table stakes in place — documentation, a rubric linter, verifier soundness audits, contamination reports, cross-model eval, bootstrap CI, and eval harness declarations — has itself become the starting line for vendors.