General Reasoning releases RLCR recipe for training agent swarms via collective rewards

Details

General Reasoning released the "reinforcement learning from collective rewards" (RLCR) recipe, which uses a core policy gradient algorithm with a shared policy and flat collective advantage to train agent swarms.
RLCR incentivizes "collective inference-time scaling" by imposing latency pressure through time limits, rewarding faster completion rather than explicitly penalizing every second, unlike previous latency penalty methods.
In the Avalon-M1 implementation, a 5-8 hire collective starts below average performance but reaches nearly 2x the average useful work by step 200, surpassing single-agent baselines at a matched-token budget where single agents take 4x as long.
The approach addresses the high latency of sequential reasoning models by dividing work among agents, though it faces the "village problem" of scaling from 9 training agents to thousands in deployment and the "uneasy triangle" of credit assignment in long-horizon, multi-agent environments.
Benchmarks mentioned in the results include EdgeBench, SWE-Bench Pro V2 (Hard), Terminal-Bench 2.1, and IMO-ProofBench-Advanced.
Communication primitives vary by system, such as Kimi K2.5 Agent Swarm (orchestrator with up to 300 sub-agents in K2.6) and OpenAI's Navier-Stokes system (~10,000 agents in island groups).

Key point