Human-Agent-Society Releases 'Reef', a Continuous Learning Infrastructure

Human-Agent-Society has released 'Reef', an open-source infrastructure that continuously updates model weights and harnesses using user feedback accumulated after agent deployment. Previously, there was a gap between inference engines (vLLM, SGLang, etc.) and reinforcement learning frameworks (slime, veRL, etc.), requiring teams to build separate pipelines, but Reef integrates this process via standardized HTTP endpoints.

Key Features and Tech Stack

Reef combines SGLang (inference), slime (weight learning), and cordis (harness evolution) to update models without service interruption. Updated versions are managed via Git-based version history and are only released when evaluation results show superior performance compared to previous versions, managing regression risks. Installation is done via the reef-infra package in Python 3.10+ environments, with Git LFS required.

Learning Cycle and API

The learning process consists of four stages: Serve → Observe → Grow → Commit. It supports requests in OpenAI and Anthropic-compatible formats, specifies scenarios via the x-reef-scenario header, and sends numerical scores or text feedback through the /reef/report endpoint.

Application Recipes and Benchmarks

It provides various learning recipes suitable for in-house model serving teams. Key recipes include OpenClawRL, which learns solely from traffic without explicit reporting; harness_evolve, which evolves the harness tree; and SkillClaw, which updates skills only without GPUs. For the SAO (Single-Rollout Asynchronous Optimization) recipe, based on the Qwen3-30B-A3B model, the cumulative average reward at 48 rollouts improved to 0.479 compared to the baseline model (0.458), outperforming GRPO(+DIS) (0.417) (note: sample size is small, for reference only).