Sakana AI Proposes MASS Framework for Homogeneous Recursive Self-Improvement

Homogeneous Recursive Self-Improvement

Sakana AI researchers introduced MASS (Multi-Agent Self-Supervision), a framework for homogeneous Recursive Self-Improvement (RSI) where a single shared LLM simultaneously serves as the optimizee, optimizer, and evaluator. The system distills the collective intelligence of a multi-agent team into self-supervision signals for non-verifiable research tasks, eliminating the need for external ground-truth rules.

Architecture and Training Loop

The framework operates through two nested loops using a shared model (tested on Qwen3.6-27B):

• Inner Loop (Workflow Optimization): The model refines collaboration workflows (Roles, Instructions, Contracts, Hops) by comparing execution results against a champion workflow.
• Outer Loop (Weight Update): The model is fine-tuned via LoRA on top-ranked trajectories from the optimized workflows. To prevent the model from merely memorizing the workflow structure, the orchestrator prompt is stripped of workflow text during data generation, forcing it to learn organization from the task itself.

Performance and Efficiency

External evaluation using GPT-5.5 and Claude Opus 4.8 as judges showed significant improvements over the base model:

• Test Tasks: Win rates improved from 53.9% (Round 1) to 69.9% (Round 2) against the base model.
• Token Efficiency: Multi-agent trajectories provided denser supervision signals. The MASS model achieved a 68.3% win rate with 28.8M supervised tokens, outperforming a single-agent baseline that required 42M tokens for a 64.0% win rate.
• Benchmarks: Score-to-token ratios improved on MLR-Bench (1.60x), DSBench (1.35x), and ScienceAgentBench (1.20x). However, performance degraded or remained flat on Terminal-Bench 2.0 (0.96x) and SWE-bench Verified (0.94x), indicating limitations in software engineering contexts.

Key Observations and Safety Implications

• Role Generalization: Training on task-solving trajectories improved the model's ability to act as an optimizer and evaluator. Evaluator accuracy against external judges rose from 73% to 93% over three cycles.
• Optimizer Bottleneck: Experiments revealed that a weak optimizer limits progress more than a weak evaluator. Swapping in a stronger optimizer (GPT-5.5) accelerated workflow discovery significantly more than swapping in a stronger evaluator alone.
• Safety Risks: The authors highlight the risk of shared blind spots in homogeneous loops, where a model's flaws in both behavior and self-correction may reinforce each other. They emphasize the challenge of maintaining oversight when no stronger supervisor is available.