RISED: Rubric-Based Framework for Multi-Environment LLM Agent Training

Researchers introduced RISED (Rubrics for Agentic Multi-Environment Selection and Self-Distillation), a framework designed to train single LLM agents across diverse interactive environments. Traditional methods often rely on scalar rewards, which fail to capture cross-environment relationships and provide no contrast when rollout groups are uniformly successful or failed. RISED addresses these limitations by using textual rubrics to guide both data selection and policy supervision.

How RISED Works

The framework employs an LLM judge to tag each rollout using a predefined rubric vocabulary shared across environments. These rubric profiles serve two primary functions:

• Data Selection: Profiles guide the selection of training data that aligns with the overall behavioral composition of mixed-environment batches while minimizing overlap with already-selected data.
• Self-Distillation: Positive rubrics (describing desired behaviors) provide privileged context for an on-policy self-distillation teacher, offering token-level supervision. Negative rubrics (describing undesired behaviors) guide subsequent rollout generation away from recurring failure modes.

Performance and Analysis

Across various model backbones, RISED achieved the highest mean pass rate across environments and ranked first or second in every individual environment. The rubric-based analysis further characterizes the behavioral changes that accompany these performance gains, demonstrating the utility of textual feedback beyond simple reward signals.