SkillOS: Learning Skill Curation for Self-Evolving Agents

SkillOS is an experience-based RL recipe that enables LLM-based agents to accumulate and curate skills from past experience. It targets the problem that relying on manual curation or short-horizon skill operation, as before, makes it hard to learn long-horizon policies mixed with delayed feedback.

The core is the separation of execution and curation. The frozen agent executor retrieves and applies skills from an external SkillRepo, while the trainable skill curator updates the SkillRepo based on accumulated experience. By keeping execution fixed, learning is focused solely on what to keep and how to refine it.

• The learning signal is constructed from composite rewards and grouped task streams that reflect skill dependencies.
• Trajectories from earlier tasks update the SkillRepo, and subsequent related tasks evaluate the effect of those changes.
• This structure allows direct learning of a long-horizon skill curation policy.

In experiments, SkillOS consistently outperformed memory-free baselines and strong memory-based baselines across both multi-turn agentic tasks and single-turn reasoning tasks. This curator also generalized well across different executor backbones and task domains, and induced more goal-directed skill usage.

Over time, the skill documents in the SkillRepo evolved into more structured Markdown form. This means the repository shifted from simple task instructions into a repository containing higher-level meta-skills.