UiPath Reveals Case Study on Measuring and Improving Claude Code Skill Activation Rates

UiPath released insights and an open-source evaluation tool discovered while measuring and improving the activation rates of 20 of its Claude Code skills.

Measuring Skill Activation Rates and Initial Issues

In a progressive disclosure approach, if an agent does not select a skill, the value of that skill is close to zero. UiPath's measurements showed that the initial micro-recall was 46.3%, meaning more than half of the prompts where a skill should have been invoked did not call any skill.

The first discovery was a bug in the evaluation harness itself. The existing detector was designed to detect only Claude's Skill tool calls, so Codex and Antigravity, which lack the Skill tool, were incorrectly recorded as not using skills. In reality, they were applying skills via file reading, and even Claude sometimes read SKILL.md directly for the second skill onwards.

Improving Descriptions with Unique Markers

Adding a one-line description in the format "Always invoke for X" that specifies unique markers (such as file names or product-specific metrics) owned only by that skill resulted in micro-recall rising from 46.3% to 67.3%. In particular, diagnostics skills improved significantly from 0.16 to 0.68, and maestro-flow from 0.32 to 0.72, while precision remained at a level of 0.96.

However, cross-functional skills without unique markers (e.g., 'review') may actually see performance degradation, suggesting that not all skills can be anchored by unique markers.

Open-Source Evaluation Harness Released

Emphasizing that it is difficult to distinguish between harness bugs and agent bugs when the agent and evaluation harness are based on the same model (Claude Code), UiPath released an Apache 2.0 open-source harness for cross-agent evaluation on GitHub.