Signals, curating agent traces

Signals extracts structured signals from agent interactions to first pick out trajectories worth reviewing, without needing humans or an additional LLM judge.

• Defines signals using a taxonomy that groups interaction, execution, and environment patterns.
• Captures states such as misalignment, stagnation, disengagement, failure, looping, and exhaustion.
• Does not change online behavior, and is a lightweight method that can be computed without a GPU.

In the τ-bench annotation study, the informativeness of signal-based sampling was 82%, higher than random sampling's 54%.

As a result, efficiency per informative trajectory improved by 1.52x, and the paper was released as arXiv 2604.00356. The implementation is available at katanemo/plano.