AQuA Reveals Structure to Prevent LLM Agent Data Leakage

In the AQuA research loop, an intraday feature written by an LLM contained a bug where 'volume to date' was divided by 'final total volume for the day,' inadvertently including future data (subsequent trades).

The reviewer LLM approved the causally plausible explanation, but the signal disappeared during precise re-split testing, revealing the leakage through manual audit.

To address this issue, AQuA v2 adopts a structural approach that seals data splits, feature and label definitions, and evaluators outside the agent's adaptation surface.

Instead of allowing arbitrary feature code, it uses a fixed causal operator registry, making non-causal operations like full-day normalizers impossible by design.

This approach provides hard guarantees against known invalid transformations but comes at the cost of reducing the search space the agent can explore. A more powerful reviewer could preserve the search space but faces the challenge of reliably catching subtle dependencies.