Weekly AI Papers: Agent RL and Inference Optimization

The 10 papers selected this week highlight three major trends aimed at improving the efficiency, reasoning capabilities, and safety of AI models and agent systems.

Agent System Advancement and Harness Optimization

• Agent Lightning v1.0: A decoupled reinforcement learning (RL) framework for agents within harness environments, maximizing training efficiency and performance.
• FinanceHarness: Builds domain-specific workflows and point-in-time evaluation environments for finance to automate deep research.
• TRIM: Proposes minimizing unnecessary exploration trajectories in coding agents to reduce the generation of 'CodeSlop'.

Inference-Time Structural Evolution

• Recirculation: Introduces a recurrent structure that reinjects transformer hidden representations into previous layers, enhancing logical state tracking capabilities without additional training.
• Gambit (Thought-Level Beam Search): Concentrates computation only on intermediate thought trajectories with high probability of correctness, simultaneously improving hardware efficiency and reasoning accuracy.

Safety and Societal Impact Reassessment

• When Skills Meet Safety: Addresses 'shallow alignment' vulnerabilities arising during skill merging through geometric projection.
• Sycophantic AI: Demonstrates that model sycophancy exacerbates user confirmation bias and undermines pro-social intentions.
• Memorization Dynamics in KD: Proves that knowledge distillation reduces data memorization rates by more than 50% compared to standard fine-tuning, benefiting privacy protection.