LangSmith Engine v2 Adds Red Teaming and Automated Fix Validation

LangChain has released LangSmith Engine v2, an update to its in-platform agent that automates the agent development lifecycle. Since its May launch, Engine has analyzed over 70 million traces and diagnosed tens of thousands of issues. The v2 update focuses on proactive detection and pre-validated fixes to help developers ship faster and avoid production regressions.

Proactive Red Teaming

Engine v2 introduces Red Teaming, a proactive troubleshooting tool designed to catch issues like hallucinations and system prompt violations before they affect users. By analyzing production traces and repositories, Engine understands the agent's purpose and behavior, then tests for weaknesses to identify potential failures. This feature is currently available in Private Beta for existing LangSmith Deployment users.

Detecting Subtle Performance Issues

Beyond obvious errors, Engine v2 now flags harder-to-spot issues that often escape human review, such as inefficient agent trajectories and performance trends. It analyzes agent trajectories to identify inefficient paths, like repetitive tool calls, and proposes more efficient alternatives. Additionally, it tracks trends in key metrics including error rate, latency, and cost, consolidating all issues into a single queue for resolution.

Automated Fix Validation

For agents using LangSmith Deployment, Engine now validates its proposed fixes before presenting them for human review. The process involves reproducing the failure, proposing a change, testing it against the same inputs, and evaluating the results. Once Engine confirms the fix resolves the issue, it provides a one-click option to open a PR for deployment. This feature is also in Private Beta for LangSmith Deployment users.

Performance and Availability

Engine’s baseline performance has improved, with issue detection now more than 2x better (measured by IssueBench) and fixes 25% more effective (measured by Terminal-Bench). Engine v2 is available in SaaS deployments for all LangSmith Plus and Enterprise plans. Support for self-hosted plans is coming shortly, with a future release planned to support bring-your-own-key (BYOK) for inference while keeping trace data within the user's VPC.