Regression Testing with LangSmith

In LLM application development, fast and reliable evaluation helps AI engineers conduct iterative experiments with confidence. Unlike typical software testing, AI applications don't always achieve a 100% pass rate on test results, making it critical to track performance trends and compare individual data points across experiments.

LangSmith provides the following improved regression testing infrastructure to meet these requirements.

• Comparison View: Provides a view where multiple experiments (Runs) can be selected simultaneously to compare results at a glance.
• Display Options: Provides control features that let users select and customize the information they want to see (full text content, per-call latency, etc.).
• Data Drill-down: Automatically marks data points where metrics have increased or decreased relative to the Baseline in red and green, allowing immediate identification of where performance changes occurred and viewing of the details.