LangSmith adds AI Search to trace agent behavior with natural language

LangSmith has introduced AI Search, a feature that allows users to query agent traces using natural language to find specific behaviors, such as user frustration or PII leaks. Unlike traditional full-text search or vector embeddings, which can lose critical details in compression, AI Search uses a fast decision model to read the actual content of traces and determine if a described behavior occurred.

How It Works

The system relies on SmithDB, a database purpose-built for trace data, which normalizes traces from various sources (OpenAI, Anthropic, LangChain, etc.) into trajectories. A trajectory is a distilled view of an agent's session, listing each message, tool call, and result once to remove redundancy. AI Search analyzes these trajectories rather than raw traces, ensuring consistent results across different SDKs and reducing context bloat.

To handle large conversations, AI Search applies a fixed 8K token budget to each trajectory. It prioritizes user messages and agent replies, filling the remaining budget with intermediate reasoning and tool outputs. This approach addresses the reality that while some agent conversations exceed 1 million tokens, most behavioral questions can be answered with just a few thousand tokens of relevant context.

Availability and Performance

Searches are optimized for speed by first applying structured filters to narrow candidates, then running the decision model in parallel across batches. Matches are streamed back as they are found, preventing users from waiting for a full scan. AI Search is currently in public beta on LangSmith Cloud in the GCP US, EU, and APAC regions, available at no additional cost for org admins to enable.