AI BriefingLangSmith LLM Gateway: Runtime Control for Agents
Key point
LangSmith LLM Gateway provides runtime control for centrally managing the cost, speed, and data protection of agent model calls.
Details
LangChain has launched the public beta of LangSmith LLM Gateway. This tool acts as a central governance layer positioned between agents and model calls, providing runtime controls to proactively prevent issues such as cost overruns, failures, and sensitive data leaks in production environments.
LLM Gateway supports cost management, rate limiting, model fallback, and sensitive data handling features. Spending limits can be set at the organization, workspace, API key, and user levels, and when limits are exceeded, a 402 response is returned to halt agent execution. Additionally, customer-specific policies can be applied in multi-tenant environments.
For sensitive data protection, information such as PII or secrets can be masked (redacted) during model calls, blocking sensitive data from being sent to model providers. By centrally managing both closed and open-weight models through a single gateway, vendor lock-in is prevented, and flexibility in changing model providers is ensured.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.