LangChain integrates ScaleDown constrained decoding models for efficient long-running agents

LangChain has released langchain-scaledown to integrate ScaleDown's constrained decoding models (CDMs) into long-running agent workflows. These specialized models are optimized for single tasks such as summarization, compression, extraction, and classification, offering significant efficiency gains over general-purpose LLMs. ScaleDown reports its models are 200x faster and 467x cheaper than frontier models, with specific benchmarks showing the Summarize model is 41x cheaper and the Compress model 138x cheaper than GPT-5.5/5.4 averages.

Middleware Integration

The package provides three middleware components for LangChain and Deep Agents that replace standard context management steps:

• ScaledownSummarizationMiddleware: Replaces built-in summarization, keeping the agent's planning model intact while offloading summary generation to ScaleDown.
• ScaledownToolCompressionMiddleware: Adds a compression_query parameter to tools, compressing tool results to only relevant parts before passing them to the model. In one example, a docs page was compressed by 90% (18,402 to 3,110 chars).
• ScaledownExtractionMiddleware: Extracts structured data from user messages before the first model call, storing it in the agent state for routing or cumulative tracking.

Gateway Support and Usage

ScaleDown is now a direct model access provider in the LangSmith LLM Gateway at the /scaledown path. This allows admins to register a single API key, ensuring all ScaleDown calls are traced and counted against standard spend and rate limits. The models support up to 1M tokens input and multimodal inputs (images, PDFs) for extraction and classification tasks.

ScaleDown is recommended for high-volume structured decision steps rather than open-ended generation like coding or drafting. New accounts receive 50M free tokens to test the integration.