Open Models Have Crossed the Threshold

Open weight LLMs like GLM-5 and MiniMax M2.7 have reached performance comparable to closed frontier models on core agentic tasks such as file operations, tool use, and instruction following. According to LangChain's Deep Agents eval results, open models have now risen to a level where they can be a practical deployment option in terms of cost and latency.

The criteria for evaluating open models are cost, latency, and task performance. The article notes that closed frontier models can be 8-10x more expensive in high-throughput environments, and also have longer response latency. For example, for an application outputting 10M tokens/day, the daily cost is roughly $250 on Opus 4.6 versus roughly $12 on MiniMax M2.7, an annual difference of about $87k.

Performance was measured across 7 eval categories.

• file operations, tool use, retrieval, conversation, memory, summarization, unit tests
• Success criteria look at both correctness and efficiency
• Metrics consist of correctness, solve rate, step ratio, and tool call ratio

The results show that open models are already sufficiently competitive. GLM-5 recorded 0.64 correctness (94/138), and MiniMax M2.7 recorded 0.57 correctness (85/138). The comparison frontier models scored Claude Opus 4.6 0.68, Gemini 3.1 Pro Preview 0.65, and GPT-5.4 0.61, showing the gap isn't large. In particular, GLM-5 achieved 1.0 on file ops, retrieval, and unit tests, and was also faster in terms of latency, at 0.65s / 70 tokens/s based on OpenRouter data, compared to Claude Opus 4.6's 2.56s / 34 tokens/s.

Based on these results, Deep Agents has made using open models easy. Switching models takes just one line, and the harness automatically adjusts for context window, unsupported modalities, and model identity.

• In the SDK, you can swap directly with something like baseten:zai-org/GLM-5, openrouter:minimax/minimax-m2.7
• In the CLI, you can switch models mid-session with the /model command
• Role splitting is also possible, such as using a frontier model for planning and an open model for execution

The article states that going forward, it plans to further experiment with open model family-specific harness tuning, and multi-model configurations such as frontier orchestrator + open subagents. The core message is clear: open models are no longer a supplementary option, but have become a practical option for agentic workflows.