AI BriefingSWE-Pair Benchmark: DeepSeek V4.1 Flash Outperforms GPT-6 Luna on Dual-Bug Fixing Tasks
·2026.11.11 03:33
Key point
In a new benchmark requiring simultaneous fixes for two real bugs, DeepSeek V4.1 Flash achieved a 90% success rate at a cost of 6 cents per task, surpassing GPT-6 Luna.
Details
- SWE-Pair is a new coding-agent benchmark consisting of 161 tasks from 69 Python projects, requiring agents to fix two real bugs simultaneously.
- DeepSeek V4.1 Flash at medium effort solved 90% of tasks at a cost of approximately 6 cents per solved task.
- GPT-6 Luna at max effort solved 78% of tasks at a cost of approximately 23 cents per solved task.
- Increasing reasoning effort did not improve performance for DeepSeek; max effort scored 88% (slightly below medium) while costing 1.7x more.
- The benchmark dataset is available on Hugging Face, and results from every agent run are published on the project's lab site.
- Users can run their own agents on the benchmark using a single Harbor command.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.