DeepSeek v4.1 flash (max) becomes first open-weight model to exceed 1% on SWE-sweep bug finding benchmark

·2026.11.14 01:31

Key point

Gemini 4 Hydrogen leads the proprietary field with 13.5%, while GPT Astra 6 follows at 9.5% on the same challenging benchmark.

DeepSeek v4.1 flash (max) is the first open-weight model to achieve a score greater than 1% on the SWE-sweep bug finding benchmark.
The benchmark requires models to fix hundreds of bugs in a large repository snapshot without hints, making it a reasoning-heavy, long-horizon task.
Gemini 4 Hydrogen is the top proprietary model with a 13.5% score, followed by GPT Astra 6 at 9.5%.
Evaluations were conducted in a standardized harness, with comparisons showing that Claude Code and Codex harnesses did not significantly improve performance.
The benchmark code is available on GitHub for independent evaluation.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.