SWE-rebench Multilingual Benchmark Update

·2026.07.29 01:37

Key point

SWE-rebench announced a multilingual software engineering benchmark update that goes beyond Python to include 5 languages such as Go, Java, and Rust.

Details

SWE-rebench updated its leaderboard by adding a Multilingual slice based on real software engineering tasks. Moving beyond its previous Python-centric focus, it now newly includes 5 languages: Go, Java, Python, Rust, and TypeScript.

The Pass@1 performance results for major models are as follows:

GLM-5.2: 62.9%
MiniMax M3: 47.2%
MiMo V2.5 Pro: 46.5%
DeepSeek-V4 Pro: 40.2%
Qwen3.6-27B: 31.2%

This update focused on verifying the coding capabilities of open-weight models from multiple angles, and an additional update targeting models suitable for local deployment is planned within the next 3-4 weeks.

software-engineering benchmark llm-coding multilingual-eval swe-rebench

Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.