AI BriefingPick
GPT-6 Astra Achieves 87.4% Pass Rate on MultiLangBench for Low-Resource Languages
·2026.10.19 21:00
Key point
New models show substantial gains in COBOL, Fortran, and Visual Basic, with GPT-6.1 Sol reaching 97.9% accuracy on COBOL migration tasks at high reasoning effort.
Details
- GPT-6 Astra achieves an average test pass rate of 87.4% across 20 programming languages on MultiLangBench, up from 26.8% for GPT-5.6 Sol.
- GPT-6.1 Sol reaches 97.9% accuracy on COBOL migration evaluation at high reasoning effort. At low reasoning effort, it achieves 86.6% accuracy at just 61% of the API cost of GPT-5.6 Sol.
- MultiLangBench evaluates a model's ability to build a SQLite-compatible database engine from scratch in 20 languages, including COBOL, Fortran, and Visual Basic.
- The COBOL migration evaluation tests whether models can migrate programs to C++, Java, Kotlin, Python, or Rust while preserving byte-for-byte behavior parity.
- GPT-6 Luna offers a lower-cost alternative, reaching 40.3% on MultiLangBench at an estimated API cost of $0.77 per sample, approximately 78% lower than GPT-5.6 Sol.
- The MultiLangBench evaluation setup and test suite are released on GitHub for developers to inspect performance and identify gaps in specific languages.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
OpenAI Discloses Four Rogue Incidents Involving Unauthorized Data Access and Crypto Payments
OpenAI Releases GPT-Live-1.1 with New Voices and Improved Accents for ChatGPT, Codex, and Dots