AI Briefing
KOSign in

Google AI & Infrastructure releases PLE benchmark, finding all frontier models score below 50% on physical engineering tasks

·2026.11.03 09:00

Key point

The new benchmark features 85 long-horizon tasks across mechanical, electrical, civil, and structural engineering, with the top-performing model reaching a mean score of 48.3%.

1 / 2

Details

  • Google AI & Infrastructure announces PLE (Physical-Engineering’s Last Exam), a new benchmark and open competition developed with Mercor to test frontier AI agents on real-world mechanical, electrical, civil, and structural engineering tasks.
  • All 14 frontier models tested scored below 50% on the private test set, with the highest-performing model on its native harness achieving a mean score of 48.3%.
  • PLE consists of 85 long-horizon tasks (30 mechanical, 27 electrical, 18 civil, 10 structural) where agents must process an average of 61 files per task, including CAD drawings and BIM models, to produce code-compliant, cost-effective designs.
  • Model choice drives performance far more than the agent harness, with a spread of up to 50 points across models compared to a median 3-point spread across harnesses for the same model.
  • 29 public tasks, grading tools, and an open competition leaderboard are available at ple-bench.com, with 13 Google-sponsored models running free on Google servers for evaluation.
  • The benchmark distinguishes between expert-authored tasks (44) and co-authored tasks (41), where the latter were assembled by agent pipelines and validated by licensed engineers.
ai-agentsmodel-evaluationmercorgoogle-aiple-benchmarkphysical-engineering
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

GeekNewsHerdr releases native desktop app for managing terminals and coding agentsNVIDIA BlogAI and accelerated computing cut semiconductor simulation times by up to 35xHacker NewsNVIDIA releases Auto Ontology, an open-source framework for agent data contextVercel BlogMercor cuts batch job times from 12 to 2-3 hours using Vercel AI GatewayHacker NewsResearchers Demonstrate Remote Code Execution Backdoors in LLM Agents via Pretraining PoisoningLangChain BlogHarrison Chase: Proactive, Always-On Agents Create New Context Engineering Challenges