AI Briefing#artificial-analysis
The latest AI and developer news about #artificial-analysis, with the original source and a short summary.
The latest AI and developer news about #artificial-analysis, with the original source and a short summary.
The new 200-task MKA benchmark, built with Artificial Analysis, evaluates agents on visually complex work across 10 verticals and 83 occupations.
The upgraded weights rank #1 open-weight and #6 overall in Provider Voice, while also leading in Controlled Voice and Pronunciation Robustness.
The new index weights 25% each for knowledge work, coding, science, and general tasks, showing a gap between frontier closed models and the best open-weight model, Kimi K3.1 Pro.
Google's Gemini 4 Carbon scores 59 on the Intelligence Index v5, leading in complex reasoning and scientific tasks while ranking second only to Claude Opus 5.5 on coding agents.
The updated index features 60% private datasets and new benchmarks, with Claude Opus 5.5 scoring 58 compared to GPT-6 Astra's 57.
The 2.8 trillion parameter model ranks 4th out of 215 models on the index but is noted for slow output speed and high verbosity.
The 816B parameter model scores 39, slightly below the 180B Qwen3.8-Flash-Next which achieves a score of 40.
The new Speech to Speech models support 90+ languages and 17,000+ voices, with v1 Max achieving 98.8% accuracy on Big Bench Audio and v1 Mini scoring 75.9%.
The open-weights MoE model achieves a score of 45, significantly above the median of 19 for similar-sized models, while offering 230.4 tokens/sec output speed.
Grok Voice 3.0 achieves a score of 86.9 on the Artificial Analysis Speech-to-Speech Index, ranking #1 among tested models.
Opus 5.5 scores 58 on Artificial Analysis Intelligence Index but costs $5.98 per task compared to GPT-6 Astra's $3.26 due to higher token usage.
Although its intelligence index is below average, it offers approximately 7x faster output speed and lower cost compared to models in the same price range.
The MiMo-V2.6-Pro released by Xiaomi recorded higher intelligence scores compared to similar-sized open-weight models and features competitive pricing, though its output speed was found to be slower than average.
In the 2026 open model ecosystem, China has seized leadership by overtaking the US in downloads, adoption rates, and performance metrics.
Ant Group has released the finance-specialized open-weight model 'Ling-3.0-flash-Fin', achieving high efficiency with 5.1B active parameters.
Artificial Analysis released Capability Indices v1.1, with Claude Fable 5.1 taking first place in all six occupational indices and Grok 4.5 demonstrating high cost efficiency.
Artificial Analysis released Intelligence Index v4.3, introducing Terminal-Bench 4.0 and AutomationBench-AA.
Upstage released Solar Pro 4, which enhances agent reliability, and joined the ranks of global frontier models with a score of 42 on the Artificial Analysis Index.
Artificial Analysis, in collaboration with Liquid AI, released a benchmark measuring the intelligence and inference speed of small LLMs under 8GB on the iPhone 17 Pro.
Alibaba's Qwen3.8 27B model recorded 52 points on the Artificial Analysis Intelligence Index.
Alibaba's Qwen3.8 27B model achieved a score of 52 on the Artificial Analysis Intelligence Index.
Grok 4.6 achieved frontier-level intelligence and agent performance at a low cost.
Qwen3.8 Max has taken the top spot overall on Artificial Analysis's Agentic Index.
Word Error Rate (WER) is a key metric for measuring the accuracy of speech recognition (ASR) models, evaluating performance through insertion, deletion, and substitution errors.