AI Briefing#benchmark
The latest AI and developer news about #benchmark, with the original source and a short summary.
The latest AI and developer news about #benchmark, with the original source and a short summary.
In an M3 Pro benchmark, Nemotron 3 Nano took first place among 4B-class models.
Qwen 3.6 Plus and Gemini were compared in a soccer video analysis benchmark.
OpenAI announced it is stopping SWE-bench Verified reporting and shifting to new evaluation axes.
By bypassing the vDSO on x86 Linux, timestamp latency was reduced by 30%.
In an 8GB VRAM / 32GB RAM environment, Q4_K_XL was slightly faster than Q4_K_M and also produced fewer output tokens.
A single H100 80GB benchmark confirmed the advantage of Gemma 4 E2B and Qwen 3.6 A3B FP8.
Lubuntu 26.04 outperformed Windows 11 in llama.cpp performance.
A dual RTX 6000 build passed a long-term bench test under 1650W wall power and 95°C CPU conditions.
A 36B MoE model built with Darwin V7 achieved 88.4% on GPQA Diamond.
Xiaomi announced the upcoming release of MiMo V2.5 Pro's weights, and the model ranked 54th on the Artificial Analysis Intelligence Index.
In a single H100 GPU benchmark, Gemma 4 E2B-it delivered the best results in both throughput and TTFT.
Retrieval selection boosted coding agent performance across 9 software tasks.
Stanford released its 2026 AI Index report, compiling changes in model performance, investment, and hiring.
A benchmark evaluating models' coding ability using 120 pure lambda calculus problems has been released.
The top score in the public inference challenge for Tesla T4·Qwen2.5-0.5B is 3,536 tok/s.
DeepSeek unveiled two 1M token context V4 preview models, pushing a low-price strategy.
OpenAI unveiled GPT-5.5 and revealed both performance improvements and pricing changes.
A study that quantifies Over-Editing (excessive rewriting) in coding models and reduces it via RL.
In a benchmark of 42 documents and 7,560 calls, cheaper/older models outperformed on OCR performance and cost efficiency.
Connecting Qwen3.6 to Claude Code on a dual 3090 setup cut the estimated 8-hour cost from $142 to under $4.
Gemma4 and Qwen 3.5/3.6 were compared on RTX 5090 using the same document-writing task.
On an RTX6000 Blackwell, Qwen 3.6 27B BF16 generated a Pac-Man HTML.
In a comparison of the new 3.6-27B models, the MoE model is rapidly catching up to the Dense model, particularly in coding performance.