AI Briefing#benchmark
The latest AI and developer news about #benchmark, with the original source and a short summary.
The latest AI and developer news about #benchmark, with the original source and a short summary.
Qwen3.6-27B achieved 95.7% on SimpleQA in a single RTX 3090 local environment.
In a 30-day Bitcoin prediction benchmark, Perplexity ranked first.
MiniMax M2.7 AWQ-4bit was compared for performance and energy efficiency on 2x Spark and 2x RTX 6000 96GB.
An open-source tool that checks AI misalignment using 32 tests has been released.
Ouroboros outperformed plan mode and superpowers on a real DES benchmark in the Claude Code environment.
Open-source Ouroboros beat Claude Plan Mode to take the #1 spot on a modeling and simulation benchmark.
Ran a Qwen3.6-27B UD-Q5_K_XL benchmark on an RTX 5090 with 19 runs and 93.9k tokens.
In a MacBook Pro M5 Max test, Gemma 4 31B outperformed in Pac-Man generation quality.
In an 18GB M3 Pro bench, Gemma4:e4b ranked first with 72%, while Qwen3.5:9b scored 68%.
On LoCoMo, the local 20B model dropped only about 4 points after 1,100 poisoned memories were injected.
Local inference speeds for Qwen3.6 and Gemma 4 on an RTX 4070S 12GB were shared.
Re-measured the quality and long-context speed of Qwen 3.6-35B-A3B KV cache quantization on M5 Max.
Claude Code with MCP code health guidance increased refactoring improvements by 2-5x.
On 4x 5060 Ti, vLLM recorded pp 1444.9 tok/s and tg 47.4 tok/s.
DharmaOCR released specialized 3B and 7B SLMs along with cost and performance benchmarks.
The 2026 MLC-SLM Challenge has opened registration for a multilingual Speech LLM task with a $20,000 prize pool.
Anthropic CVP released 6 rounds of evaluations of 4 Claude models.
Benchmark results were released for 17 local LLMs and 5 agent harnesses.
Compared the performance and weights of 4 abliteration techniques applied to GLM 4.7 Flash.
Across 36 test runs, v4-flash had zero downtime while v4-pro was the slowest.
Raising effort on Opus 4.7 barely changed refusal patterns, but made answers longer.
For Opus 4.7, refusal tendency stayed the same as effort tier increased, with only response depth growing.
Qwen3.6 27B and 35B-A3B were compared across multiple engines on 2x 5060 Ti 16GB.