AI Briefing#benchmark
The latest AI and developer news about #benchmark, with the original source and a short summary.
The latest AI and developer news about #benchmark, with the original source and a short summary.
In a 98-question political benchmark, GPT-5.3 declined all of them once an opt-out option was given.
A fast, uncensored local model optimized from Gemma 4 26B into MLX 4-bit.
Results from training a shopping agent with RL across 8 verifiable e-commerce environments.
The core of the Claude Code performance-degradation controversy is less about model nerfing and more about hidden operational variables.
The authors propose **ManyIH**, which resolves agent instruction conflicts using up to **12 tiers** of authority.
Qwen3-1.7B surpassed GLM-5 744B using only synthetic data fine-tuning.
A tuning log running Qwen3.5-35B at 64k context on an RTX 4060 Ti 16GB, hitting 40-60 tok/s.
Built a paper-based web app with Qwen3.5 35B and confirmed it maintains context well up to 70-80K tokens.
A diffusion LLM has achieved both AR-level quality and parallel generation speed together.
Running 90 queries across 8 models showed that 86% of results appeared on only one provider.
Ran Gemma 4:26B on a Mac Studio M4 Max using OC + LM Studio, achieving 40 tok/s.
In the latest oMLX commit, DFlash optimization significantly boosted tg TPS for small contexts.
On an M2 Max, DTree showed MLX results **1.07x** faster than DFlash.
ScienceWorld and DiscoveryWorld reveal how AI science agents actually perform.
Complaints about Ollama Cloud's response latency and low TPS have escalated to the point of subscription cancellations.
After testing 30 TTS engines on a MacBook Air M4, Kokoro turned out to be the optimal choice, and quantization actually made things slower.
Give local LLMs 'private time to think alone,' and each model reacts differently.
A benchmark comparison of read performance across files, memory, SQLite, and indexes.
VAKRA exposes agent reasoning failures in enterprise-style API and document tasks.
Robyn was more stable and had lower tail latency than FastAPI under high load.
The limitations of natural language metrics in SQL code generation evaluation and a **20% false positive** rate have been pointed out.
An experiment interpreting reasoning degradation and catastrophic forgetting as the same kind of structural collapse.
A dynamic expert cache in llama.cpp boosted Qwen3.5-122B-A10B token generation speed by 27%.
With DFlash in oMLX 0.3.5 RC1, Qwen3.5 27B went from 9 to 22 T/S on a Mac M5 Max.