AI Briefing#llm-optimization
The latest AI and developer news about #llm-optimization, with the original source and a short summary.
The latest AI and developer news about #llm-optimization, with the original source and a short summary.
The reusable reading control layer preserves task quality in 97% of settings across six models, including Claude Opus 5.
InfraRSI achieved a geometric mean speedup of 1.83x on H100/H200 by simultaneously exploring configurations and code.
The new quantization method reduces token usage by up to 50% and improves accuracy by 15% compared to other quants of the same size.
Apache 2.0 licensed tool processes 491 questions per second on an RTX 4090 by reading content once for multiple decision queries.
Reduced inference costs by 77% for a Qwen3-4B-based grading model through cost-weighted vocabulary selection.
The post-trained model achieves token reductions of 22% to 51% across six benchmarks while retaining 88.8% average accuracy.
The new quantization offers three GGUF sizes ranging from 10.93 to 20.82 GB on Hugging Face.
A drop-in component for transformers that reduces peak memory by 1.5-3.4x while maintaining accuracy.
A hybrid rules-and-LLM graph construction method reduces agent input tokens by 92% and model calls by 66% while maintaining 100% accuracy on firm-wide queries.
Optimizing prompt caching and switching to Sonnet 5.5 reduced median review time from 4m 09s to 1m 32s while maintaining finding quality.
The system uses a compiler to resolve code structure, freeing AI cognition for higher-level decisions at a cost of $12.96.
The method reduces pretraining costs by over 99% and enables controlled comparisons of hybrid architectures like Gated KalmaNet, Gated DeltaNet, and Mamba-2.
Integrating the hosted Jev decision model with Qwen 3.8 27B reduced execution API costs by 26.6% while maintaining 100% task accuracy on 360 SOP-Bench cases.
The new model, available in Microsoft Foundry, delivers calibrated probability scores for routing and classification tasks at 35 times the speed of GPT-6 Sol.
Using Qwen3-8B on FanOutQA, CacheBack achieves 3.2× faster median task completion and 14.7 percentage points higher accuracy than text-based communication.
The training-free framework reaches 4.81x speedup on Huginn-3.5B by concurrently batching drafting and verification.
The new Jeff-Code agent uses a small decision model to pre-fetch information and manage thinking levels, reducing average task time to 0.68x of the baseline with no significant drop in accuracy.
A new benchmark, the Telegraph Test, demonstrates that LLMs can read compressed 'cablese' records with accuracy equal to or better than plaintext records, offering 25–49% token savings for machine-to-agent communication.
FLEET reduces sampling iterations by half on GSM8K and cuts them from 32 to 9 on LiveCodeBench by using past rewards to adjust logits.
The new middleware reduced median cost per coding task by 64% in Open SWE experiments without impacting quality.
Benchmarks on Terminal-Bench 2.1 show Mixdog achieves comparable accuracy to Codex CLI and Claude Code while reducing costs by 19-39% and context usage by 28-46%.
The hybrid approach uses three tiny MLPs on a frozen Gemma 4 backbone to decide whether to speak, defer, or wait, achieving up to 99.0% start-turn accuracy on local benchmarks.
Devin Fusion now leads FrontierCode 1.1 Extended with a score of 68.8 at an average cost of $0.60 per task.
The new commands automate eval design and iterative improvements, achieving up to 98.9% accuracy at one-fifth the cost in internal benchmarks.