AI Briefing#performance-optimization
The latest AI and developer news about #performance-optimization, with the original source and a short summary.
The latest AI and developer news about #performance-optimization, with the original source and a short summary.
Inco AI has released Splash, an Apple Silicon inference engine that applies model-specific kernels and draft models.
A case study on implementing Databricks Metric View to resolve metric definition inconsistencies between dashboards and AI agents, resulting in approximately 50% reduction in agent operational costs.
Browser Use, in collaboration with TypeSafe, released an open-source agent that increases browser control speed by 25% using a dynamic indexed action space.
Migrated over 1,200 pipelines from Oozie and Sqoop to Airflow and Spark, reducing ingestion speed by up to 70% and improving query performance by 66x.
This article outlines methods to maximize bulk data read and write performance using Querydsl-based cursor pagination and batch processing.
OpenAI significantly improved performance by rewriting Habitat, a storage platform handling 1 billion users of traffic for ChatGPT, in Rust.
Research results have been published demonstrating that no KV cache policy outperforms LRU in simulations of real agent workloads.
This article summarizes the rendering, query, and incremental lookup optimization process encountered while implementing a 100k-point scatter plot using a custom Grafana plugin.
Kakao Bank rebuilt its notification sender by applying Service/Core Layer separation and Age-Based Work-Stealing to achieve 10,000 TPS and low latency.
The Rust-based CPU-only LLM inference framework eLLM has been released in beta, claiming to offer faster performance than GPUs for long inference tasks.
ik_llama.cpp has officially merged MTP support for Qwen3.8-Flash-Next, resulting in approximately 2x faster inference speeds on RTX 5090 environments.
Applying DSpark speculative decoding to LFM2.5 models improves GPU and on-device inference speed by up to 3.2x.
A new technique leveraging parallel ANE and GPU execution on Apple Silicon improves LLM Prefill speed by up to 50%.
AST-grep rewrote Tree-sitter's C core in Rust to improve performance for AI coding agent optimization.
gigatoken, a Rust-based tokenizer offering up to 1000x faster speed than HuggingFace tokenizers, has been released.
GigaToken, a tokenizer that boosts text processing speed to GB/s levels using SIMD and caching techniques, has been released.
Vercel precompiles bytecode when building Python functions, significantly reducing cold start time.
The Flags SDK has reduced microtask queue overhead, speeding up bulk evaluation of multiple feature flags by 10x.
Miricanvas resolved the stored fields bottleneck and optimized search performance by upgrading to Amazon OpenSearch 3.3.
Miricanvas introduced dual vector search based on Amazon OpenSearch Service for 40 million design resources and optimized performance.
SGLang is introducing executable skills for agents (SKILL.md) to automate complex LLM and Diffusion workflows.
Yeogiotae's Exhibition Development Team shares a case study of analyzing a CPU 100% outage caused by using Redis Repository and resolving it by switching to RedisTemplate.
This covers how es-toolkit, once an internal utility library at Toss, grew into a global project through performance optimization.
An update has been released for llama.cpp that optimizes prompt decoding performance when using MTP.