AI Briefing#optimization
The latest AI and developer news about #optimization, with the original source and a short summary.
The latest AI and developer news about #optimization, with the original source and a short summary.
Through CUDA kernel optimization in llama.cpp, the inference speed of the Bonsai model was improved by about 5%.
Model routing in AI agent systems isn't a model-selection problem but a system optimization problem involving caching, infrastructure, and regulations.
ExLlamaV3 v1.0.0 has been released, offering improved inference speed and support for a variety of models.
This introduces LOTUS, an LLM-based data analysis optimization framework developed by researchers at Stanford and Berkeley.
Under resource constraints, specialized AI models focused on specific domains show higher efficiency than general-purpose versatility.
SSG.COM discovered a duplicate query issue in its promotional event API and optimized performance through structural improvements.
A new training technique is proposed that interleaves the management of 3 layer states to improve GPU memory efficiency.
CUDA backend synchronization optimizations and asynchronous copy functionality have been added to the ggml library.
Dropbox leveraged DSPy to refine its LLM-based evaluation model, thereby improving both the response quality and efficiency of the Dash Chat agent.
This proposes an algorithm that practically solves the theoretically intractable problem of finding the optimal Tokenizer via Integer Linear Programming (ILP).
This covers a case study on introducing MLXP optimization technology to improve LLM serving efficiency in Kubernetes environments.
A monokernel technology for LLM inference has been unveiled, achieving up to 3,300 tokens/s per request on AMD MI300X.
This is the first post in a guide series covering how to use torch.profiler to identify and optimize performance bottlenecks in PyTorch models.
Proposes CODA, a GPU kernel abstraction that uses the GEMM-Epilogue approach to solve the memory bottleneck in Transformer training.
A study on Mix-Quant has been published, improving efficiency by applying aggressive quantization to the prefill stage while keeping the decoding stage at high precision
Running an agent multiple times and combining the results can be more efficient than improving the model itself.
Parax, a JAX-based parametric modeling library, has been updated to version v0.7.
A new theory explains deep learning generalization using output space and eNTK.
It outlines the possibility of establishing a learning mechanics theory that explains deep learning training.
A new arXiv preprint unifies four ML failure modes into a single supervised learning theory.
The llama.cpp CUDA MMQ kernel improved speed in some ranges using fastdiv and dynamic tiling.
A Metal kernel implementation combining INT3 model compression with INT2 KV cache has been released.
By restructuring static embeddings, it achieved runtime speeds up to 100x faster than existing models.
A case study in optimizing Bootsnap and Ruby's `require`, `Dir`, and `File.join` bottlenecks down to about 2x.