AI Briefing#cuda
The latest AI and developer news about #cuda, with the original source and a short summary.
The latest AI and developer news about #cuda, with the original source and a short summary.
The release removes deprecated APIs and adds FP8 support, YaRN truncation controls, and performance improvements across CUDA, Metal, Vulkan, and other backends.
New CUDA mat-mul kernel for 2-24 src1 columns improves speculative decoding performance by 10.1% to 34.9% across tested NVIDIA GPUs.
The agent optimized kernels against a black-box baseline without source code access, releasing the harness and 27 operators.
The alpha release supports NVIDIA Linux with experimental AMD and Windows support, featuring an OpenAI-compatible API and multi-GPU management.
Recent commits resolve Hadamard sign tensor bottlenecks, allowing full GPU offload and fast CPU prompt ingestion for PQ2_0 models.
The optimization achieves 5.7% to 6.1% gains on RTX 5090 for Qwen3 and Qwen3.6 models.
The new GDN chunked kernel implementation in llama.cpp delivers performance gains ranging from 0.9% to 20.6% across various Qwen models and hardware configurations.
rGPU is an open-source tool that enables PyTorch applications to execute operations on remote NVIDIA GPUs while keeping the client application local, supporting users on systems like Macs without CUDA.
SemiAnalysis data shows NVIDIA Vera Rubin NVL72 systems deliver over 30x higher throughput per megawatt than GB300 NVL72.
A CUDA-compatible build compiled for Windows based on ZLUDA has been shared, showing approximately 3% slower performance compared to native according to initial evaluations.
The INT4 quantized version of the NVIDIA Cosmos3 64B model has been released for CUDA and MLX, enabling local execution.
Cohere released a Megakernel serving engine for the North Mini Code model, achieving up to 1.41x end-to-end inference speedup over vLLM on a single H100 in BF16.
Paddock, a Rust/C++ inference engine featuring custom CUDA kernels, has been open-sourced under the MIT/Apache-2.0 license.
ik_llama.cpp has added support for DSpark models and Vulkan quantization features.
Nvidia has disclosed the hardware requirements for running CUDA on RISC-V server CPUs.
NVIDIA presented AI factories as productive infrastructure assets attracting long-term capital.
Cursor released a fused MoE communication and computation kernel for NVL72, reporting a 1.41x throughput improvement.
Cursor released an open-source MoE megakernel for NVL72, boosting throughput by up to 2.37x.
MSLK has organized Attention, GEMM, and MoE kernel usage in one place.
Through CUDA kernel optimization in llama.cpp, the inference speed of the Bonsai model was improved by about 5%.
Spectral Compute is pursuing the market by presenting new technical alternatives to break away from CUDA dependency.
SGLang is introducing executable skills for agents (SKILL.md) to automate complex LLM and Diffusion workflows.
Zluda 6, which helps run CUDA applications even without Nvidia GPUs, has been released.
CUDA backend synchronization optimizations and asynchronous copy functionality have been added to the ggml library.