AI Briefing#triton
The latest AI and developer news about #triton, with the original source and a short summary.
The latest AI and developer news about #triton, with the original source and a short summary.
New kernel fusion techniques for LayerNorm and RMSNorm hide up to 90% of normalization latency in GEMM kernels and achieve up to 35% speedup in attention kernels on NVIDIA B200 GPUs.
PyTorch's Helion now uses compile-time heuristics to select fast kernel configurations without autotuning, achieving 1.18x speedup over Flash Linear Attention and 1.88x over torch.compile for matmuls on H100.
The Triton Low-Level Extensions (TLX) framework enables explicit warp-group orchestration, delivering up to 2x forward speedup in GDPA kernels and reducing RecSys inference latency by up to two-thirds.
cuTile kernels that boost Mamba-2 forward propagation speed by up to 1.82x on B300 GPUs have been released.
The open-source library ParaRNN, which supports parallel training of nonlinear RNNs, has been released, achieving up to a 200x speedup.
Naver Cloud optimized the Mamba-2 kernel for B200 GPUs using Helion DSL, increasing single-layer computation speed by 1.33x.
Meta has disclosed its upstream synchronization and verification framework for FBTriton, a Triton fork.
Netflix built an efficient LLM serving system by integrating vLLM and Triton into its existing ML infrastructure.
It implements a block-diagonal attention kernel that is up to 3.5x faster than Flash Attention v2 on NVIDIA Blackwell GPUs.
Meta has unveiled IKBO technology that eliminates the redundant data duplication overhead occurring during recommendation model inference.
**OCTOPUS**, which maximizes transformer KV Cache compression performance by combining octahedral parameterization with joint quantization, has been released.
Stanford's CS336 course covers the entire process of LLM development, from data curation to Triton-based FlashAttention2 implementation and DPO alignment.
Using OpenAI Triton, a high-performance MoE inference kernel supporting both NVIDIA and AMD has been developed.
A Fused MoE dispatch kernel implemented in Triton delivers higher efficiency than Megablocks along with AMD compatibility.
The article argues that portability in AI kernels is a myth, and top performance comes from hardware-specific DSLs.
The single-cell TritonSigmoid recorded up to 515 TFLOPS on H100.
FlashQLA optimizes GDN Chunked Prefill with CP and fused kernels, achieving up to 3x speedup over FLA Triton.
Meta pushed ETT% above 90%, significantly reducing training inefficiency.
Kakaopay shared its process of comparing and optimizing frameworks to improve the consistency and performance of model serving.