AI Briefing
KOSign in

JSSA Achieves 6.43x Decoding Throughput on B200 GPUs for 128K Context

·2026.10.29 17:08

Key point

The method maintains near-full-KV accuracy while supporting 700B scale models at 96-99% of DSA throughput.

Details

  • Joint Subspace Sparse Attention (JSSA) addresses token-level selection challenges by learning a shared low-dimensional query-key subspace to preserve ranking accuracy.
  • Achieves up to 6.43x decoding throughput compared to full-KV attention at 128K context on an NVIDIA B200 GPU.
  • Utilizes a gather-free token-level indexed-attention kernel that supports distinct per-KV-head selections under standard grouped-query attention (GQA) layouts.
  • Reaches 96-99% of model-native DSA decoding throughput at 700B scale without requiring model training.
  • Demonstrates consistent performance across standard GQA, hybrid sliding-window/global attention, and Multi-head Latent Attention (MLA) models.
  • Avoids generation-length inflation and architecture-dependent failures observed in competing sparse attention methods.
long-contextkv-cachellm-inferencearxivsparse-attentionnvidia-b200
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

RedditGandalf extends SSD lifetime by 7.3x for LLM KV cache offloadingRedditVirtual Head Attention Reduces KV Cache by 4x and Improves Quality Over GQAPyTorchKRRed Hat Publishes Performance Comparison Commentary on vLLM vs SGLangPyTorchKRvLLM-Omni Unveils Multimodal Unified Serving ArchitectureGeekNewsgpu-unlock Linux module doubles FP/BF16 instruction rate on Nvidia 3000 series GPUsTLDR AIStreamVibe-VL Improves Proactive Video Interaction by 10.7% Over Baselines